Most people in IT know that the role of the data scientist is growing in importance and many will know that the result of this demand means there is a shortage of people with the necessary skills to do the job. Search for the term “data scientist” and LinkedIn will return over 287,000 responses; data scientists are very much in demand and their services do not come cheaply. Six-figure salaries are mainstream.
First, let me try to define my terms, because data science is a slippery term that sloppy thinkers use interchangeably with other fashionable phrases. For me, it’s to be located at the intersection of mathematics, statistics, business domain knowledge, programming and research. It’s an attempt to combine the power of the technologist with the experience of the line-of-business person via the ability to mine data and apply statistical discipline and software engineering. Combine all this, and you have a chance to detect underlying patterns and see tease out meanings…you can see why these people are relatively scarce in number.
But if we accept the conventional wisdom that use of data can be a hugely powerful weapon – and we should – what can we do to bridge the gap between demand and supply of people with this cocktail of skills? The answer lies in reskilling and a programme of education and recruitment to create a new generation of data scientists from the ranks of developers, analysts, engineers and business users. Just as at times of national crisis the government could call on conscription, companies are recruiting armies of citizen data scientists.
Of course, they don’t have all the know-how listed in my attempt to define data science, but citizen data scientists are emerging as a pragmatic response to the need for better use of data. These people often are employees who may not have been formally trained in mathematics or statistical analysis, but they are smart, good at communicating and possess a strong understanding of business challenges and needs. They should also be excited by the possibility of changing their organisations and learning to become better data scientists as they go along.
At its most sophisticated, data science can be a daunting task that involves complex algorithms and programming knowledge, but advances in AI, predictive analytics and cloud computing are democratising the discipline. In the age of Big Data and cloud platforms such as AWS and Microsoft Azure, every organisation has access to huge data sets and the compute capacity to query them. And the citizen data scientist who has deep knowledge of the company’s culture, market and profitability drivers might be in a better position than the specialist data analyst with only very limited knowledge of the employer’s gears and levers.
Paving the way
But how best to encourage and assist the citizen data scientist? It’s certainly the case that companies with an open culture of information sharing will be best positioned. They should also be equipped with analysis tools that mask underlying complexity and have user-friendly front-ends with wizards and templates. Thankfully, the rise of assistants such as Alexa and Siri are driving more conversational user interfaces that hide complex calculations and encourage users to ask direct business and operational questions. Similarly, software development is increasingly dependent on ‘low code’ platforms and visual drag-and-drop user interfaces. The user should have knowledge of data management and data structures and dependencies, but not necessarily programming.
This means that the citizen data scientists don’t need to call on the IT department at every turn, but can quickly create, test and deploy applications. And this in turn means that analyses can quickly be delivered to business decision makers and feedback accommodated. This again fits in with the zeitgeist of our times when rapid application development, agile and scrum methodologies are increasingly favoured over laborious, time-consuming waterfall approaches.
As organisations we all need to do better with data, corralling it, probing it, testing suppositions and seeking the insights that provide scope for competitive differentiation. Citizen data scientists provide a way to span the chasm between what we want to do and the shortfall in supply that lets us do it. If you feel that you could benefit from better use of the data, but struggle to hire experts, consider creating an internal programme and join the clarion call for citizen data scientists.
