The role was relatively new at the time, but as more companies attempted to make sense of big data, they realized they needed people who could combine programming, analytics, and experimentation skills. At the time, that demand was largely restricted to the San Francisco Bay Area and a few other coastal cities. Startups and tech firms in those areas seemed to want all the data scientists they could hire. We felt that the need would expand as mainstream companies embraced both business analytics and new forms and volumes of data.
At the time, we defined the data scientist as “a high-ranking professional with the training and curiosity to make discoveries in the world of big data.” Companies were beginning to analyze voluminous and less-structured data like online clickstreams, social media, and images and speech. Because there wasn’t yet a well-defined career path for people who could program with and analyze such data, data scientists had diverse educational backgrounds. The most common qualification in our informal survey of 35 data scientists at the time was a PhD in experimental physics, but we also found astronomers, psychologists, and meteorologists. Most had PhDs in some scientific field, were exceptional at math, and knew how to code. Given the absence of tools and processes at the time to perform their roles, they were also good at experimentation and invention. It’s not that a science PhD was really required to do the work, but rather that these individuals had the rare ability to unlock the potential of data, wading through complex, messy data sets and building recommendation algorithms.
A decade later, the job is more in demand than ever with employers and recruiters. AI is increasingly popular in business, and companies of all sizes and locations feel they need data scientists to develop AI models. By 2019, postings for data scientists on Indeed had risen by 256%, and the U.S. Bureau of Labor Statistics, predicts data science will see more growth than almost any other field between now and 2029. The sought-after job is generally paid quite well; the median salary for an experienced data scientist in California is approaching $200,000.
Many of the same headaches remain, too. In our research for the original article, many data scientists noted that they spend much of their time cleaning and wrangling data, and that is still the case despite a few advances in using AI itself for data management improvements. In addition, many organizations don’t have data-driven cultures and don’t take advantage of the insights provided by data scientists. Being hired and paid well doesn’t mean that data scientists will be able to make a difference for their employers. As a result, many are frustrated, leading to high turnover.
Even so, the job has changed — in both large and small ways. It’s become better institutionalized, its scope has been redefined, the technology it relies on has made huge strides, and the importance of non-technical expertise, such as ethics and change management, has grown. The many executives who recognize that data science is important to their businesses now need to create and oversee diverse data science teams rather than searching for data scientist unicorns. They can also begin to think about democratizing data science — still with the aid of data scientists, however.
Data Scientists in Relation to Other Roles
The data science role is also now supplemented with a variety of other jobs. The assumption in 2012 was that data scientists could do all required tasks in a data science application — from conceptualizing the use case, to interfacing with business and technology stakeholders, to developing the algorithm and deploying it into production. Now, however, there has been a proliferation of related jobs to handle many of those tasks, including machine learning engineer, data engineer, AI specialist, analytics and AI translators, and data oriented product managers. LinkedIn reported some of these jobs as being more popular than data scientists in its “Jobs on the Rise” reports for 2021 and 2022 for the U.S.
Part of the proliferation is due to the fact that no single job incumbent can possess all the skills needed to successfully deploy a complex AI or analytics system. There is an increasing recognition that many algorithms are never deployed, which has led many organizations to try to improve deployment rates. Additionally, the challenges of managing increased data systems and technologies have resulted in a more complex technical environment. There have been some attempts at certification of data scientists and related jobs, but these are not yet widely sought or recognized. Some companies, like TD Bank, have developed classification structures for the many data science-related careers and skills, but these are not common enough in organizations.
As a result of this proliferation of skills, companies need to identify all of the different roles required to effectively deploy data science models in their businesses, and ensure that they are present and collaborating on teams.
No comments:
Post a Comment