The AI Skills Gap and How to Address It

To many workers, artificial intelligence is a faceless terror, lurking in the digital borderlands and just waiting for the chance to replace them with fast, cheap algorithmic labor. While Dell’s apocalyptic prediction that 85% of the job roles that will be available by 2030 do not even exist yet has been discredited, the workplace is certainly in a period of rapid change. We may not be looking at an entirely new employment landscape in the next decade, but workers’ inability to leverage burgeoning AI technology and work alongside it is a real concern.  

A 2019 survey of IT decision makers found that 93% were fully invested in AI programs. Some 70% of decision makers in the IT industry itself were working on AI projects. Yet workers who possess the skills to implement these programs -- such as deep learning and natural language processing -- are in short supply. A 2024 survey indicates that 81% of IT professionals think that they can use AI, but only 12% actually have the skills to do so. And 70% of workers likely need to upgrade their AI skills. 

Reuters reports that this year there will be a 50% hiring gap for AI-related positions. Some 60% of IT decision makers think AI constitutes their largest skills shortage. A 2022 Deloitte survey indicates that there are a mere 22,000 AI specialists globally. Equinix found that 62% of IT decision makers view these types of shortages as a major business threat. And a Redhat survey of IT leaders found that 72% thought AI skill gaps needed to be urgently addressed. 

Related:What You Need to Know about AI as a Service

Even workers dealing with less technical aspects of AI utilization -- who are actually in higher demand -- have found themselves struggling to integrate AI into their workflows. Only 27% of UK business leaders think their non-technical employees are capable of effectively using new technology. 

The Deloitte report also claims that a third of organizations are looking at skills gaps but only 17% were actively trying to address them. This is a problem for both new entrants into the workforce and veteran employees. A survey by Randstad found that half of Gen Z workers might quit if not offered on-the-job training in this area. And 82% of workers saw lack of training as a potential reason for switching jobs, according to a report by Skillsoft. 

Identifying mechanical and repetitive tasks that AI is suited to will likely be the most parsimonious approach in the near-term. Rather than replacing workers, it will allow them to spend their time on other work that requires a greater degree of ingenuity and creative thinking. But identifying the skills it will take to actually integrate AI into daily tasks -- rather than simply outsourcing them to the machines and forgetting about them -- are another issue entirely. The literature suggests that we are some distance from actually articulating how employees can effectively collaborate with their AI coworkers. 

Related:Is an AI Bubble Inevitable?

Here, InformationWeek investigates the complexities of the AI skills gap, with insights from Florin Rotar, chief AI officer at cloud and AI service provider Avanade; Apratim Purakayastha, chief product and technology officer at digital learning company Skillsoft; Greg Benson, chief scientist at generative integration provider SnapLogic; Bob Audet, partner in the data and AI solutions practice at digital consulting firm Guidehouse; and Bassel Haidar, leader of Guidehouse’s artificial intelligence and machine learning practice. 

Fears of Job Replacement 

As AI insinuates itself into most aspects of the digital world, workers have expressed concern that it will replace their roles and put them out of a job. Interviewees in one study worried that “it could cause unemployment” and that it “could do someone out of a job.” Another subject fretted that AI “takes the human element out of a very human activity.” 

Related:Is Innovation Outpacing Responsible AI?

In 2018, the Organization for Economic Cooperation and Development (OECD) predicted that 14% of jobs were at risk of replacement by AI within 15-20 years. These predictions have fed workers’ fears -- the Randstad survey found that 37% of workers were worried about how AI would impact their careers. Still, 42% were encouraged by the prospect of change. 

Despite reassurances that training will allow them to collaborate with AI technology, a 2023 survey of nearly 13,000 employees by Boston Consulting Group found that while 86% needed AI training, only 14% were receiving it. Over 60% of executives believed that they would need to replace or retrain a quarter of their workforce in 2019. 

“People genuinely need to feel that AI is done with them and not to them,” Rotar exhorts. “And it is not just a way of getting people to work harder or automate their jobs away. When you have trust that AI is used to empower people to become a better version of themselves and to thrive in their career, you get adoption at scale.” 

Still, these changes are happening far more rapidly than in other periods of revolution, during which older workers could deploy their skills until retirement and then be replaced by a new generation versed in the technologies facilitating the change. Now, certain manual labor jobs such as factory work may be completely replaced by AI-driven technologies, with far less room for workers to adapt. Some may indeed need to retrain entirely and pursue new lines of work. 

What Skills Are Missing? 

For all the understandable hand-flapping over how AI will affect both employers and employees, the precise skills that are lacking are strangely absent from the literature on the subject.  

Part of this is due to the extreme specificity of AI applications across industries. And part of it is likely attributable to the speed at which the technology is advancing. Everyone is scrambling to leverage it, and few have assessed what exactly will be needed from human employees. But it’s certainly on the radar: 43% of tech leaders rate the AI skills of their underlings as low per Skillsoft’s report. One-third rate finding employees skilled in AI as one of their main hiring challenges. 

One rough breakdown indicates that there are a number of fundamental skills, such as numeracy and literacy, that are a barrier of entry to AI-assisted jobs. Beyond that there are skills required by professions that use AI and then the skills required of those who will either implement AI in broader digital ecosystems or design new AI technologies themselves. 

“We’re seeing the greatest skills gaps in areas like Generative AI (e.g. ChatGPT), predictive analytics, natural language processing, and large language models. The impact of AI is deep and broad,” Purakayastha says. 

“In my recent experience both in academia and in industry, the two biggest gaps are an open mindset toward large language model (LLM) use and the understanding of tooling that can automate the use of LLMs for specific tasks,” Benson adds. 

While some of the skills necessary to effectively deploy AI have been identified in a general sense, how they will be used in practice is uncertain. This has been framed as the tension between point-based skills and domain integration. For example, skills in quantum computing, statistical analysis and reinforcement learning are crucial to AI implementation. But how can they be most effectively deployed in industries with highly specific needs? Everything from regulatory compliance to security to data analysis intersects with AI technology.  

“There is also a critical shortage of technical skills required to develop, implement, and maintain AI systems, such as machine learning, data science, and programming. This skill gap is particularly pronounced in industries like healthcare, education, and government, which traditionally rely more on manual processes and are now facing a steep learning curve with the integration of AI,” Audet relates. 

According to a Salesforce survey from last year, 62% of workers believe they don’t have those skills yet, whatever they may be. And 70% of their bosses don’t think they can yet use them safely.  

Executives surveyed in an IBM report think that 40% of their workers will need to reskill. More than half of business leaders don’t think they have enough AI talent on staff according to SnapLogic. The lack of a defined taxonomy of essential skills and proficiencies further amplifies the problem. Soliciting job applicants when you cannot even accurately define the role makes filling these gaps nearly impossible in some cases. 

“Periodic training needs assessments (TNAs) can provide insight into gaps that may exist. Additionally, due to the rapid pace of technological advancement, training interventions should be continuously monitored and adjusted for both content and to maximize learner retention. Training should be tied to KPIs and linked to desired outcomes where possible,” Haidar says. 

A paper from the Oxford Internet Institute suggests that analyzing the content of job ads may actually be helpful in determining which skills are in demand and how well they are matching to job seekers. By identifying skills that don’t tend to appear in the same type of employee, the actual skills gap can be narrowed. This information, the authors suggest, can also be fed back into the education system, directing coursework and producing graduates that might fill these gaps in a matter of years. 

In addition to developing skills that allow for collaboration with AI, it has been pointed out that employees also need to further cultivate skills that distinguish them from AI -- critical thinking, social and management skills. Still, they need to remain attentive to how AI intersects with their other responsibilities. 

“While jobs that require complex decision-making and emotional intelligence are less prone to automation, professionals in these roles still need to learn how to integrate AI into their work effectively. This includes using AI-generated insights for better decision-making and understanding the ethical implications of AI,” Audet says.  

And AI programs themselves facilitate the development of certain capabilities -- time management and scheduling, for example. Feedback on how time was spent and how it might be spent more efficiently in the future may be an effective, neutral mechanism helping to structure the workday. These skills are transversal -- generalizable across industries -- giving some workers a wider range of potential career possibilities.  

“While some roles are adjusting to this new way of working more than others -- such as individuals in customer service or software development -- nearly every professional needs to adapt and gain new skills,” Purakayastha says. “Examples include marketing, product management, HR, finance, legal, and many other frontline departments. AI presents a significant opportunity for improving job efficiency and effectiveness.” 

Leveraging these skills may be impeded by siloed work environments that fail to emphasize collaboration and condensing of necessary tasks across departments and by lack of available time for upskilling due to high workloads. The rapid speed at which AI programs are developing makes it even harder to ensure that employees are able to utilize the technology. 

Rotar sees a more substantial gap that needs to be filled as well. “The main gap is actually trust, not skills,” he thinks. “An analogy would be the internet and web browsers in the 90’s. Yes, some internet education was needed for some people in the early days to ‘dial up’. And people tried to make a big deal out of ‘training how to use Netscape’. But in the end, it is more about positive curiosity, good intent, and value.” 

AI Training Programs 

Primary educational institutions and universities are stepping up their game on developing programs that integrate AI utilization. So, too, many organizations are now emphasizing AI training, channeling employees toward upskilling that will aid in more deeply integrating machine learning into their skill sets and the procedures that propel everyday activities.  

“Recent graduates, particularly from STEM fields, often have a more up-to-date theoretical understanding of AI concepts and tools compared to longtime workers,” Haidar explains. “However, they may lack the practical experience and industry-specific knowledge that longtime workers have accumulated over the years. Longtime workers have a deeper understanding of the challenges within their domains, which is invaluable in applying AI solutions effectively. They have also developed crucial soft skills, such as communication and problem-solving. However, they may be more resistant to change and less familiar with the latest AI technologies compared to recent.” 

Observers have strongly indicated that AI needs to be more deeply integrated into the curriculum for younger students. The UK’s AI Council has suggested that the formation of an online academy may assist teachers in developing curriculum and facilitating lifelong technological learning. The National Centre for Computing Education is already addressing some of these issues. 

At the university level, AI and machine learning courses are increasingly in demand. Between 2012 and 2018, enrollment in AI courses increased by five times and enrollment in machine learning courses increased by 12 times according to an AI Index report from 2019. Snaplogic’s report also found that 49% of companies are actively recruiting AI talent from universities. But recruiting and hiring are time-consuming, expensive -- and far from a sure bet. It is nearly impossible for universities to develop curricula that keep up with the current pace of change. Some 75% of decision makers would prefer to upskill rather than recruit new talent according to SAS. And another paper found that 82% of executives at large companies believed that at least half of the gap needed to be closed by upskilling rather than hiring. 

The research on upskilling rates is conflicted. Some have suggested that upskilling efforts are lagging -- Randstad found that only one in 10 employees had been offered AI upskilling. However, research commissioned from Vanson Bourne by Snaplogic has found that 68% of organizations are investing in upskilling and reskilling -- a far more encouraging figure. Per research by Skillsoft, 45% of IT professionals did not see training as beneficial in 2022, but by 2023, only 15% saw no benefit. This aligns with observations that most AI training to date has been reactionary rather than proactive. 

Boston Consulting Group found that some companies are allocating as much as 1.5% of their budgets to these initiatives. This is a crucial development given the fact that 72% of IT professionals surveyed by Pluralsight indicated that their companies often take on new technology without understanding whether or not their staff will be able to effectively use it.  

“One needs to be awake and have their critical thinking switched on,” Rotar suggests. “One way of doing this is with company-wide schools of AI.” 

Users need to be tested, he continues. “Introduce an error on purpose once in a while. If the user does not catch that, they get a slap on the wrist just like one gets when you click a phishing email test attachment. Do that a few times, and you lose your GenAI privileges until you’ve retaken the training and get certified again.” 

Organizations are quickly beginning to realize that reskilling is not only a more efficient approach to dealing with skill shortages. It is also key to attracting and retaining talent that will stay and grow with the company. These training programs can be both internal and tailored to the demands of the company and external and aimed at developing more general skills. The average half-life of job skills is believed to be around five years, so this approach is beneficial to both workers and the organizations they serve. 

The most effective approaches include not only the raw skills themselves but strategies for dealing with the challenges that will inevitably arise -- such as the likelihood of AI hallucinations generated by machine learning programs. So, too, training programs need to address potential ethical and legal issues, such as entering confidential or proprietary data into unprotected tools.  

“Holding internal GenAI workshops, in which individuals and teams showcase LLM use cases is a great way to cultivate the GenAI mindset,” Benson says. 

Training needs will differ substantially across job roles. Some jobs may need only brief modules that teach them to create efficiencies in their non-tech roles. IT workers may require more intensive learning that enables them to safely deploy AI in programming and cybersecurity. Identifying employees’ existing skills and then deciding how best to augment them is the first step toward developing a more effective workforce.  

“A role-based, curated approach to upskilling has proven more effective than offering a large catalog of content or sending people to classes,” Purakayastha claims. “We observe that customers managing this change successfully are rolling out programs that have a few levels. A foundational level for every employee; then a layered approach for every major role family.” 

“Once organizations identify the relevant personas and desired skills for each, they should determine which gaps exist for their personas to appropriately address them,” Audet adds. “It is important that the skills needed align to further organizational priorities. There is not one ‘right way’ to close skills gaps. However, some modalities and strategies are more effective than others.” 

Companies may keep internal lists of desirable skills or rely on those developed by outside organizations, such as the World Economic Forum. Then, employers must figure out how to implement them, either by redesigning existing roles or designing entirely new ones, depending upon business needs.  

While recruiting newly trained graduates will undoubtedly help fill some of the gap in the next several years, organizations can use some of the same institutions that produce these graduates to reskill their existing employees. For example, the Institutes of Technology in the UK have partnered with industry to offer reskilling programs. And the National Retraining Scheme may offer assistance to those without degrees in the UK. 

“To execute an upskilling program, look for training partners that offer benchmarks and assessments, strong quality of engaging content, opportunities for hands-on practice, and innovative experiential, interactive training methods to keep employees engaged, motivated, and at the forefront of emerging technologies like GenAI,” Purakayastha says. 

One of the base-line skills that has proven useful across all domains is how to properly engineer a prompt -- that is, how to query an AI program so that it spits out the most useful result. While AI programs are increasingly intuitive, knowing how their logic works can help workers to tailor their utilization so that they do not have to fuss with the products of their queries once they arrive. Templates and libraries of effective prompts may be helpful as employees begin to deploy their training. 

“A prompt can carry sophisticated transformations on arbitrary inputs. All you have to do is ask for what you want in clearly articulated statements with possibly some examples of what you want to achieve. On top of this, you can also use LLMs to help you understand how you should create your prompts,” Benson explains. “Once you understand that you can effectively program with prompts, the next step is to make these prompts work in a repeatable way. Here is where GenAI tooling comes into play. While we are still in the early stages of making prompt automation generally available to non-programmers, knowledge workers will increasingly need to not only upskill in terms of combining data with prompts but also learn how to deploy these new types of programs to be reusable on new data from a variety of data sources.” 

Examples of Effective AI Training Implementation 

Companies have begun sharing their AI training success stories -- from which useful lessons may be drawn. 

“Combining online learning with hands-on workshops and real-world project involvement has proven to be particularly effective. This approach allows learners to engage with theoretical knowledge through online courses and apply this knowledge practically,” Haidar says. 

Some companies are slotting time for employees to simply play around with AI technology and see if it can assist them in their roles. These “AI playgrounds” allow employees to experiment with AI in a controlled context that allows them to make mistakes and learn from them without impacting operations.  

“Combine self-directed learning with structured training, mentorship, and practical application opportunities,” Haidar suggests. 

Amazon’s Machine Learning University has provided upskilling to thousands of employees on AI topics -- and has now made many of the courses available to the public, some for free. The courses include instructional videos and labs -- and the student sets the pace of instruction. Its efforts are not purely digital. The company has also offered mentorship and coaching programs that pair more experienced employees with those who are working on new skills to aid them in deploying their newfound abilities effectively. 

“Peer-to-peer learning sessions can also encourage a more informal, continuous learning environment, helping demystify AI technology and fostering a collaborative learning culture,” Haidar concurs. 

Plenty of other large organizations are eager to get their employees up to speed on AI. Ericsson has partnered with educational institutions such as Concordia University to deliver a suite of AI training programs to its employees. Its initiatives have resulted in some 15,000 employees learning new, AI-related skills. Johnson & Johnson has developed a platform called J&J Learn that serves as an immersion program that tailors AI skills to the company’s projected operational needs.  

And of course, Google has rolled out a range of AI training programs, both internal and external. Its Grow with Google program has helped some 12 million people in Europe develop new digital skills, including AI. And the company’s AI Opportunity Initiative for Europe aims to build on that with programs for disadvantaged people and for startups. 

The EU itself is attempting to head off potential conflicts between human work and AI with its own Up-Skill program. It emphasizes the identification of skills that will be required for workers to collaborate and leverage AI through intensive study of their interactions across industries. In examining where these collaborations succeed and fail, the project aims to develop a set of best practices for AI integration and skill development. 

Cultivating these practices as AI forges ahead will be key to leveraging the talents of both human and digital workers.