AI skills shortage is top challenge for UK IT leaders in 2024

A skills shortage has been identified as the foremost challenge facing IT leaders in the implementation of artificial intelligence (AI) in 2024, according to a recent study by data streaming company Confluent. The study surveyed over 500 UK IT decision-makers and highlighted various obstacles that these leaders anticipate in their AI adoption processes.

The research found that 68% of IT leaders cited "insufficient skills and expertise" as the primary hindrance in rolling out AI initiatives. Over a quarter (28%) consider this skills shortage a "major" challenge in the upcoming year. This shortage of skilled personnel surpasses other notable concerns, such as the inability to integrate data and fragmented systems, which 66% of respondents flagged as significant issues.

Other challenges reported by the respondents include inadequate infrastructure for real-time data processing, with 65% of participants highlighting this as a major roadblock. Additionally, 62% cited poor data timeliness and quality as a critical issue. These challenges collectively underscore the complexities and evolving nature of AI technology that businesses must navigate.

Such challenges come at a time when the demand for Generative AI and similar technologies is expected to surge. The research indicates that 79% of IT leaders foresee growth in demand for these technologies over the next two years. Consequently, many IT departments are overhauling their data strategies, leaning towards real-time processing methods to keep pace with this increase in demand.

Over two-thirds (68%) of surveyed IT leaders noted that incorporating real-time data streams into AI and machine learning (ML) strategies will be crucial in the coming year. Furthermore, 29% of respondents identified this as their "top priority" for the near future. Real-time data processing is regarded as essential for facilitating the agility needed to adapt to evolving business requirements and for supporting AI models with accurate data.

Richard Jones, Vice President for EMEA at Confluent, commented on the findings, stating, "AI has been a marketing buzzword and not a technological reality for most companies — until now. But as AI becomes mainstream for businesses, many find they are still in the early stages of AI implementation." He further explained that the shortage of AI skills and the lack of infrastructure for real-time data processing are significant barriers for businesses dealing with complex and rapidly changing technologies.

Jones emphasised the importance of accurate, real-time data streams for AI models, saying, "With data streaming, IT teams can be incredibly agile and adapt in-line with evolving business demands. The adoption of AI can be accelerated by enabling the rapid scaling and deployment of real-time AI applications, while also tracking the data being inputted into AI models — which is fundamental for regulation."

As IT leaders look to tackle these challenges, the study suggests a shift towards advanced real-time data strategies could be pivotal. This approach might not only surmount the identified obstacles but also enhance the efficiency and accuracy of AI implementations, thus supporting broader business objectives and regulatory compliance.