AI is far from meeting expectations

The rise of generative AI raises many questions about its effect on our economy and labor market. All sorts of stories circulate about huge productivity gains and job replacement. But research, for now, shows it is very far from meeting those expectations.

That does not stop organisations from investing more and more in artificial intelligence (AI) and haphazardly bringing in AI tools whose value is not yet clear. After all, what problem are they actually trying to solve with this AI technology?

Overblown expectations

The study The Pulse of Work in 2025: Trends, Truths, and the Practicality of AI by Workplace Intelligence commissioned by GoTo takes a dim view of these overblown expectations. The United States would miss out on USD 2.9 trillion in efficiency gains because it is not yet adequately deploying AI.

That calculation is based on the estimates of employees who were asked how much of their work - in hours per day - could be taken over by AI. This is an invalid question whose outcome is extrapolated to the entire U.S. labor market with great ease. We should be sceptical about this.

Still few gains in productivity

In Large Language Models, Small Labor Market Effects Anders Humlum and Emilie Vestergaard describe their research on the labor market effects of AI chatbots among 25,000 workers in 7,000 workplaces

This study shows that - despite significant investment - the economic effects remain minimal. AI chatbots had no significant effect on earnings or recorded hours in any occupation. Modest productivity gains (average time savings of 3%), combined with weak wage pass-through, help explain these limited labor market effects. The findings of this study call into question narratives of an imminent transformation of the labor market due to generative AI.

The first regrets are already there

And that has also been the experience of the CEOs of Klarna and Duolingo among others, who introduced AI into their organisations with much fanfare and drove hordes of employees out the door beforehand. They currently have only one concern: how to get those people back.

The Klarna CEO was very energetic in the beginning, but later came back to that: “I made a mistake.”

Read - and listen - to the story Duolingo's $7B AI Disaster: Enterprise Lessons for AI Implementation by Magnus Hedemark. The article tells how Klarna could have handled it better.

Mythical Man Hour

Anders and Emilie's research shows that while AI tools, such as ChatGPT, can boost productivity, these gains are often marginal. For example, workers can generate concepts or look up information more quickly. However, the researchers caution that the effect on overall productivity may be small.

This aligns with the concept of the "mythical man hour" - the idea that technology always leads to significant time savings that can be freed up for other tasks. In practice, however, we often see time gains in one area being filled with new, less efficient tasks or work becoming more complex, negating the gains.

AI with people, an ideal combination

In their own AI research, Gartner predicts that 50% of organisations will abandon their plans by 2027 to reduce customer service headcount by deploying AI. They see that human employees are and will remain an essential part of customer service strategies.

Gartner's survey of 163 leaders, found that 95% of customer service leaders plan to retain human employees to strategically define the role of AI. This approach ensures a “digital first, but not digital only” strategy, avoiding the pitfalls of a hasty transition to an agentless model.

"While AI offers significant potential to transform customer service, it is not a panacea. The human touch remains irreplaceable in many interactions, and organisations must balance technology with human empathy and understanding," said Kathy Ross, Senior Director Analyst at Gartner Customer Service & Support. "A hybrid approach, where AI and human employees work together, is the most effective strategy for delivering exceptional customer experiences."

Create a digital strategy

Organisations must clearly define the role of AI, prioritise strategic objectives and define the roles of human employees. This strategic alignment is essential to improve customer service without sacrificing quality, so that AI complements rather than replaces human interaction.

Of course, there can be value in deploying AI. But as with any other digital technology, that tool must solve a problem. Otherwise, it is at best “occupational therapy” and at worst a financial failure.

Use the strategy game

Therefore, create a digital strategy derived from the business strategy.

What problem needs to be solved, how will the AI solve it, when will it be successful and how will we measure it?

Use the TIMAF strategy game to create such a digital strategy. This will help you get a grip on your digital technology, including AI.