Anyone reviewing the research from recent years will see that chatbots have not improved quickly enough over the past five years to meet customer expectations. They underperform, cause frustration, and human intervention is often required. We will summarize the available research once again.

 

In recent months, chatbots have regularly been in the news. For instance, there was research by both Wiep Hamstra and Jules Ernst, as well as by the National Voice Monitor. Both studies showed that chatbots can be unreliable and a source of frustration.

There were some pitying reactions from the customer contact sector: it is logical that the telephone is the preferred channel when chatbots do not function properly. And 'switching off' poorly functioning chatbots would merely be treating the symptoms.

Meanwhile, consumer program Radar called on consumers to share tips on how they bypass chatbots. Not long after, the Customer Service Federation announced it would conduct research into how The quality of chatbots can be improved.

A few years ago, customer experience expert argued Nienke Bloem for a second generation of chatbots that are relationally designed and place customer interests at the center; bots should, for example, proactively call back when they fail.

Promises

“Faster answers for the customer and fewer calls for the contact center”—that is often the promise from suppliers. With the added bonus: many chatbots improve after every interaction. Despite the poor performance, there are sufficient incentives to continue buying, selling, and deploying this technology: revenue for the former and savings for the latter. In any case, it is 'not the technology's fault' if chatbots fail to live up to these promises.

The improvements must therefore take place on the part of CX specialists and contact center managers. The KSF intends to map out what these improvements should entail through its own research. Anyone who has followed the discussion surrounding underperforming rules-based chatbots can already somewhat predict the direction things are heading in terms of challenges.

Number 1: information management (with a lack of governance, direction, and ownership as problems), number 2: implementation (rollout too fast, insufficient testing, insufficient resources for management). A successful pilot is not always sufficient for scaling up. And number 3: the wrong incentives and expectations.

The annoying thing for the customer is that all three are interconnected. If the contact center wants or needs to score points with cost reduction, there is high pressure to put the chatbot to work right away. After all, with just a little training, the bot can already start handling a number of questions. The downside of this inside-out approach is that the customer or citizen has no idea what the chatbot can do, but does have expectations.

The contact center manager also has expectations, namely that the chatbot improves quickly. But if the implementation team gets something else on their plate again after the first few weeks, the chatbot (which, just like an employee, constantly needs to be updated on the latest information) becomes a hopeless mission.

Drawing lessons from chatbot 1.0

The potential is great: there are many rules-based chatbots that could be replaced by a better AI-based variant. The challenge, then, is to avoid making the same mistakes with the second generation of chatbots as with the first. If the incentives to implement quickly are not removed, Chatbot 2.0 will remain problematic as well. It is therefore not so much the knowledge base and rapid implementation itself, but rather the underlying drivers that need to be in order. Is the goal to achieve better (or possibly unchanged) service delivery at lower costs, or are lower costs the primary goal, and is that allowed to come at the expense of service delivery? The latter is what is currently happening in the municipal sector.

Success factors

That interplay of success factors – information management, implementation pace, and wrong incentives – can also be found in the research of the Smooth Operators Program (from the universities of Utrecht, Tilburg, and VU). The researchers state that organizations often only look at call reduction, while metrics such as call quality en usability be much more relevant.

Seasoned chatbot users understand that just because a chatbot works well today doesn't mean it will work well tomorrow. A chatbot must be constantly updated to remain relevant. KPN, for example, reported that the generative chatbot Dot ensures a call reduction of approximately 10% and handles a third of the initiated sessions independently. However, manual quality checks remain necessary and knowledge provision is a bottleneck, according to KPN.

Self-learning? Forget it.

The idea that chatbots automatically improve is refuted by the facts. Over the past five years, various studies have shown that chatbots fall short. Since 2021, the picture has been fairly consistent: the pace at which chatbots are rolled out is higher than the rate at which performance and satisfaction increase. Although the promise of chatbots is a reduction in customer contact via the telephone or other 'expensive' channels, the negative returns of chatbots remain underexposed. This is therefore something that should be addressed in detailed research: what is the impact on customer experience and on the contact center when chatbots underperform?

In the following two parts, we will discuss research into both the performance of chatbots and customer preferences regarding chatbots.

What does research say about the performance of chatbots?

In a telecom survey, 66% of customers gave the use of chatbots a rating of 1 (on a scale of 1 to 5); a survey by the Dutch Consumers' Association (2021) found that 66% of more than 10.000 participants are dissatisfied with chatbots and only 10% have confidence in the technology.

An international survey of 1.700 consumers, conducted in 2022 by contact center platform UJET, shows that 80% of chatbot users experience more frustration after a conversation. 78% felt compelled to contact an employee after all because the chatbot did not solve the problem, and nearly three-quarters considered the use of a chatbot a 'waste of time'. More than half of the respondents believe that calling an employee leads to a solution fastest.

A 2022 study by Cyara shows that 80 percent of customers feel that chatbots cannot answer simple questions. 76 percent report being forwarded to an employee and then having to explain everything again. 76 percent say that using the chatbot takes too much time. Two-thirds (65 percent) say that the conversation is insufficiently relevant to the actual problem.

In State of Chatbots and Conversational AI In (2023), organized and sponsored by TeamDynamix and published in InformationWeek, IT professionals speak out about the use of chatbots in IT service management (ITSM). Although this involves a small group of respondents (N=98), primarily CIO/CTO or VP IT and helpdesk or ITSM managers, only 5% of respondents indicated that their chat tool resolves more than half of the interactions, but 40% expected to reach this level by 2026. Slightly less than half (48%) believe that the tools do not resolve questions well or misinterpret intent, and 38% consider management time-consuming because the bots are not self-learning. Almost one in three IT departments still has to manually enter intent/response pairs. Respondents cite not understanding the question (61%), incorrect or inaccurate answers (45%), and insufficient understanding of natural language (43%) as the top causes for failed conversations.

In 2023, it appeared from the National Voice Monitor that only 10% of the Dutch trust chatbots (compared to 64% in human employees).

From the follow-up study of the National Voice Monitor 2024 It turned out that only 8% of consumers are enthusiastic about chatbots and that 8% of chatbot questions are answered correctly; only 1% of consumers prefer automated chat. In terms of rating, human conversations (telephone or live chat) ended up with an 8 or higher, while chatbots scored a 5 or lower.

A study of Conclusion Mission Critical (2025) among 1.012 Dutch people shows that 68% believe that automated systems (including chatbots) work well. More than half (51%) prefer a quick answer from a chatbot to a long wait, but 48% are annoyed by poorly functioning bots.

In 2025, Wiep Hamstra and Jules Ernst observe that the 36 municipal chatbots they subjected to a series of questions give incorrect answers or refer to web pages without an answer, even to simple questions – such as opening hours.

In an analysis of more than forty chatbots on B2B websites, the report reported Death by Chatbot (2025) that in 15% of conversations the problem remained unresolved, that 86% of these sessions still had to be handed over to a human, and that on average only four to six messages were exchanged before users became frustrated. More than half of the users considered the bots unreliable, according to a report by chatbot platform Breakout AI.

In 2026, the study by Hamstra and Ernst was repeated, and it turned out that the performance of municipal chatbots had deteriorated rather than improved.

The developers of Gem subsequently acknowledged that chatbot Gem makes mistakes, but emphasized that it still answers approximately 62% of the questions. And finally: the results of the latest edition of the National Voice Monitor (2026) show that the chatbot as a contact channel has not improved: consumers receive a usable answer in only 12% of cases.

 

What does research say about customer preferences and experiences with chatbots?

Research by the National Voice Monitor 2024 Among Dutch consumers, it appears that the telephone is still the preferred channel; poorly functioning chatbots (40%) are among the top three frustrations. Chatbots are considered useful for simple questions but are perceived as impersonal.

Of 5.728 surveyed American consumers, 64% indicate they prefer not to use chatbots or AI for customer service, according to Gartner Research (2024).

Civic Science (2024) showed that 45% of American adults find chatbots unpleasant; only 19% have a positive view. 43% of American adults prefer calling, while only 17% prefer using chat or chatbots. Of the respondents who used a chatbot, 44% found it useful; in 2022, that was 34%.

According to a survey of 600 American consumers conducted by Acquire AI (2024) consumers are 2,5 times more positive about a conversation with a human (82% positive) than with a bot (33% positive); about 5% prefer bots over humans. And a study by SurveyMonkey (2025) found that four out of five American consumers (79%) prefer human interaction over an AI agent.

Conclusions

After five years of 'chatbots in customer contact,' the picture is consistent: performance is barely, if at all, keeping pace with expectations. The problems surrounding chatbots do not lie primarily with the technology, but in the way organizations deploy them. The pressure to implement quickly and save costs has led to structurally underperforming solutions that frustrate customers and actually place a heavier burden on the telephone channel.

The transition to generative AI entails the same risks. Anyone who, with chatbot 2.0, once again chooses speed over quality, considers poor information management sufficient, and measures success in call reduction rather than customer experience, will end up in a similar overview to this one in five years.

The question for contact center managers and CX directors is not whether to switch to AI agents, but whether they want to be honest in advance about the primary goal of second-generation chatbots: is the goal better service, or cheaper service?

(Ziptone/Erik Bouwer)

Customer Experience, Featured