When you ask a contact center manager about the results achieved with the chatbot, you will often be the first to hear what percentage of the conversations are handled by the chatbot. Less often heard: customer satisfaction has gone up or the chatbot scores well on the customer effort score. What are useful KPIs for chatbots?

 

An inventory of possible KPIs for chatbots shows that KPIs are predominantly transactional in nature. This makes the KPIs for chatbots similar to the KPIs that are often used for customer contact employees. When looking at outcomes at all, it often concerns realized cost savings – something that is rarely part of an organization's mission, vision and strategy.

When is a KPI a good KPI? – KPIs (Key Performance Indicators) are measurement results of variables that provide insight into the performance of processes. These measurements are not an end in themselves, they ultimately provide insight into the performance of the organization. The best KPIs align with the company's mission, vision and strategy. They must be 'actionable', which means that the KPI has a process that you can intervene on. And: a good KPI is simple and easy to understand. Here you will find more articles about KPIs in customer contact.

 

Transactional or outcome-related?

Anyone who measures the performance of a chatbot on the basis of 'hard', transactional KPIs may ignore the role or function of the chatbot within the customer journey. What is the added value? Is the chatbot a valuable addition to the user if self-service falls short or an equivalent alternative to another contact channel?

In addition, for measuring performance it is relevant to keep in mind where the chatbot is activated (for example on the general page of customer service or on specific pages about a certain topic). This can contribute to realistic expectations among customers regarding the capabilities of the chatbot, but can also provide more insight into performance.

Finally, chatbots are increasingly being used to support agents or counter staff. Also use specific KPIs for chatbots in this area (employee resolution percentage, information targeting, knowledge level of the chatbot). From this perspective you can consider the chatbot as an IT tool; interest in measuring IT user experience among IT end users is growing.

Ziptone presents a list of 26 chatbot KPIs that give you insight into the effectiveness of chatbots in customer service.

Technical KPIs

1. Availability or (technical) downtime: in what percentage of the time is the chatbot function (not) available to customers.
2. Response time: how fast the chatbot responds to user input.
3. Session duration: the average time users spend interacting with the chatbot.
4. resource load: the amount of computing power used by the chatbot.
5. Error rate: the number of errors or misunderstood commands compared to the total number of interactions.

KPIs related to usage

6. Active users: The number of unique simultaneous chat sessions within a given time period
7. New users: the number of new users who started using the chatbot within a certain time period
8. Average number of chat sessions: The average number of chat sessions over a period of time
9. Total number of sessions: the number of chat sessions started and completed within a certain time period.
10. Engagement score: how often and for how long do users interact with the chatbot.
11. Return rate: number of users returning for a new session with the chatbot.

KPIs related to users and customer experience

12. Channel change: which part of the chatbot users asks for a live chat session even before a substantive question has been asked.
13. Escalation rate: which part of the interactions should be handed over to a human agent because the chatbot is unable to solve the problem.
14. Handover time: how long did the customer have to wait for a live agent?
15. Successful Handovers: how many handovers to a live agent have been successfully handled by the agent?
16. Handover abandonment: how many times has a customer left the chat while waiting for a live agent?
17. Abandoned Conversations: how many dialogues were unilaterally abandoned by the customer?
18. Context retention: to what extent is a chatbot able to effectively use provided customer information?
19. Concept recognition: how often must the chatbot indicate that it does not understand the customer's question?
20. Intent recognition success: in what percentage of the conversations did the intent recognition go right the first time?
21. Conversation efficiency: how many messages are needed from the chatbot or from the customer to solve the problem.

Outcome-related KPIs

22. First Contact Resolution (FCR): which part of the questions or problems can the chatbot solve independently in the first chat session.
23. Completion rate: how many sessions are completed successfully compared to the number of sessions started.
24. Conversion rate: how many customers proceed to a certain (intended) transaction after interacting with the chatbot – for example a successful referral to a specific self-service solution
25. User Satisfaction: the customer's opinion of the chatbot's performance. This can be asked in a satisfaction score or via the NPS or CES.
26. Realized cost savings: how much cost was saved by not handling the question in question via an agent, but via a chatbot?

The following formula is used to calculate this last KPI:
Savings = (number of questions answered correctly * average agent response time * average agent hourly rate) – (bot creation cost + chatbot subscription cost). A negative outcome means that the chatbot can independently handle too few questions completely and correctly.

 

This article was written in collaboration with Steve Elshout (Huxam).

Customer Experience, Featured, Knowledge base