Ever-improving algorithms and a growing mountain of data: the ideal learning environment for artificial intelligence. What happens if systems use speech recognition to continuously listen in on what customers and employees are saying to each other? Part 2 of a conversation with Arjan van Hessen (Telecats/University of Twente) about the use of smart technology in customer contact, in which we also take a look at the future.

Frequently asked question in the world of customer contact: will computers take over human tasks? “Computers can analyse, increasingly predict and in the future will provide advice,” says Arjan van Hessen (Head of Imagination, Telecats/University of Twente). “But first there will be a period in which computers will increasingly support people. “Artificial Intelligence (AI) is a process in which the computer 'learns' to link input to output. A well-known example is learning to recognize pictures with cats.” That recognition is about classifying information; the goal is to arrive at an outcome that we as humans can do something with. Van Hessen explains: “Even when artificial intelligence advises us, it is humans who determine the context. Think of AI in customer contact: is it the intention to advise as many customers as possible or to advise customers as well as possible? Determining that end goal also has consequences for earlier stages in that process.”

Artificial Intelligence: Humans determine the target

AI solutions gain applicability if they are able to learn. For this, an AI solution needs a large amount of material to work with. “By keeping track of each result to see whether it was right or wrong and, in case of errors, also storing what the correct result should have been, a training corpus can be created. This allows the system to be retrained, so that the chance of errors is smaller the next time. By keeping track of when a training set was made and assigning more weight to newer sets, systems can also respond to changing circumstances.”

Because enormous amounts of information are continuously generated in customer contact, customer contact lends itself perfectly to AI solutions. Speech recognition can be a means of converting the input – the speech – into text. So-called emotional 'markers' can be added to this, such as pitch, volume and speaking speed. This output from the speech recognizer is then used as the input for the AI ​​engine, usually together with as many other parameters as possible (time, date, caller properties).

A learning system

Customer questions and comments can be analyzed with speech recognition, but the results of that process can also be used by an AI application, for example by coming up with an automatically generated answer. Again, the designers of the AI ​​system will have to define in advance what the desired output is. Van Hessen: “A goal could be, for example, to provide callers with an answer as well as possible, to realize as much cross-selling as possible, or to keep the call time as low as possible. So it is the goal, the determined output, that sets the requirements for 'learning' by the computers.”

“While speech recognition has become so good that we see this technology in all kinds of mobile apps and software, speech recognition is not something that can be turned on and then sent all human employees home. Speech recognition is certainly not flawless, but neither is the customer. Sometimes callers ask two questions in one sentence or formulate a question in such a way that it is not clear even to experienced employees what they mean without further clarification. Interpreting the question behind the question is, for the time being, something that people are even better at.”

From recognition to understanding

The use of artificial intelligence is therefore promising, but there are still some hurdles to overcome. Van Hessen: “Language is a complex phenomenon and there is still a great challenge in understanding speech correctly. For people it is immediately clear what the difference is between 'Jan kisses Marie' and 'Marie kisses Jan', but for the computer both statements contain the same words. Computers also still have great difficulty with an ambiguous sentence such as 'Jan sees Marie with the binoculars'.”

To be able to solve these kinds of problems, it is necessary for the computer to analyze the recognized text grammatically, so it determines what the subject, object and the relationship between the two are. In language technology this is called a triple: subject, object and predicate; parts of the standard known as Resource Description Framework. The possibilities of RDF and Tripples are very powerful, as the Newsreader video below shows, but it will be some time before this technology will roll into the contact center world.

Applications of AI in customer contact: in different phases

The results of speech recognition – in the form of recognized text – are already being used in various parts of customer contact. Van Hessen explains: “The customer contact process consists of various parts and phases, and each phase has different application options for speech recognition. By entering their 'personal details', customers can in many cases do so without human intervention identified become. For this identification you can, for example, work with speech recognition of zip code and house number. This offers the opportunity to immediately and automatically present the relevant customer data to the employee before he/she accepts the call.”

The next step is that, in addition to identification, the customer's question is also used as input, for example by asking the customer to briefly say what the question or the subject of the contact is. Van Hessen: “The answer can be used to explain the call classify and that makes it possible to route the conversation to the right team without human intervention. You can also present relevant information to the employee in advance. When classifying the question or subject, we are now able to correctly classify the question in 90 percent of the cases based on the recognized speech and available data of the caller.”

Open question speech recognition

At this point, a combination of identification, classification and routing of the call – the most extensive application of speech recognition at the moment – ​​applied at KPN. The customer says what he or she is calling for; if possible, the customer is automatically identified based on the telephone number. The system connects the customer to the most appropriate agent, who then not only has access to the caller's CRM data, but also sees the customer's question in text on the screen. Relevant information from the knowledge system is added to this. It is a good example where the output of the speech recognizer – the recognized question from the customer – is used by an AI system to provide new information to the employee.

Offline and real-time analysis of conversation content

And finally, speech recognition also offers the possibility to record all speech real-time and/or offline analyze for quality purposes; for example to monitor when which topics or sentiments have been mentioned in a conversation. “With text mining, large amounts of recorded conversations can be analyzed. For example, you can check which processes or topics predominate in conversations with a low NPS or customer satisfaction score. In a real-time variant of this analysis, you can present certain signals about the situation to the employee.” Van Hessen gives real-time emotion detection as an example: 'this customer is agitated'; or making suggestions: 'this customer may benefit from our new product' or presenting relevant information ('customer calls about poor Wi-Fi reception, see an analysis tool here').

Artificial intelligence and the future of customer contact

When we look into the future, all kinds of innovative solutions present themselves. It helps that not only speech recognition has developed well. "Also the having the computer pronounce texts – something that has made great strides in the Google project Deepmind with WaveNet – is now so good that you no longer immediately realize that it is a computer that generates the speech,” says Van Hessen. Here too, a combination of AI solutions will ensure that you can, for example, call the municipality to ask about the status of your requested building permit. “With Natural Language Generation (NLG), a technique that allows the computer to construct clear, correctly formulated sentences, the system can then give you a clear answer based on a number of parameters from, for example, a database. Until recently, slot filling was still used – the number of pasting together all kinds of separate pieces of spoken text – but NLG is considerably more flexible and makes sentences that sound and come across more beautifully and clearly.”

If you go one step further, you can think of the possibility to get spoken sentences from, for example, a French customer as Dutch text on the screen, or to hear it as Dutch speech through your headset. ecosystem already does this for certain language combinations in Skype. Here too, there are all kinds of new possibilities in the combination of speech recognition, automatic translation and speech synthesis. It is precisely in those combinations of different techniques and technologies that the innovation opportunities lie – and we can expect a lot from this in customer contact in the near future.

Also read part 1 of the interview with Arjan van Hessen about artificial intelligence.

Customer Experience, Technology Concentrix Catalyst, Telecats