The business case for real-time (speech) translation (RTT) is clear: you can serve customers in small language areas in their own language and you can benefit from a wage cost advantage. Sabio has implemented RTT for two different customers. There is a lot involved.

 

Real-time speech translation (RTT) is an interesting and important application for companies with a multilingual customer service. There are different ways to set up RTT, but the end result is always that employee and customer communicate in their own language. Where employee is written, you could also read voicebot.

Also read: Real-time speech translation at Transcom: handling 'expensive languages' in low-wage countries

Implementing a 'tool' like RTT is no easy task. There are multiple applications that need to be linked together. In addition, implementing RTT has consequences for the human side: you will have to look at the user experience of both customers and employees and the CX of the customer. RTT also has consequences for your operation, with the most striking being the AHT - which shoots up. Enough reasons to talk to three specialists from Sabio.

The starting point is a conversation in two languages, so Klaus Failenschmid (Head of User Experience, Sabio) off to a start. Not everyone speaks or understands every language. “At the United Nations, this problem is solved per language with simultaneous translators who listen and speak the translation at the same time. In this, the parties in the conversation ensure that no one, let alone the translator, is overloaded with input. In the application that Sabio has developed, both processes are separated: the input is converted and the response is converted. Both processes take place in turn and not simultaneously as in the UN.”

CX and UX

In these conversions, both CX and UX are important, says Failenschmid: UX if you use the application as a customer or employee, CX if you have used the application as a customer. “Customers who are dealing with this technology for the first time – think of the scenario 'I can't log out of my e-scooter' – are not focused on listening to instructions. So you may only have to give the instruction after the customer has finished formulating his or her question.

The customer will also have to learn how the 'turn-taking' mechanism works. First, by letting the customer know at the beginning of the dialogue that his or her input will be converted into another language. Second, by indicating when the customer's input is being processed. "This can be done by so-called 'noise tokens' such as 'I have received your information', 'I will take a look for you', 'just a moment' or by general background noises," Failenschmid explains. "But you can also, after your explanation, emit a beep every time the customer says something, with or without a short message such as 'just a moment' or 'this is being translated now'. In short, using RTT requires that you explain the process to the customer.

UX for the employee

Not only the customer has to learn to use this application, the employee also has to work differently than normal. First of all, the employee has to learn to use text-based communication. Where you can still formulate in telegram style with chat ('OK' or 'I'll check'), the text that the agent enters with RTT is the material that the voicebot has to convert into speech that is played to the customer. And where you can still correct e-mail if you read something over, with RTT the input of the employee has to be converted into speech as quickly as possible.

A second obstacle is the information the agent has to work with. If the customer does not formulate a clear sentence, the translation module may come up with an unclear translation result. The question then is whether the agent should respond to this with an answer or ask for clarification.

Words misspelled by the agent can also impact the translation. The extent to which you should always correct typos is tricky, but agents do tend to correct their own typos, says Failenschmid.

When converting to another language, you also have to take into account semantic and cultural differences, even within a language area. For example, there is a difference between Dutch and Flemish-Dutch. Failenschmid therefore prefers to speak of 'localisation' rather than 'translation'. Localisation is the process of making products such as games, software, websites and audiovisual material suitable for use in other language areas.

Voice-to-voice and agent assist

Sabio's experience shows that if you support the agent as much as possible in formulating the answer to the voicebot, the speed of the interaction increases. For example, by offering ready-made answer snippets: 'quick replies', a form of agent assist. (text continues below image)

 

 

Another solution is to circumvent all these text-related challenges, for example by having the agent work not with text input but with his or her own speech – and then convert it.
Rutger Hugen (solutions architect Sabio Nederland) responds: “A PoC has already been realized for this variant in which the agent responds with speech (voice-to-voice) to what is on the screen. However, so far, practice has shown that the text-based model including agent assist currently works the fastest, especially when the employee does not actually have to type. Text-based interaction (from the agent) also has an advantage, according to Hugen. It increases the consistency of the answers; with the help of generative AI, you could also adjust the agent's input towards the right tone of voice. But it is possible that we will exchange 'text to speech' for 'speech to speech' in the near future.”

Operational implications

One of the most noticeable changes will be the AHT. 'Tap' is slower than speech and the question is whether you should still consider this way of customer contact as a 'call'. When setting up the agent desktop, you will also have to think carefully about what works best for the employee: the entire dialogue on the screen? Or only information that relates to the phase of the conversation and the intent of the customer? Failenschmid: "You want to give the agent quick access to relevant information where possible."

Anyone who wants to get started with RTT must have all kinds of processes in order, including knowledge management and AI policy that provides guardrails, for example. It is difficult to imagine that RTT works in practice with all these caveats and points of attention. But Failenschmid assures us: it currently works at shared scooter company Bird, which has a customer service team of Transcom in India users of shared scooters in various European cities. “Indeed, you have to have a lot in order, just like when managing regular agents and monitoring the quality of regular agents,” says Failenschmid, “AI is just like a flesh and blood agent that you have to train and guide.”

Practice often or for a long time makes perfect

Finally, as with any AI-powered solution, this one also gets better as more contacts are handled. For scooter-sharing company Bird, the voice channel is an escalation model. Hugen explains that Bird deals with a relatively small volume in each language area. “How are you going to arrange personal contact if you only have a small contact volume in each language? If you lack volumes, you will have to rely on a longer period of time in terms of fine-tuning. That is also the aim of a partnership: continuous improvement.”

(Ziptone/Erik Bouwer)

Featured, Knowledge base, Technology NICE, Wise