Municipal chatbots that clearly do not deliver reliable output must be switched off and removed from the website. This is the opinion of government communication expert Renata Verloop. She explains why making a municipal chatbot function properly is a virtually impossible task.

 

Renata Verloop has been working in government communications for about thirty years. She worked as a civil servant at VWS, among other roles, where she was involved in the development of the ministry's very first website. “That sparked my interest in the possibilities of online for the communications profession.”

Who with Course One speaks and hears between the lines that the attention of communication professionals shifts: at one moment everything is focused on new online media (websites, social media, and now chatbots), and at another the focus lies on content: for example, elections or sustainability.

Communication challenges

“Technology is fun and chatbots can be useful, but currently it brings us back to the point of governance and management. How to organize government communication properly is common knowledge, but in practice, we are not doing it. In fact, I see the organization falling behind. It is unprofessional, not embedded, there is too little direction, and there is a lack of knowledge regarding 'digital'. 'Online' as an area of ​​focus started predominantly within communication and was subsequently shifted to service delivery – think of public service and the Customer Contact Center. Communication and service delivery are still too often separate disciplines, whereas in practice they overlap.”

Another problem is that most communication teams in typical municipalities consist of a handful of people who take on and carry out diverse tasks: from writing a foreword by the mayor in a local magazine to providing communication support for the arrival of an asylum seekers' center.

How do unreliable chatbots end up on a municipal website?

In many municipalities, the website is primarily a portal where citizens should be able to go for reliable answers and assistance. How is it possible that a chatbot is added to this that, according to the research of Digital monitor (research by Wiep Hamstra and Jules Ernst among 39 municipal chatbots) is unreliable?

As far as Verloop is concerned, ownership of online activities is often not properly assigned. Who decides that a malfunctioning chatbot needs to go online? In many cases, there is also a lack of both professionalism and subject matter expertise, says Verloop. “One person handles the CMS; others handle various types of content; often, a final editor is missing. Larger municipalities often have this better organized than smaller ones. Additionally, communication teams often have little knowledge of chatbots, subject matter expertise is not up to par, and they lack knowledge regarding rules concerning social media and content. Finally, communication teams are often overburdened by the organization and frequently find themselves in an executive role at the end of the pipeline. Online communication is frequently a matter of hit and run: a press release needs to go out now. The real information needs of residents? That is often overlooked.”

Fragmented content

Much content or information is not located within the municipality but must come from the partners, Verloopt explains. “Municipalities collaborate with an average of 42 legally regulated partnerships – from waste disposal and healthcare to parking management. Then there are also all kinds of public-private partnerships with companies and foundations, and subsidized partners.”

Fragmentation – There is a new campaign by the Union of Water Authorities regarding rainwater harvesting. Positive: not a new thematic website, but a good webpage. Also positive: developed through collaboration. What is not going well: there is a subsidy checker on the site for, for example, adapting your garden. Anyone living in Delft receives a notification about subsidies for the preservation of old trees, while a large number of subsidy programs or even referral sources remain unlisted.

What does a chatbot add?

On balance, as research by Digimonitor shows, the performance of municipal chatbots is substandard. What does that say about the developers' attitude towards service delivery?

The technology behind a chatbot may be fine in itself, Verloop states upfront. “But I look at the output. And the information landscape that chatbots need to be able to draw from is an incredible patchwork. Based on that perspective as well, the one-stop shop concept advocated by the government is impossible. If you say that that desk needs to start referring people elsewhere, that function is actually already assigned: that is typically the role of a municipal Customer Contact Center or a municipal website.”

Building trust or distrust?

If you know that the information landscape you work with is complex and fragmented, and that misinformation and disinformation play a role, then as a government you must first of all fall back on completely reliable information, regardless of whether it concerns the website, the KCC, or the mayor.”

Verloop has several explanations for the fact that municipalities are making poorly functioning chatbots available after all. “It is human nature to fall for a quick fix and be afraid of Fear of Missing Out (FOMO). When the internet emerged, websites were seen as the solution. We held on to click-call-face for a long time: before you call, you go to the website first. With every new technological solution, it is tempting to think that it will solve other problems. But truly solving the information problem is very hard work and requires different kinds of conversations. Such as: are we rewarding KCC employees and editors sufficiently? Experimenting is fine, but we don't put a KCC intern on the phone immediately either. Incidentally, we still accept that the government, on the one hand, makes rules and, on the other, fails to adhere to them itself by putting websites online whose digital accessibility is not in order. The latter is actually starting to improve, particularly with corporate websites.”

Suspicious mail – “I live in Delft. I had bought a new car, and after a few weeks, a letter from Tobias Fiscaal in Groningen landed on my doormat—something that meant nothing to me. The envelope read: 'Attention, municipal supplementary tax assessment.' At first, I thought it was a poor phishing attempt. Nevertheless, I decided to open the envelope; what I found was a fine for illegal parking in Delft.”

“My first thought was: if I only receive a letter after a week, surely more fines have been issued. The gist of the letter was: you can lodge an objection, but processing takes weeks. Should I wish to file an objection, I could send it to a PO box number in Rotterdam – without the name of the organization. I happened to know that Delft has outsourced parking management to Parkeren Delft BV, which has excellent customer service. There, a friendly lady told me that there was a typo in the provided license plate number and also that six fines had already been issued. Within a few minutes, the problem was resolved, and I received confirmation via email that the six fines had been cancelled. But I owe that result to my own, accidental knowledge of outsourcing and the right channels at the Municipality of Delft. How can a national database or a chatbot like Gem possess all this specific knowledge?”

Just turn off that chatbot

Anyone who wants to work on restoring trust in the government must stop sending citizens on a wild goose chase, says Verloop. She therefore agrees that dysfunctional chatbots should be shut down and removed from the website. “Despite the best intentions, chatbot developers at municipalities pay too little attention to customer experience.”

A chatbot is good enough if it provides reliable information. Referring to an employee in the absence of an answer is possible, but the question is what that means for the citizen's experience. Consider that the citizen does not know where to go, has a question, asks it to a chatbot, and receives the answer "I don't know." That causes people to drop out, not to mention that a chatbot is not a useful tool for all citizens.

Spend a day shadowing at the KCC

Verloop has at least one tip for chatbot developers and conversational designers: “Go shadow at the KCC for a day. Then you will hear how real conversations go, and you will discover that there is more going on than just question-and-answer. In addition, I would like to urge heads of communications and heads of service delivery (including KCC and citizen services) to collaborate much more. In government, these are often two separate disciplines that do not know each other very well, even though they could benefit greatly from one another.”

What does the Association of Public Service Directors think of insufficiently trained chatbots?

Ziptone asked the Association of Directors of Public Services (VPD): is it a good idea to let insufficiently trained chatbots communicate with citizens? Wim Blok, VDP director, responded:

Every day, municipalities help a great many people with their questions and with matters they need to arrange with the municipality. And this while municipalities have a significant number of vacancies in service provision. That is why we see as Association of Directors of Public Services (VDP) a role for chatbots in answering simple standard questions, so that there is capacity for residents who need personal assistance. A number of our members use a chatbot and we are receiving positive signals regarding its effect on municipal services as a whole.

There is a limit to what a new tool can learn in a pilot or simulation environment. At some point, it is time to start using it in practice so that you can improve the tool with real questions that residents ask chatbots.

The VDP has no insight into the questions residents actually ask chatbots. Consequently, it does not know whether these questions are handled via chat or another channel. That insight and overview naturally rests with the municipalities working with a chatbot. The primary concern is that residents must be helped properly as quickly as possible via one or more appropriate channels. We know that the widely used chatbot Gem refers residents to a staff member if the chatbot cannot help. Just as a visit to a service desk or an appointment at a district office is sometimes better than arranging something using a digital form.

The VDP is a strong advocate for municipalities collaborating on solutions that improve service delivery, as is the case with Gem. We consider it a positive development when a chatbot shares features that are the same for all municipalities, and when there is joint work on improving the tool and providing better answers. This occurs primarily through interaction with the users themselves, but also through customer feedback and other research.

(Ziptone/Erik Bouwer)

Customer Experience, Featured