European rules approaching for the use of AI

Will developing and using AI-powered solutions such as chatbots become more complicated in the coming years? It is certain that effective AI starts with a good data set. It is precisely the use of (especially 'human') data for AI that will be subject to new rules. Ziptone spoke with Menno Weij (Partner Tech & Privacy Law at BDO Netherlands) and with Arjan van Hessen (Telecats/UTwente).

It is only a matter of time before new regulations for artificial intelligence will come from the EU. The first parts of the guidelines are published last spring and it shows that the new regulation is based on risk assessments. The following applies: the size of the risk is probability times impact. If there is a small chance that something will go wrong, but the impact is large, then the risk is greater and the rules will be more restrictive. Chatbots, for example, are classified in risk category 2, where there is an obligation to provide information: companies must be transparent about how the application works. It is precisely at this point that customer service may be faced with an additional duty to report to consumers. In this article the explanation.

Algorithm watchdog

“The tone has been set with the legislative proposals and policy memorandums of the EU that have been published so far,” says tech and privacy lawyer Menno Weij (BDO). “It is good that there will be rules, but I am currently concerned about the impact that these rules will have on companies and consumers. For example, there is talk of setting up an algorithm watchdog, a task that should be assigned to the Dutch Data Protection Authority in the Netherlands. The AP has been complaining for years about too little money and resources and is lagging behind on privacy files alone.”

Openness and transparency

A second line from the EU's plans relates to openness and transparency regarding the use and deployment of AI. Weij: “Legally, it comes down to making it clear that as a consumer you are dealing with, for example, a chatbot instead of a real person. The idea behind this is to give that person the opportunity to make a well-considered choice or to distance himself from a certain situation. For example, that that person can call customer service instead of settling for a chatbot.”

Data as training material: great impact

In addition, Weij also expects the necessary impact of another striking element in the upcoming legislation and regulations. “One of the drivers of European privacy policy is the French privacy watchdog CNIL (Commission Nationale Informatique & Libertés). CNIL, a progressive and respected privacy watchdog in Europe, has analyzed the different roles of AI vendors in the context of the GDPR. That analysis can have far-reaching consequences for companies that deploy AI applications towards consumers.”

That needs some explanation. In its analysis, CNIL argues for a distinction between data use by a party to improve (through the use of AI) services to its own customers on the one hand and data use to develop and train AI solutions for third parties on the other hand . Weij explains: “Let's take bol.com as a fictional example, which uses a supplier's chatbot. In this case, the customer of bol.com is a 'data subject'. Because bol.com determines the purpose of the data processing (such as: 'improving our services to our customers'), bol.com is the 'responsible'; the chatbot supplier is the 'processor'. But when the chatbot supplier asks bol.com to provide customer data with the aim of training the chatbot, then the aim is not 'improving our service to our customers', but to set up the chatbot supplier's service. It therefore changes color,” says Weij, “The chatbot supplier becomes responsible.”

Duty to inform chatbot supplier

“From the view of CNIL, this could lead to an obligation to provide information to the data subject (for example, the consumer, ed.). In practice, the chatbot supplier has two roles and can no longer hide behind bol.com or in the processing agreement. It is not inconceivable that the customer should receive a notification from the chatbot supplier when visiting the bol.com website.”

Chatbot Notification – At the moment it is not yet clear how the obligation of transparency should be fulfilled. You could think of a message such as 'you are now using an automated service that uses algorithms and data, whereby the data is collected, among other things, during your use of the service'. Weij suspects that it will move in that direction from the perspective of AI legislation: that people know they are dealing with a robot.

In addition, the consumer, as the person involved, should also know what the result of such an interaction with the robot is, says Femke Schemkes, a colleague of Weij. “On the one hand, automated answers are generated in response to the questions asked; on the other hand, data is collected and used during the shift, and perhaps also to allow the AI system to learn. For a layman who knows nothing about AI systems, this will not be immediately clear.”

Weij: “From the point of view of 'privacy', the variant in which the supplier also becomes responsible is particularly interesting. The party with which the consumer does business will then have to point this out, whereby that party should refer to, for example, a privacy statement from the original supplier.”

Dataset has a major influence on AI effectiveness

Then the technical side of AI. Also Arjan van Hesse (Head of Imagination, Telecats/UTwente) believes it is high time to develop legislation and regulations, “because technological developments are moving at lightning speed.” He explains that the operation of AI is highly dependent on the way in which datasets are compiled.

Van Hessen: “AI comes down to recognizing patterns in collected data. The broader the data set is, the better AI usually works. As you begin to impose more restrictions on the use of certain data from your dataset, AI can become more imprecise. For example, whether or not special personal data is used. In the US and especially China, privacy rules are considered less important and the result of this may be that Chinese AI, for example, will soon 'score' better than European AI. In Europe, the starting point is that you do not use sensitive personal data unless you can explain it very well. In addition, the European Union is strongly committed to the explainability of algorithms, so that the (end) user can be told why a certain decision was taken. That is certainly not easy with AI, in contrast to the more classic algorithms, but it is good that it will at least be tried.”

AVG already offers guidance for data use in AI – The legislation for the use of (special) personal data was amended in 2018, both nationally (GDPR) and European (GDPR). The GDPR and AGV stipulate that processing data for 'statistical purposes' is permitted. However, the result of AI training should not be applied directly back to the people involved, and certainly not if an automated decision-making (Article 22 GDPR).

Quality of the solution

There is a second reason why preventing bias is important: in addition to the risks of discrimination and abuse, the performance of the solution is also important. Van Hessen: “If AI is built from a dataset that is not a good representation of reality and developers then start using this solution widely, the solution will ultimately work less well. If you develop speech recognition without including Ukrainians who will speak Dutch, then the solution will not work for society as a whole.”

Van Hessen emphasizes that this problem already exists: after all, speech recognition works less well with the elderly, children and foreigners who speak Dutch as a second language, simply because these groups are usually not involved in training the speech recognizer. “So if we want to make the speech recognizer more inclusive, the speech data of all smaller groups that speak Dutch will also have to be used. And this process never stops, because 'new' Dutch people are added all the time, so that the speech actually changes constantly.”

Everything starts with the right training material

The training material must therefore be representative of the target group for which you want to use AI. That target group can also change over time – think of the arrival of new groups in your society, aging or people with dementia – and then AI must also be trained in this, says Van Hessen. “In the GDPR, your voice – because a biometric characteristic that can be used for unique identification – is now considered special personal data. That makes improving speech recognition technology more complicated. You don't want to leave the making and keeping of technology inclusive to Microsoft or Google.”

Strengthen expertise

Van Hessen would think it would be a good idea if organizations had more expertise about their own data collection and what is done with it. “Otherwise you end up with situations such as at the Tax and Customs Administration where the Supplements affair shows how things can get out of hand if you don't look closely at your data collection and your analysis method. Evaluation of methods used by experts, in order to learn from them, is something that happens too little in my opinion. Consider the relationship between the nature and quality of outcomes and the dataset. I increasingly hear from young data scientists that they find it difficult to convince their managers why something does or does not work well or why something is not statistically justified.” Of course, those managers will gradually get a better picture of data, algorithms and AI, but it would help enormously to speed things up a bit, says Van Hessen.

What should we take into account?

Back to the inevitable laws and regulations. For consumers, this could amount to a similar practice as with the cookie notifications: soon you may also have to click away an AI notification from a chatbot. “The question is what kind of explanation the consumer will get about this. And whether that explanation is understandable. Of course, offering a choice is a great thing. But such an extra notification can also ensure that the 'consent fatigue' of consumers is further increased," Weij fears. "The means is then worse than the goal."

There is a good chance that the new rules will create a lot of extra work for companies – and a lot for lawyers lawyer's paradise, said Wei. When it comes to easily developing applications – think of low code chatbots, something that can simply be picked up within the contact center – he is convinced that the current laws and regulations are sufficient for this. “Of course, low code developers must also be compliant. Companies that do not have their governance in order in this area and allow low code applications to be developed and rolled out without thinking, will sooner or later run into the lamp.” (Ziptone/Erik Bouwer)