ChatGPT is "not comparable to the average chatbot" but the application possibilities are (also with version 4, later this year) the same as with the current AI-based chatbots, according to a Gartner analyst. How is that possible? 

 

Although the next version GPT-4 is already in the pipeline and is said to be 'even better', it also has the same shortcomings as GPT3. This concerns 'lack of understanding' and risks in the field of security, privacy and reputational damage. We list the most important conclusions of experts about the properties of ChatGPT, including interpretation.

 

1. With ChatGPT, the bar for chatbots has been raised a lot higher

That says Gartner senior director analyst Uma Challa in an interview with MyCustomer. According to her, ChatGPT “is not comparable to the average chatbot as we know it today”. The current version ChatGPT-3 is a “human-like chatbot that is superior in understanding intents, can take and answer user questions, answer follow-up questions, admit its mistakes, challenge false assumptions, generate contextualized content, translate, extract and organize information, generate executable code, and most importantly, moderate its content.”

2. ChatGPT itself doesn't know what it understands and what it doesn't

The question is whether ChatGPT's "superior ability to understand intent" (as Gartner puts it) really exists and if so, whether it is sufficient. ChatGPT rarely asks a clarification question, so responds fairly primary and comes up with one response, unless you explicitly ask for multiple alternatives or rephrase the question. ChatGPT does not refer to (other) sources in answers and does not currently consult the internet. ChatGPT-3 itself also does not quite understand what exactly is understood: the intention, text, language or something else:

Can ChatGPT3 understand text? – “Yes, ChatGPT-3 can understand text. It is a powerful language processing model that can understand and generate human-like text. It can perform a wide range of language tasks, such as language translation, text summarization, answering questions and more.”

Is ChatGPT able to recognize or understand intentions? – “ChatGPT is a language model, it is not able to recognize intentions like a human being can. However, it can recognize specific words and phrases and understand contextual meanings, allowing it to respond to questions and commands you give. But it is not able to understand the intentions behind those words and sentences as a human being can.”

What is the relationship between ChatGPT that uses statistical rules to create text and ChatGPT that understands the underlying meaning of text? – “ChatGPT uses statistical rules, based on patterns it has learned from a large dataset of text, to generate text. When it generates text, it doesn't necessarily "understand" the underlying meaning of the text in the same way a human would. Instead, it uses the patterns it has learned to predict which word or phrase is likely to be next in a given context.

However, ChatGPT can also be used for understanding the underlying meaning of text through tasks such as question answering, language translation, and text summarization. In these tasks, the model uses its understanding of the relationships between words and sentences in a text to generate appropriate answers or translations. So in terms of text comprehension for certain tasks, ChatGPT can understand the text.” (Answers from ChatGPT-3)

 3. ChatGPT is suitable for the chatbot tasks as we already know them

Challa believes that ChatGPT will be able to improve the chatbot experience and provide a much more advanced service. The ability to communicate in natural language stands out with ChatGPT-3.

“The relatively high level of intelligence that ChatGPT shows compared to all chatbots on the market is the reason for the hype and sparks ideas around potential applications of this technology,” Challa told MyCustomer.

However, the use cases that Challa sees for ChatGPT are the same applications that the current generation of chatbots are used for: FAQs, product information, knowledge articles, self-service guidance, and low-complexity customer intents that can be solved with content. As far as Sander Hesselink (Telecats) is concerned, these are applications in “the periphery of customer contact".

However, Challa is hopeful: “As models improve in maturity, complex tasks can be delegated to chatbots in subsequent stages of adoption.” The question is whether her hope is justified, see point 6 below.

4. AI-powered chatbots have limitations and risks

Can customer service managers suffice with an upgrade? No, because Challa also warns about ChatGPT's vulnerabilities. For example, it is not clear what the accuracy of ChatGPT is when it comes to generating incorrect answers. The language model can also learn from bad examples, so that the quality decreases instead of increasing. She also points to the high cost of ChatGPT, which means that the technology is not open and available to a limited number of companies. According to Challa, the model is trained on huge amounts of data “from the internet” (which is not true), she warns of legal problems in reusing this information. She also points out the risks that arise when companies or individuals provide data to the ChatGPT3 basic model.

Risks – Cyara, a specialist service provider in the field of CX assurance, points out the risk of data security issues when using chatbots. On the one hand, for example, AI helps to detect data breaches faster (up to 28 days faster than through conventional security). On the other hand, chatbots lead to new risks. Chatbots can impersonate employees, spread ransomware and malware, and engage in phishing. Part of this is 'bot repurposing', in which an existing bot of a company is used for other purposes. Cyara also warns against applications such as ChatGPT, which can mislead customers with incomplete or incorrect information, or where customer data is misused. Incidentally, the limitations and risks are also explicitly described the user interface of ChatGPT.

5. ChatGPT-3 itself is not entirely clear about compliance and personal data

On the one hand, it is stated that ChatGPT-3 can understand and recognize certain types of personal data when they appear in text, such as names, addresses and telephone numbers. Processing uses named entity recognition (NER) models, regular expressions, and other techniques designed to identify and extract personal data from text (response 1). But if you ask about GDPR compliance, OpenAI's solution comes with a different explanation (response 2).

How does ChatGPT-3 recognize personal data that may be part of a request? – (response 1) “As an AI language model, GPT-3 can understand and recognize certain types of personal data when they appear in text, such as names, addresses, and phone numbers. However, it is unable to recognize all types of personal data, and it is unable to understand the context in which personal data is used. (…)

The data you provide to me will be sent to OpenAI servers to process your request, but the data will be deleted from the servers immediately after the request is completed. (…) It is also important to note that GPT-3 is unable to understand the meaning of personal data and does not store or use it for any purpose other than generating a response to the user's request. The personal data will be removed from the servers immediately after the request has been completed.”

How does ChatGPT comply with the GDPR? (response 2) - "As an AI language model, I don't have the ability to collect, store, or process personal data, so I don't have the option of whether or not I'm GDPR compliant. However, My Creators (OpenAI) have implemented strict data protection measures to ensure that any personal data used to train me is treated in accordance with the GDPR and other relevant data protection regulations.” (Answers from ChatGPT-3)

6. The next version is mind blowing

ChatGPT-3 is currently under development. According to AI expert Gary Marcus the new version, which will probably be released in the spring, will generate even more buzz and make the current version pale. “GPT-4 is going to be a monster,” said Marcus. The difference is mainly in the training of the new version, with more parameters and more data. But Marcus also foresees that ChatGPT's limitations will not be resolved, as the underlying architecture (based on a Large Language Model) remains unchanged. As a result, real understanding of the world remains out of reach and other ways will have to be sought to improve the reliability of the output. (Ziptone/Erik Bouwer)

Featured, Knowledge base, Technology