Researchers have developed a new method to detect, predict, and influence personality traits in large language models (LLMs). The technique could be of great value for customer contact applications, but it also introduces new risks. 

 

The central question posed by the researchers (from Anthropic, the University of Texas at Austin, UC Berkeley, Truthful AI, and Constellation) was how to objectively measure and control personality traits in an AI assistant. These included hateful or fanatical behavior, as well as the tendency to hallucinate.

In their paper, pre-published on Arxiv, they conclude that personality traits of AI assistants are measurable, predictable, and controllable via so-called persona vectors in the so-called activation space of language models.

Persona vectors

This activation space is the mathematical space in which the states of a language model are represented during text processing. It can be considered the model's framework, which, based on coded patterns, determines how meaning, language style, and behavior are generated. Certain behaviors of the model are stable and measurable, just as human traits can be. Manipulating the persona vectors relatively easily changes the personality traits that generate the output.

The researchers essentially isolated these persona vectors from LLMs and then used them to induce or suppress certain characteristics. By measuring before and after the interventions, they gained insight into the effect of personas in LLMs.

Behavioral change

Their findings indicated that relatively small adjustments to these persona vectors can cause major, and sometimes undesirable, behavioral shifts. Consider a chatbot that reacts differently to a specific dataset after fine-tuning. These relatively small adjustments, however, involve complex technical processes, such as inference-time steering and preventive steering.

The researchers also found that for fine-tuning a bot, manipulating the persona vectors is much more effective than adjusting based on prompts or applying filters.

Identifying and correcting (un)desirable behavior

The research provides insight into new opportunities for CX and AI specialists when it comes to the high-quality deployment of AI chatbots and virtual assistants in customer service and contact centers.

The downside of the technique is that the controls used to manipulate personality traits can also be used maliciously. For example, seemingly positive personality traits can be exploited for ulterior motives. The method opens the door to manipulative chatbots, but the researchers didn't mention this.

The persona vector method offers a new and automated way to make (desired and undesired) AI behavior measurable, controllable, and predictable, or even prevent it – think of the tendency to hallucinate or exaggerately empathize with or agree with a conversation partner. The research also provides insight into how programmers can pre-analyze datasets for data that cause behavioral shifts in bots. These kinds of interventions could be part of the governance increasingly required for the deployment of AI-powered chatbots.

The full study is available on Arxiv

Featured, Knowledge base, Technology