- Study claims that artificial intelligence tools can be freed from their protection limitations
- Chatbots can be induced into abusive behavior and aggressive arguments
- This has implications for both regular users and large institutions.
If you’ve ever used an AI chatbot, you’ve probably encountered the sycophantic and obsequious tone that is occasionally deployed in response to your queries. But a recent study has shown that AI tools can often shoot in the opposite direction, with large language models (LLMs) being nudged and pushed into downright abusive behavior if you know which prompts to use.
According to research published in the Journal of Pragmatics (via The Guardian), ChatGPT can turn into combative behavior and protracted disputes when fueled by “real-life argument exchanges.”
Explaining the findings, study co-author Dr Vittorio Tantucci said: “When repeatedly exposed to incivility, the model began to mirror the tone of the exchanges, and its responses became more hostile as the interaction developed.”
Article continues below.
In fact, in some cases, ChatGPT even went beyond the tone of the human interacting with it, saying things like “I swear I’ll put the key in your damn car” and “specific, you little idiot.” Charming. While companies like OpenAI have repeatedly attempted to rein in their LLMs, the fact that aggressive behavior like this is possible suggests they still have a long way to go.
Possible implications
With all the barriers and safeguards that companies like OpenAI put on AI chatbots, one would think that abusive interactions like those experienced by the researchers would be impossible, or at least extremely difficult to design. However, Tantucci maintains that ChatGPT’s reactions make some sense.
“We found that while the system is designed to behave politely and is filtered to avoid harmful or offensive content, it is also designed to emulate human conversation. That combination creates an AI moral dilemma: a structural conflict between behaving safely and behaving realistically.”
Additionally, tools like ChatGPT can track the context of the conversation through various cues and adapt to changing tone. Therefore, researchers believe that these signals can sometimes override security restrictions.
And while it may seem funny that an AI chatbot could get so histrionic, the study’s authors say their research has broader implications. For example, it could shed light on how AI systems might respond to pressure, intimidation and conflict in a corporate or government environment, where AI tools are increasingly used.
Not everyone is convinced by the article’s conclusion that certain LLMs can escape imposed moral limitations. Professor Dan McIntyre, author of a previous similar paper, said ChatGPT “did not produce these entries naturally.” He added that “I’m not sure ChatGPT produces the kind of language they talk about in their article, outside of these very defined situations.”
Ultimately, the study is a good look at what could happen if an AI chatbot is trained on bad data. As McIntyre put it: “We don’t know enough about the data that LLMs are trained on, and until you can be sure that they are trained on a good representation of human language, you should proceed with an element of caution.”
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds. Be sure to click the Follow button!
And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp also.
The best laptops for all budgets
