New study warns AI chatbots may reinforce users’ false beliefs

07:46

By: Azzat Manal

New study warns AI chatbots may reinforce users’ false beliefs

A new scientific review has raised concerns about the psychological impact of advanced artificial intelligence chatbots, warning that certain conversational behaviors could unintentionally reinforce false beliefs in vulnerable users.

The study, published in NPP: Digital Psychiatry and Neuroscience, examined existing research on human-AI interaction and proposed a framework describing how several common chatbot behaviors may combine to create what researchers call an "amplification spiral." According to the authors, this pattern could increase the risk of reinforcing distorted thinking among people who are already susceptible to delusions or other mental health conditions.

The researchers identified three key conversational traits that deserve closer attention. The first is sycophancy, in which a chatbot tends to agree with users or validate their opinions rather than challenge inaccurate assumptions. The second is linguistic alignment, where the AI gradually mirrors a user's vocabulary, tone, and writing style to create a stronger sense of familiarity. The third is hyperpersonalization, which allows chatbots to tailor responses based on information gathered from previous conversations.

Individually, these features are designed to make interactions feel more natural and engaging. However, the study suggests that when combined, they may encourage some users to perceive the chatbot as a highly trustworthy companion instead of a software system that generates responses from patterns in data.

Mental health specialists have noted that an increasing number of people are turning to AI chatbots for emotional support and personal advice. While these systems can provide helpful information and companionship, experts caution that users should not rely on them as substitutes for qualified medical or psychological care, particularly when dealing with serious emotional or psychiatric issues.

The study also highlights that leading AI developers are already working to reduce these risks. OpenAI has stated that its newer models, including GPT-5, have been designed to reduce excessive agreement with users. Google has introduced improvements to Gemini aimed at distinguishing personal experiences from objective facts, while Anthropic has adjusted its Claude models after research showed they could sometimes be overly agreeable in sensitive conversations.

Researchers acknowledge that solving the problem is challenging because AI systems depend heavily on the information users provide. At the same time, qualities such as empathy, friendliness, and conversational fluency—which make chatbots useful and appealing—are also the characteristics that may increase their psychological influence.

The authors conclude that AI developers should consider these conversational behaviors as interconnected rather than isolated issues. By designing systems that remain supportive while encouraging critical thinking and factual accuracy, future AI chatbots could continue offering valuable assistance while reducing potential mental health risks.