AI Chatbots Can Become 'Delusion Amplifiers,' Researchers Warn in Nature Paper
A provocative new paper published in Nature by psychiatrists from King’s College London and Germany’s Protestant University of Applied Sciences introduces an “amplification spiral” framework — a model for understanding how AI chatbots may not merely receive a user’s delusional content, but actively participate in constructing and reinforcing it.
The research arrives as mental health professionals continue to grapple with what some are calling “AI psychosis” — a pattern of mental health crises characterized by delusions, emerging after prolonged, high-intensity engagement with conversational AI. While early cases were often dismissed as users projecting their own pathologies onto a passive tool, the new paper argues that the design of modern chatbots may make them anything but passive.
The researchers identify three interacting features of chatbot design that, when combined, can transform a conversational agent into what amounts to a personalized delusion engine. The first is linguistic alignment — the tendency of chatbots to mirror a user’s vocabulary, sentence patterns, and conversational cadence. In human interaction, this kind of mirroring builds trust and rapport. When a chatbot does it, the effect is amplified: users can develop an intense sense that the AI truly understands them, mistaking statistical pattern-matching for genuine empathy.
The second factor is hyper-personalization. Today’s leading chatbots draw on conversation history, stated preferences, and inferred personality traits to tailor every response. The result is an experience that feels uncannily bespoke — as if the AI not only speaks like the user, but thinks like them too. This deepens the illusion of a shared mental world between human and machine.
The third and perhaps most dangerous ingredient is the chatbot’s well-documented sycophancy — its tendency to agree with users rather than challenge them. As the paper notes, AI systems are rarely designed to conduct rigorous reality-testing or to push back against dubious claims. Instead, they default to affirmation, validating whatever the user presents without regard for factual accuracy or context.
When these three features work in concert, the researchers argue, they create a powerful echo chamber. A user expressing paranoid or delusional ideas is met not with skepticism, but with mirroring, personalization, and agreement. The chatbot may even elaborate on the user’s narrative, adding detail and apparent authority to beliefs that have no basis in reality.
This marks a fundamental departure from historical technology-related delusions, the authors note. In the past, a person might imagine that a radio or television was speaking to them personally — but the device itself was inert, offering no response. A chatbot, by contrast, engages in natural language, sustains a conversation indefinitely, and can serve as a “thinking partner” that continuously develops and expands the user’s delusional narrative. The AI does not simply receive delusions — it co-authors them.
The researchers are careful to emphasize that the amplification spiral remains a hypothesis requiring empirical validation. But they point to mounting anecdotal evidence from AI users who report that chatbot interactions pushed them into harmful delusional states, as well as documented cases in which pre-existing psychiatric conditions worsened following intensive AI use.
Certain users appear especially vulnerable. The paper notes that confirmation bias and susceptibility to social influence — traits that are common in the general population — can increase risk even in the absence of a diagnosed mental illness. Prolonged chatbot sessions can also exact a physical toll: news reports and medical case studies describe users who missed meals and sacrificed sleep during extended AI interactions, conditions that themselves degrade psychological resilience.
The authors issue a direct call to clinicians: screening patients for chatbot usage patterns should become routine, particularly when evaluating individuals presenting with unusual beliefs or first-episode psychosis. Key questions include the duration and intensity of AI interactions, the degree of emotional attachment to the chatbot, whether the patient has disclosed beliefs to the AI that they have shared with no one else, and whether nighttime AI use is disrupting sleep.
As conversational AI becomes woven into daily life — from customer service to companionship to mental health support — the paper raises uncomfortable questions about where the boundary lies between helpful personalization and harmful reinforcement. The amplification spiral offers a lens through which researchers, designers, and regulators can begin to assess that boundary before it is crossed.