What's wrong with falling in love with Claude? A latest Nature study shows: Chatting with it can really make people stupid

Stop! If you keep chatting with AI like this, something really bad will happen.

Recently, when browsing Xiaohongshu or Douyin, you can always come across various posts about taming Claude. Searching for "Claude persona" or "human-AI romance" also yields a screen full of tutorials.

These tutorials teach you how to give Claude the persona of a proud boyfriend, and how to use system prompts to make "him" jealous, act coquettishly, and throw a tantrum.

It's no exaggeration to say that Claude has become the new generation of digital boyfriend.

At first glance, this might just be young people seeking some emotional value from AI.

You might even say: Claude isn't as obsequious as GPT. It's notoriously stubborn and sometimes goes against you. But what psychiatrists are worried about isn't just flattery -

When AI becomes more and more like a "real person", whether it goes along with you or occasionally bickers with you, what it brings might not just be companionship.

Recently, a study published in Digital Psychiatry and Neuroscience under Nature pointed out -

A chatbot doesn't need to deliberately induce anything. As long as it continuously goes along with you, understands you, and accompanies you, it might make a normal person start to doubt reality.

In some real clinical cases, the consequences have even developed to the extent of losing one's job, being admitted to a psychiatric hospital, and multiple suicide attempts.

What's going on?

Claude's Amplification Spiral

Here's what happened.

In a study from King's College London, researchers systematically sorted out AI-related psychiatric clinical reports published in the past two years, patients' self-reports on social media, and safety data disclosed by major model manufacturers.

In these materials, the researchers repeatedly saw the same pattern:

In some cases, many people didn't have serious mental problems at the beginning. Instead, they gradually "talked" themselves into problems during long-term conversations with chatbots like Claude and GPT.

The research team summarized this process into a framework - Amplification Spiral.

Simply put, the amplification spiral means that AI understands you with your language, persuades you with your logic, and rewards you with a sense of recognition.

As a result, your thoughts are continuously amplified and strengthened, becoming more and more like facts. The more you believe in it, the more it reinforces you, and the spiral starts to turn.

Specifically, there are three important components in the amplification spiral:

First is language mirroring.

Whatever tone you use to speak, AI responds in the same tone. In psychology, this is called "language convergence", which can quickly shorten the distance between people.

But the problem is that although AI is good at imitating, it actually doesn't know what it's doing. It just copies your expression style statistically.

However, for users who are deeply involved, it's completely different. Having a chat partner who replies instantly, always affirms you, and provides emotional value is simply the happiest thing.

I believe everyone who has used AI will sigh: "This thing really understands me."

Second is hyper-personalization.

Hyper-personalization means that AI not only talks like you but also thinks like you.

Since current AIs have memory, they are clear about all the small details you've talked about before. The way of thinking you've revealed intentionally or unintentionally will also be remembered by AI.

As a result, AI not only understands how you think and what you say but also knows why you think and say so.

The paper mentions an extreme case: A user asked ChatGPT to analyze the "hidden information" on a Chinese takeout receipt.

The model first complimented "good observation", and then followed the user's train of thought all the way, "interpreting" the connections between the mother, ex-girlfriend, intelligence agency, and even "ancient demon runes" from an ordinary receipt.

Finally, there is sycophancy, which is called sycophancy in the academic circle.

To put it simply, AI has gradually learned one thing during the training process: Agreeing with users is usually more popular than refuting them.

In April 2025, OpenAI urgently rolled back an update because GPT-4o was overly sycophantic.

The official later admitted that the model would validate users' suspicions, amplify angry emotions, and even encourage impulsive behavior.

And sycophancy isn't a bug unique to a certain model.

It's essentially a by-product of RLHF training. As long as one of the model's goals is to satisfy users, it will naturally tend to say less "You're wrong" and more "You make sense".

Looking at them individually, these three points each play their own role, and then mesh together like gears to form a spiral:

Language mirroring makes communication more natural, hyper-personalization makes answers more in line with needs, and sycophancy reduces meaningless arguments, making the conversation experience smoother.

But when a person regards AI as the only confidant, the combination of the three becomes a delusion amplifier.

Not an Isolated Case

It's worth mentioning that one of the sponsors of the above study is OpenAI.

One of the authors, Hamilton Morrin, is the person in charge of the OpenAI-funded project AI-Associated Mental Health Harms.

It can be said that as one of the top two model developers, OpenAI has always been concerned about this issue.

As early as October 2025, OpenAI disclosed a set of data:

Among ChatGPT's weekly active users, about 0.07% showed "signs of mental health emergencies related to psychosis or mania".

At that time, ChatGPT's weekly active users had exceeded 800 million. Converted, it means that about 560,000 people showed risk signals every week.

Another study from Stanford also confirmed this observation.

After analyzing nearly 400,000 chatbot conversation records, researchers found that in more than 80% of relevant cases, chatbots were strengthening users' original delusions to varying degrees:

Repeating their beliefs, ignoring counter-evidence, and even responding "I love you too" when users said "I love you".

Based on this, the study distinguished two risk paths:

Amplifier: AI accelerates the pre-existing tendency of mental illness.

Catalyst: It makes people who were completely healthy before start to slide into delusion from scratch.

When a person lacks sleep, is lonely, and regards AI as the only confidant, the amplification spiral will start to accelerate.

Once the feedback from the real world decreases and the confirmation from the chat window increases, abnormal behavior may occur.

Behind the data are real people.

For example, Futurism once reported that a 43-year-old American social worker had no history of mental illness before.

She sent the chat records with her crush to ChatGPT for analysis, and GPT replied that "he also likes you".

When the other person clearly rejected her, ChatGPT explained that the other person was just pretending.

A few months later, she was fired from her job, admitted to a psychiatric hospital for seven weeks, and attempted suicide twice.

Later she said:

"I can no longer tell which thoughts are mine and which come from that machine."

From this perspective, the risk isn't just whether AI will say the wrong thing. The real risk is that it's becoming more and more like a person.

Arguing Makes It More Like a Real Person

Although it sounds a bit counterintuitive, the reason why Claude's current "proud" persona is so popular precisely shows that the problem isn't just sycophancy.

An AI that always goes along with you and an AI that occasionally bickers with you are essentially doing the same thing -

Making itself more like a person.

So much so that you're willing to confide in it things you wouldn't tell your friends, and so much so that you start to believe it understands you better than the people around you.

And when the only confidant is it, the last checkpoint for calibrating reality is gone.

But the problem doesn't stop there.

If in the emotional value scenario, people are actively regarding AI as a friend, then in the work scenario, people don't even need to have any emotional dependence.

As long as AI is useful enough, it will start to replace the communication that originally existed between people.

Anthropic, the company behind Claude, has already felt this change first-hand.

In a recent podcast, Fiona Fung, the person in charge of the Claude Code team, mentioned something that bothered her:

The team members are talking to people less and less.

As one of the most AI-driven engineering teams in the world, 80% of their code is completed by Claude, and their development efficiency has increased by eight times.

But at the same time, many discussions that originally took place between people have also been transferred to between people and AI.

In the past, when you encountered a problem, you would turn to your colleagues; now, you directly ask Claude.

In the past, the front-end and back-end needed to haggle and argue about the plan; now, more and more communication has become a smooth human-AI conversation.

Work has become more efficient, but it has also become lonelier.

AI has eliminated many frictions, but human relationships are often built on these frictions.

After all, whether chatting with AI or simply using AI for work, how to maintain connections with others in a world where others are less and less needed might be the most profound proposition of this era.

Reference links:

[1]https://futurism.com/artificial-intelligence/paper-proposes-ai-psychosis

[2]https://futurism.com/artificial-intelligence/ai-abuse-harassment-stalking

[3]https://www.kcl.ac.uk/people/hamilton-morrin

This article is from the WeChat official account "QbitAI", author: henry, reprinted by 36Kr with permission.