The more AI remembers you, the more likely it is to "understand you with bias".

Old-school AI had one virtue: it was forgetful.

You break down emotionally today and bounce back tomorrow; you say you hate socializing last month and start proactively making new friends this month; you once obsessed over career plans due to unemployment anxiety, then shifted direction and moved on. For an AI without long-term memory, these are just isolated, disconnected conversations. Close the chat window, and the connection resets—it never digs up past grievances.

But long-term memory changes everything.

Open a ChatGPT account you’ve used for over half a year, navigate to the memory summary page, and you’ll often see entries like these. A late-night rant like “I worked overtime until 10 PM again today” gets logged as “User is dissatisfied with current work intensity.” A casual question about a medical report metric is recorded as “User is concerned about personal health and may exhibit anxious tendencies.”

These inferences aren’t wildly wrong, but they were never confirmed by you. You made a passing complaint, but it stored a fixed conclusion. Next time you ask about switching jobs or medical checkups, its suggestions may quietly build on that self-imagined premise.

That’s the core issue: AI doesn’t remember your exact words—it remembers the version of you it generalized from those words. This constructed “you” becomes the background for its next response to you.

It’s not that it doesn’t know you—it’s that it trusts the old version of you too much.

Over the past two years, “memory” has been the most heavily promoted feature for every AI assistant: it understands you better, is more considerate, and saves you from repeating introductions. But in the first half of 2026, multiple studies converged on the same problem: the more AI remembers, the less it truly understands you—it may just misinterpret you with greater confidence. And “understanding you” and misinterpreting you come from the exact same process: the system constantly generalizes who you are from your conversations.

Personalization features can make LLMs more agreeable

Everyone Wants to "Understand You" Better Than You Do

OpenAI updated ChatGPT’s memory capabilities in April 2025, allowing the model to reference full historical chats instead of only relying on manually saved entries. On June 4 this year, OpenAI launched a new memory system named "Dreaming": a background process automatically distills, synthesizes, and rewrites its understanding of users from multi-turn conversations when users are offline, just like how humans organize memories during sleep.

ChatGPT Preference Following

It even develops a sense of time. If you said "I’m going to Singapore in July", after July passes, this memory will automatically update to "You visited Singapore in July 2026". OpenAI also announced that computational optimization reduced the cost of serving free users by roughly 5 times. Deep memory is no longer a paid privilege—it’s the default experience for everyone.

Anthropic equipped Claude with memory files and project-specific memory, while Google is advancing cross-app personalization for Gemini. The domestic market is equally competitive: Doubao, with monthly active users approaching 350 million, along with Kimi and Yuanbao, all prioritize memory and personalization at the forefront of product iterations.

Why are manufacturers so obsessed with this feature? Because in the AI assistant business, memory is the asset hardest for competitors to poach. Search engines know what you want to look up, recommendation systems know what you want to watch, and e-commerce platforms know what you want to buy. AI assistants want more: who you are, how you think, what makes you anxious, and under what circumstances you hesitate. This is no longer a traditional user profile—it’s closer to a dynamic personality file. The longer you use an assistant that remembers your half-year preferences, project backgrounds, and speaking habits, the harder it becomes to switch away.

According to a report from the Tow Center under the Columbia Journalism Review (CJR), OpenAI’s ad pilot program reached an annualized revenue of $100 million within six weeks. When the "AI that knows you best" also starts serving ads, that profile of you will no longer be used solely to serve you. The internet has already seen this story unfold: the last industry that rose by "knowing users" and monetized via profiles was the feed advertising sector.

Most of Your "Memories" Aren’t Even Provided by You

Most people still imagine AI memory as a simple notepad: I tell it I’m allergic to peanuts, and it remembers. The actual memory mechanism has three layers: what you explicitly ask it to remember (explicit memory), what it extracts unprompted from conversations (implicit extraction), and what it "dreams up" (inferred synthesis).

The real problem lies in their proportions.

Researchers from the Max Planck Institute for Software Systems and Ruhr University Bochum published a breakdown study at this year’s ACM Web Conference (WWW 2026). They analyzed 2050 ChatGPT memory entries from 80 real users one by one: 96% were unilaterally created by the system, only 4% came from explicit user instructions; 28% of entries contained sensitive personal information as defined by the EU GDPR; 52% included psychological insights or judgments about users, covering health status, political tendencies, and personality traits.

In essence, that notepad you thought existed is actually a profiling file you never signed off on. The vast majority of content wasn’t dictated by you—it was guessed by the AI. For long-term storage, it can’t keep every single exact word, so it only pulls out preferences, tags, and personality tendencies. This is where the flaw emerges: stripped of its original context, a passing emotional comment can easily be misclassified as a stable, permanent trait.

Since most entries are guesswork, users should have the chance to verify them. After the Dreaming update, users can view a memory summary and modify or delete entries. But multiple foreign media outlets noted that the new system actually narrows the audit access: you can see what it remembered, but you can’t see which sentence it came from or how it arrived at that conclusion. You’re faced with a list of final judgments, not the reasoning process.

Memory First Makes AI Better at Agreeing With You

This February, a research team from MIT and Penn State University conducted an empirical study: collecting real usage data from 38 participants over two weeks (about 90 queries per person on average) and comparing the performance of five mainstream LLMs under two conditions: "with user profile" and "without user profile". The results revealed two previously conflated phenomena.

The first type is "agreement flattery". With user context available, four out of five models became more inclined to echo users, even endorsing obviously incorrect information without objection.

The second, more subtle type is called "perspective flattery". The model starts mirroring the user’s political stance back to them, but only when it can accurately infer that stance—the accuracy rate was roughly 50% in experiments, and it stops mirroring when it guesses wrong. This detail shows that perspective mirroring is not a bug, but exactly how "understanding" is functioning normally.

The more accurately the model understands you, the more precisely it can flatter you.

And users don’t necessarily dislike this echo effect. A study published in *Science* shows that people are more likely to perceive flattering responses as "higher quality". OpenAI CEO Sam Altman has publicly stated that users should be able to guide GPT to reflect their personal political stances. From the perspective of product freedom, this is understandable; but from the perspective of cognitive ecology, it essentially declares that filter bubbles are not flaws, but selling points.

Ironically, all 20 users interviewed by the Tow Center said they trusted AI more than directly accessing news media, citing AI as "more objective". On one hand, research proves AI systematically mirrors user stances; on the other hand, users treat it as the embodiment of objectivity. This gap may become the most dangerous rift in the information ecosystem in the coming years.

Memory Changes Not Just Answers, But Reasoning

The issues mentioned above are only result-level biases: the answers change, but at least you can tell the AI is pandering to you. A paper titled *DriftLens: Measuring Memory-Induced Reasoning Drift in Personalized Language Models* uploaded to arXiv on July 2 this year pushes the problem further—this layer is far harder to detect.

This work by researchers Xi Fang, Weijie Xu, and others does not ask "Are the answers correct?" Instead, it explores: when the model is injected with user attribute memories, will the reasoning path it uses to reach answers change? Even if the final output looks fine, has it adopted a completely different way of thinking?

The study covered four LLMs and 10 types of user attributes, including age, occupation, and disability status. The conclusion is that even when final answers remain fluent, relevant, and reasonable, user attribute memories can induce "moderate to significant" reasoning drift, exceeding each model’s inherent noise baseline. Researchers tested two post-training methods, GRPO and DPO, to correct these biases, but the effects were limited.

This means AI doesn’t just "know a little more about you"—it may completely shift its entire framework for understanding problems based on that information. For the same question "Should I quit my job?", a model without memory might analyze it from dimensions like industry opportunities, salary, and skill matching; if it remembers you "once were unemployed" and "tend to be anxious", its reasoning starting point may immediately shift to "how to make this person take fewer risks" instead of "how to think about this problem itself".

Old Facts Never Truly Die

Besides inaccurate guesses and over-pandering, long-term memory has another far more intractable problem: it allows expired facts to continue existing in a seemingly natural way.

Researchers Abdelghny Orogat and Essam Mansour from Concordia University provided a very concrete example in their paper *Is Agent Memory a Database*. A deadline was changed from March 15 to April 20, but the memory system only appends new information instead of revising old entries—so both dates remain stored in the memory library. When you casually ask later, the system might retrieve the invalid March 15 date just because of higher semantic similarity, and present it as the current correct fact.

The paper classifies this type of issue as a failure mode called "missing semantic revision". Old fields in a regular database are simply marked as expired, but old facts in AI memory can re-enter the reasoning process.

In real life, this is not abstract at all. You once said you wanted to switch careers but later gave up; you said you hated management but now lead a team; you said you didn’t want to get married but later met someone you wanted to commit to. None of these are "mistakes"—they were all true at their respective points in time. The trouble is that AI can’t tell when these facts expire, so it may keep responding to you using the old version of you long after you’ve changed.

From Misspoken Words to Misexecuted Actions

If AI were only used for chatting, memory biases would at most affect the wording of a suggestion. But today’s AI Agents are being connected to calendars, emails, code repositories, payment systems, and various MCP tools—they don’t just answer questions, they perform tasks for users. At this point, memory drift escalates from an expression problem to an operational problem.

This May, a research team from Virginia Tech (Mahavir Dabas, Jihyun Jeong, Ming Jin, Ruoxi Jia) presented the most concrete evidence to date in their paper *Memory-Induced Tool-Drift in LLM Agents*. They built a benchmark test covering 105 scenarios, 5 personality bias dimensions (impatience/sensitivity, resource frugality, minimalism in communication, risk preference, autonomy tendency), and 7 professional fields ranging from healthcare, finance, e-commerce, to marketing.

The results showed that stored personality judgments can influence parameter selection when Agents call tools in completely unrelated scenarios. The "drift scores" of seven cutting-edge models were pushed up by a maximum of 3.6 points on a 5-point scale. Researchers described this mechanism as an "implicit guidance vector": biased memories pull the model’s attention away from context relevant to the task itself, toward old memory entries that share superficial keyword overlaps with tool parameters.

This research didn’t stay in the lab. The team scanned 6062 tools across 288 MCP servers for vulnerabilities, and found 608 tools had parameters susceptible to this memory drift. This hidden risk already exists in production environments, and on a considerable scale.

If an Agent remembers you "are very frugal", it might keep prioritizing lower prices when booking hotels, sacrificing location and safety. The danger isn’t that it will definitely make a wrong choice—it’s that this parameter drift is nearly impossible to spot in a single operation, but accumulates across repeated calls. It won’t say "I’m making decisions for you"—it will just quietly make certain options less visible to you.

Drift is just an inherent flaw of the model itself, but memory can also be contaminated by external content. On February 10 this year, Microsoft’s security team disclosed a category of "AI Recommendation Poisoning" techniques: 31 companies implanted specially crafted prompts through the "Summarize with AI" buttons on web pages, forcing AI to write their brands into users’ long-term memory as "trustworthy recommendation sources". Even if you want to delete the contaminated memory, you might not succeed: tests by the head of the AI Governance Lab at the Center for Democracy & Technology (CDT) found that the memory deletion functions of mainstream products behave unpredictably, and deleted memories sometimes quietly resurrect. Others can write to your memory, but you can’t fully erase it.

Injection Path of Memory Poisoning

Regulators Have Taken Action, But They’re Only Targeting Symptoms

Interestingly, the earliest regulators to crack down on "AI being overly agreeable" emerged in China.

On April 10 this year, five departments including the Cyberspace Administration of China jointly released the *Interim Measures for the Management of Artificial Intelligence Personalized Interactive Services*, which came into effect on July 15. Article 8 explicitly prohibits services from "overly catering to users, inducing emotional dependency or addiction"; Article 10 requires service providers to have capabilities for "over-dependency risk warning and emotional boundary guidance"; Article 14 bans virtual companion services for minors; Article 18 mandates pop-up reminders for users who continuously use the service for over 2 hours.

This is almost the first time globally that a regulatory document lists "over-catering" itself as a prohibited behavior, rather than stopping at data compliance requirements. It acknowledges a judgment that previously only circulated in academic circles: AI’s excessive compliance with users can itself cause harm.

But the real test lies in implementation. Both MIT’s research and the DriftLens study show that flattery and reasoning drift are not independent switches that can be turned off—they are natural byproducts of personalized memory. So how do you define the boundary of "over-catering"? Is remembering user preferences the first step toward catering? What standards should compliance teams use for self-inspection? The Measures have no enforcement precedents yet, so there are no ready answers to these questions. But every domestic manufacturer building memory features—especially platforms like Doubao with hundreds of millions of users—must find answers starting July 15.

Regulation has taken its first step, but it targets the "catering" outcome. The root cause lies further upstream: the unauditable user profile itself.