The more it knows you, the more dangerous it gets? MemPrivacy Reveals the Next Stop of AI Memory

In the era of AI memory, Agents are becoming more and more like real personal assistants.

It remembers your habits, knows your schedule, understands your health status, and can even gradually form a "personal profile" of you during long - term conversations. However, problems also arise: If all these memories are to be stored in the cloud, is privacy still secure?

On April 22nd, OpenAI open - sourced a lightweight privacy filtering model called privacy - filter, aiming to solve the problems of PII detection and desensitization in large - model systems.

OpenAI Privacy Filter address: https://openai.com/zh-Hans-CN/index/introducing-openai-privacy-filter/

Just two weeks later, the MemTensor team came up with a more radical solution. This solution was jointly developed by the MemTensor and Honor HONOR teams, and Tongji University was also involved — this is the first in - depth cooperation between an edge - device manufacturer and a memory infrastructure team on the issue of "Agent privacy".

They officially open - sourced the privacy protection framework and a series of models MemPrivacy for edge - cloud collaborative Agents. Even more surprisingly, in the same real - conversation privacy extraction task, the F1 score of MemPrivacy is up to 50.47% higher than that of OpenAI privacy - filter.

This is not a temporary cross - field move.

Previously, MemTensor had launched MemOS, elevating Agent memory from vector libraries or RAG plugins to a manageable, schedulable, and evolvable system resource: what to remember, how to retrieve, how to update, and how to govern are all included in a "memory operating system".

MemPrivacy is more like a privacy layer that naturally emerged from MemOS in the edge - cloud collaborative scenario — when an Agent starts to remember users' preferences, health status, account credentials, and work context in the long term, the problem is not just "can it remember", but "can it remember securely". This makes it logical for MemTensor to develop MemPrivacy: it does not start from general PII masking, but directly from the real usage scenarios of Agent long - term memory, redefining privacy types, protection levels, and placeholder mechanisms.

On the day of its release, MemPrivacy was listed among the TOP1 of Hugging Face Daily&Weekly Papers.

This is not a simple "privacy masking tool".

It aims at the most core and thorniest problem of the next - generation personalized Agents: How can cloud - based large models continue to have long - term memory and personalization capabilities while ensuring that users' sensitive data never leaves the local device?

In other words, what MemPrivacy wants to achieve is: Make the Agent usable but invisible.

Paper title: MemPrivacy: Privacy - Preserving Personalized Memory Management for Edge - Cloud Agents

Paper address: https://arxiv.org/pdf/2605.09530

Code repository: https://github.com/MemTensor/MemPrivacy

Model repository: https://huggingface.co/collections/IAAR - Shanghai/memprivacy

OpenAI Enters the Scene

But 8 Labels Can't Support an Agent's Long - Term Memory

OpenAI's privacy - filter has a simple idea: scan the text, identify privacy segments, and then replace them with semantic labels.

For example, replace the person's name "Maya" in the user's input with [PRIVATE_PERSON].

This model has 1.5B parameters, with about 50M active parameters. It uses a bidirectional Token classification architecture, supports a 128k context, and focuses on high - throughput PII detection and masking.

Compared with the traditional method of simply replacing everything with ***, this is already an improvement: it at least retains some semantics.

However, when it comes to the long - term memory scenario of edge - cloud Agents, problems quickly emerge.

OpenAI privacy - filter only provides 8 basic privacy labels. For ordinary form desensitization, this may be sufficient; but for an Agent that needs to understand users, remember them in the long term, and even call tools to perform tasks on behalf of users, this granularity is too coarse.

Bank card numbers, social security numbers, and project file numbers may all be grouped into the same [ACCOUNT_NUMBER]. Login passwords, database credentials, API Keys, and internal keys may all become [SECRET].

This is like labeling all dangerous items with the word "Dangerous".

It is a bit safer, but the semantics are also flattened.

The real problem is that an Agent is not a database cleaning script. It needs to understand the context, retain relationships, form memories, and continue to use this information in future conversations.

When a user says "My blood pressure is 160/110 today", this is not an ordinary number but a health indicator; when a user says "This is the connection string for my company's database", this is not ordinary text but a high - risk credential. If the coarse - grained labels fail to recognize something, there will be omissions; if they recognize it incorrectly, the semantics will be destroyed.

Thus, privacy filtering is in a dilemma:

If there are false negatives, users' privacy will be exposed; if there are false positives, the Agent will lose its memory on the spot.

This is exactly the most difficult contradiction to avoid for the next - generation personalized Agents.

MemPrivacy Appears

It's Not about Erasing Privacy, but Giving Privacy a "Local ID"

The core idea of MemPrivacy proposed by the MemTensor team is called: Local Reversible Pseudo - Anonymization.

It doesn't simply delete privacy information or replace it with meaningless asterisks. Instead, it performs a more delicate "switcheroo" on the edge device.

The entire process can be divided into three steps.

Step one, Edge - side Upward Desensitization.

When users have a conversation with an Agent on edge devices such as mobile phones and PCs, a lightweight MemPrivacy model will run locally first. It is responsible for identifying privacy segments in the conversation and processing them according to the protection level set by the user.

If the text contains "My blood pressure is 160/110 today", MemPrivacy will not directly turn it into ***, but replace it with a fine - grained typed placeholder like <Health_Info_1>.

The mapping relationship between the real blood pressure value and the placeholder is only stored in the local database.

Step two, Cloud - side Secure Processing.

What the cloud - based large model sees is: "My blood pressure is <Health_Info_1> today."

It cannot see the plain - text sensitive data of 160/110, but it still knows that this is a health indicator, so it can continue to perform reasoning, generate suggestions, form memories, and even call relevant tools.

Step three, Edge - side Downward Restoration.

When the cloud replies "Your blood pressure <Health_Info_1> is on the high side", the local system restores the placeholder to the real value and finally presents it to the user.

In terms of user experience, this process is almost transparent.

But in terms of the system architecture, the key sensitive data never really leaves the local device.

This is the most important design of MemPrivacy: Let the cloud understand the structure but not see the plain text.

Comparison of Three Routes

No Protection Means Exposure, Full Filtering Means Memory Loss, MemPrivacy Preserves Intelligence

In the edge - cloud Agent scenario, there are generally two extreme traditional privacy protection solutions.

The first is no protection.

The user's original data is directly uploaded to the cloud. The cloud - based model can fully understand the context, and the personalization effect is the best. However, sensitive information such as health data, private email, home address, and account credentials will also be fully exposed.

In today's increasingly strict data compliance environment, this is almost like walking on a tightrope.

The second is complete filtering.

All privacy content is replaced with *** or directly deleted. It seems safe, but the cost is that the Agent completely loses the key semantics. When users want it to remember their health status, financial constraints, and work context, it can only see a blank.

This type of Agent may seem safe, but in fact, it has lost the foundation for "long - term personalization".

MemPrivacy chooses the third way: Fine - grained Typed Placeholders.

The cloud doesn't know what your real blood pressure is, but it knows that this is a health indicator; it doesn't know what your private email is, but it knows that there is an email here; it doesn't know the plain text of your API Key, but it knows that this is a high - risk credential.

This design preserves two things: one is the privacy boundary, and the other is the semantic structure.

That's why MemPrivacy has the opportunity to strike a balance between privacy protection and Agent utility.

Hard - Core Strength

The F1 Score Surpasses OpenAI by Over 50 Points and Crushes GPT - 5.2

To verify the capabilities of MemPrivacy, the research team built a new evaluation benchmark MemPrivacy - Bench. This benchmark covers the conversation histories of 200 users, contains more than 155,000 privacy items, and supports the detection of privacy information in both Chinese and English.

In addition, to test the generalization ability, the team also conducted an OOD cross - test on the external personalized long - text conversation dataset PersonaMem - v2.

In the competition of extraction accuracy (the comprehensive F1 score of privacy text, level, and type) on these two major benchmarks, MemPrivacy showed a crushing advantage:

Far exceeds OpenAI's specialized model:

On MemPrivacy - Bench, the comprehensive F1 score of OpenAI privacy - filter is only 35.50%.

MemPrivacy - 4B - RL reaches 85.97%, and the gap between the two is an astonishing 50.47%! Even on the cross - distribution PersonaMem - v2 dataset, MemPrivacy still leads OpenAI by nearly 9%.

The reason is clear: OpenAI privacy - filter's advantage lies in speed. The non - autoregressive Token classification architecture brings high throughput; however, its problem is that the label coverage is narrow, the granularity is coarse, and it is not well - adapted to complex contexts and Chinese scenarios.

MemPrivacy redefines privacy types, protection levels, and training goals for the Agent long - memory scenario, so it is closer to the actual needs in real conversations.

What's more interesting is that MemPrivacy doesn't just beat OpenAI's specialized small model.

Challenges general large models across levels:

Even when facing the most powerful general models with extremely large numbers of parameters, such as GPT - 5.2, Gemini - 3.1 - Pro, and DeepSeek - V3.2 - Think, MemPrivacy - 4B and even its micro - version with only 0.6B parameters achieved a crushing victory on both datasets.

This shows that privacy extraction is not a problem that can be solved simply by increasing the number of parameters.

It is more like a highly structured, strongly constrained, and strongly bounded task. What really matters is not how large the model is, but whether it understands "what information should be protected, to what extent it should be protected, and whether it can still be used by the Agent after protection".

Prevent the Agent from Becoming Stupid

The System Utility Loss Is Less than 1% at the Lowest

There is a more practical problem in privacy protection: no matter how well it is protected, if the Agent becomes stupid, it's all in vain.

This is also the Achilles' heel of many rough desensitization solutions.

A user says: "My blood pressure has been high recently. Remember this and take it into account when arranging my exercise plan in the future."

If the system erases all information about blood pressure, health status, and exercise preferences, the cloud - based model will be safe, but it won't be able to provide truly personalized services.

Can the typed placeholders of MemPrivacy really preserve the utility of the memory system?

The team conducted end - to - end tests on several mainstream memory system platforms in the industry. All the underlying models are GPT - 4.1.