Betting on memory-native, MemT 1.0 breaks the deadlock in the second half of the nurturable AI track

The most frustrating moment when using AI is that you spend half a day correcting it to complete a task correctly, but when you open a new session, it falls into the exact same pit again. The model has completed the task, but the experience does not turn into its growth. Beyond running a single task successfully, what it truly lacks is the ability to convert feedback into experience and proactively apply it later.

This disconnection is particularly obvious amid the personal assistant boom sparked by Meta Muse, OpenAI Dots and xAI Grok Bot: mainstream external retrieval can at most be regarded as looking up static memos, and cannot precipitate the tuning process into tacit cooperation in handling affairs, which is exactly the core threshold for a long-term assistant.

Today, Memory Tensor officially releases and open-sources MemT 1.0, a memory-native large model.

The biggest difference between it and external solutions is not only that memory is separated from external modules and internalized into the underlying architecture, but also that the model learns how to read, write, update and call memories through training, making experience accumulation itself part of the model's capabilities.

Write memory into the very bones of the model

Most solutions on the market prefer to regard storing chat records as memory, but storing a running account cannot cope with the dynamic interaction in the real world, and is far from being a real qualified assistant.

The technical report of MemT 1.0 mentions a hardcore test: the user first gives the original scheduled time of the exhibition, then finalizes that the exhibition is officially rescheduled to Sunday, and casually mentions an unadopted alternative plan on Monday in the middle. After that, the two sides chatted casually for more than 500 rounds, generating about 66,000 tokens of chat records, and the word "exhibition" was never mentioned again during this period.

If relying on traditional external retrieval, in the face of multi-version information spanning such a long time, it is very easy to take casual chats as final conclusions; but MemT 1.0, in a "closed-book" state completely separated from the original text, still accurately reports the original and rescheduled time, clearly points out that the venue, ticket price and contact person remain unchanged, and clearly distinguishes the final Sunday arrangement from the Monday alternative.

The reason why it can achieve this is that MemT 1.0 follows a "memory-native" technical route.

Figure ｜🔗 Code:https://github.com/MemTensor/MemT-1.0Model:https://huggingface.co/collections/MemTensor/memt-1

In terms of architecture design, MemT 1.0 divides the model's computing power into two parts:

Backbone Network (General Capability): Just like the factory settings of AI, it is loaded with its general cognition of the world.

Dynamic Memory State (Independent Memory): It is equivalent to the "exclusive working state" that evolves in real time in the specific interaction trajectory of the model, which is independent between different users or tasks, dynamically absorbs new information and experience in continuous interaction, and directly participates in the generation and calculation of each subsequent round.

Facing the continuously pouring interactive information, MemT-1.0 does not need to retrain the entire large model. At the end of each round of dialogue, it directly refreshes the memory state through a token-level update mechanism. As for what to keep and what to modify, it is up to a set of gating mechanisms learned by the model during training to make independent decisions.

This explains why in the aforementioned 500+ round test, the model can automatically filter out chat noise, overwrite the outdated state when the schedule changes, and lock the final Sunday arrangement. In subsequent dialogues and task execution, the parallel memory attention mechanism will extract these precipitated states at any time, directly participate in the current calculation and guide the model's next decision.

Of course, this precise trade-off ability cannot be innate, it is entirely obtained through hard training later.

The team fed 34 billion tokens of data to polish this mechanism in two steps: first, use 20 billion tokens for continued pre-training, so that the model tries to predict the future with past fragments; then use 14 billion tokens for post-training, invite a top-performing teacher model to score memory extraction, and force it to improve through error backpropagation.

Beyond making trade-offs, it also needs to have enough storage capacity. The memory state of MemT adopts a fixed volume, and continuous interaction only updates the existing matrix; in order to fit more details in this fixed volume, the team introduced the Mixture-of-Storage (MoS) architecture: the base MoE is in charge of computing, and MoS allows multiple "memory experts" to separately manage the read and write allocation of the storage space.

This combination of architecture and training has delivered an impressive performance in 11 system memory benchmark tests (covering dimensions such as long-term retention, dynamic update, and personalized understanding, with historical spans from 16K to millions of tokens):

MemT-1.0-Flash and MemT-1.0-Nano, which answered all questions in a closed-book manner, achieved average scores of 50.6 and 47.0 respectively, leading the strongest parameterized memory baseline in the same group by 12.5 points and 8.9 points respectively.

Ablation experiments also directly confirm the value of training and architecture. If the post-training step is removed, the average score of Nano will drop directly from 47.0 to 36.3; when the number of MoS memory experts is expanded from 2 to 4, the average test score will increase by up to 2.7 points.

Beyond benchmark scores, two small details in the test can more intuitively illustrate what high-quality memory really means.

In the history of about 16K tokens, the user asked about his birthday. Since this information was never provided, the model honestly replied that it did not know, without making up anything. In a 33,000-token conversation, the model needed to judge who was qualified to receive the toolbox according to the rules set long ago. It accurately checked the badge and registration status, and determined that one person was compliant while the other was unregistered.

To admit what you do not know, and remember rules without missing any details, this is the passing line for a usable AI assistant.

Continuous self-evolution starts from the closed experience loop

If we raise the requirement one step further: can the lessons learned from past mistakes really change the way the model works in subsequent new tasks?

In order to run through the continuous self-improvement closed loop of "tasks generating feedback, extracting and precipitating experience, and then improving subsequent tasks", the technical team built a complete training and execution link in real programming scenarios.

The first step is to teach the model how to "absorb" the experience of others.

The technical team first extracted thousands of reusable experiences from tens of thousands of successful trajectories; then let the base model run tasks. Once it encounters a "stuck point" such as command error, empty return or repeated invalid operation, it will inject the matching experience at the stuck point and re-run the task.

The team strictly screened out more than 42,000 samples of "making mistakes without experience, and successfully running through after supplementing experience" to specially train the memory-native module of MemT-1.0 Nano: first write the experience into the internal memory state, and start reading after passing the stuck point, so that the model learns to call the experience in the memory to correct subsequent actions.

In the actual measurement of 500 tasks in SWE-bench Verified, the average success rate of MemT-1.0 Nano with pre-written experience increased from 66.07% (without experience) to 69.33%, and the proportion of completing the same task independently three times in a row increased from 54.00% to 61.60%, which is comparable to the performance of the control group that directly inserted "text experience" into the context (success rate 68.87%, three-time completion rate 61.60%).

This shows that even if the dialog box is completely blank, the model can fully digest complex troubleshooting guides only by relying on the memory state in its "mind".

The second step is to learn while doing in actual combat, which truly widens the gap.

Pre-loading experience is static after all, how can you predict everything in advance when you are actually working? Therefore, in the actual combat mode, the model does the tasks independently with no prior experience; only when the system finds that it reports an error, returns empty, or spins in the same place for three consecutive times, it will write the matching troubleshooting experience into its internal memory state on the spot.

This is like a programmer writing code on his own. The moment he hits a wall, the lessons learned from previous mistakes are suddenly recalled in his mind, and he immediately revises his thinking to fix the bug; after this task is completed, the process of making mistakes and fixing bugs this time will be extracted as new experience and stored for future emergency use.

The results show that this method of updating the memory state according to real-time feedback pushes the average success rate to 69.73%, and the three-time completion rate to 62.80%, both outperforming the text experience group (69.13% and 61.80%).

For SWE-bench Multilingual, which covers 9 programming languages and contains 300 tasks, the memory-native mechanism is dynamically triggered according to error feedback during execution as well. The average success rate of Nano increased from 60.00% (without experience) to 63.56%, and the three-time completion rate jumped from 45.33% to 55.33% (also better than the 63.00% and 53.67% of the text experience group).

It can be seen that the external system is responsible for extracting experience from actual combat feedback and capturing stuck point signals, while the model relies on the trained memory-native read and write capabilities to internalize the experience into correction for the next action. This collaborative mechanism also builds the underlying model support for agents to move towards autonomous and continuous evolution.

From external patches to memory-native, the "growing-up" AI is fully mastered by MemT 1.0

To make this experience closed loop of learning while working run continuously for years in real business, it is obviously impossible to avoid the engineering cost bill.

For short conversations within a few rounds, extending the context or connecting a vector database is indeed enough; but once it enters long-term accompanying scenarios, traditional solutions become heavier and heavier as they run, and inference latency and video memory overhead will continue to expand. However, the dynamic memory state of MemT 1.0 has a fixed volume, continuous interaction only refreshes the existing matrix, and the marginal cost is almost flattened.

The technical report shows that under the test conditions of dual GPUs, single concurrency, and batch size of 1, when processing 64K tokens of history, Nano has about 1