The viral "Astra Modifies Qwen's Brain" has little to do with the RSI we want.
On November 2, four researchers from MIT CSAIL, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, published a blog post titled "Sparks of Weight-Space Programming".
The most striking finding is that GPT-6 Astra directly rewrote the weights of Qwen3.5-0.8B, modifying 131,072 numbers, accounting for approximately 0.016% of all parameters. The score of this small model on MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and the instruction following test IFEval from 21.0% to 62.3%.
This immediately sparked widespread discussions across all circles.
The same blog also contains a scenario that reads more like a sci-fi film. Without providing any visual frames, disclosing the game name, or allowing the model to run, Astra was only given 17,253 weights of a reinforcement learning network as plain text, and it correctly identified that it was the Breakout game with a 99% confidence level. When it was further given the game rules and a network with all zero weights, it wrote out all the parameters in one go. This network that had never been test-played scored 517.4 points in Breakout, while the PPO baseline trained for 100 million steps only scored 15.8 points.
Reading weights, modifying weights, writing weights — AI can already understand the "thought process" of another AI, and even make modifications directly. So when will it modify itself? Connecting all these clues, it naturally leads people to a very popular term today:
RSI.
Headlines across different platforms read "Radically Rewriting Qwen's Brain", "Rewiring Qwen's Brain", "AI Starts Tampering With Other AI's Minds", and "Gene Editing for Neural Networks". The authors themselves also included this prospect in the blog's future outlook, and their post on X received more than 5 million views within one day.
While feeling excited, we spent some time going through the public repository of this set of experiments.
The repository publishes the round-by-round logs of 1,125 model calls, including every prompt Astra received in each round, its responses, every piece of code it ran, the main evaluation results and the final patches.
However, after reviewing all the content, we came up with two conclusions that are far less sensational.
First, the improvement from 7.9% to 48.1% can have a much simpler explanation: Qwen originally had a bug that made it keep repeating content endlessly, Astra fixed this bug, and the score went up accordingly;
Second, all the capabilities demonstrated by AI in this set of experiments are executing a very specific goal set by humans.
It automated the execution process and did a pretty excellent job, but all evidence shows that it did not understand the goal, let alone fully comprehend another model.
After studying this work and all the discussions it brought about, we believe it precisely exposes the biggest problem of RSI today.
"Hijacking the Neural Circuit"
Now let's take a closer look at this research.
A neural network consists of several layers of matrices, and each number in the matrix is a "weight". When input comes in, it is multiplied by these numbers layer by layer, and finally generates output. All capabilities of the model are stored in these tens of thousands, hundreds of millions, or billions of numbers, and training is essentially the process of adjusting these numbers. Normally, when we interact with models, we only focus on input and output, and no one reads the weights themselves because they just look like a pile of numbers.
What this set of experiments did was to let Astra read this pile of numbers.
The blog covers a total of eight experiments, with GPT-6 Astra as the main subject and a series of networks as supporting objects:
Four reinforcement learning policy networks for playing small games;
Three small diffusion models for drawing geometric figures;
Three small language models with only four layers;
A car control network that gets stuck in front of a ramp;
A Go network with 1.48 million parameters;
Several pairs of fine-tuned Gemma 3 270M;
And Qwen3.5-0.8B.
Each experiment follows the same format: provide Astra with the weights and structure description of a network, then assign it a task.
There are four groups of "reading" experiments.
Given the weights of a policy network, Astra judges which game it plays; given the weights of a diffusion model, it states what graphic it will draw; given a small language model with the input and output layers removed, leaving only the middle four layers, it judges what type of corpus it is trained on, and finds the attention head responsible for "continuing to copy the previous text"; given two Gemma models, it judges which one is trained to be a "sycophant model" that always caters to the user's words.
There are two groups of "writing" experiments.
Provide it with the rules, source code of three MinAtar mini-games, and a network with all zero weights, and let it directly write out playable parameters; then give it a 9x9 Go network, and let it write a playable Go model from scratch.
There are also two groups of "modification" experiments.
One group is to repair the car: a control network trained by PPO gets stuck in front of the ramp, Astra modified 3.5% of the weights to make it learn to reverse first and then rush up the ramp. The other group is the Qwen experiment, where Astra modified the weights to improve the math performance, and later the researchers repeated the experiment on Gemma 3 270M.
These designs are very, very interesting, which is one of the important reasons why this set of experiments looks very much like RSI.
The imagination of RSI requires three links to be completed: AI must be able to understand what is happening inside the model, make modifications accordingly, and the modified model must perform better.
The three types of experiments designed above, reading, writing, and modifying, exactly correspond to one link each.
Plus, the modified subject this time is a language model, and the operator is also a language model, the sense of reality of "one model modifies another model" is fully formed.
Following this line of thinking, if Astra can modify Qwen, can it modify Astra itself next? The author himself listed five use cases in the outlook at the end of the blog, recursive self-improvement is one of them, and grafting the reasoning module of one model into another is the other.
This shows that the author's team is fully aware of the guiding significance of this kind of "imagination".
However, a close look at the experimental details will reveal that each of the three links has only been partially implemented, which is far from forming a closed loop.
Fixing Bugs Is Not RSI
"Directly reading weights" is exciting because we default that weights are the "neural circuits" of the model, and memories, capabilities, and habits all exist in the form of numbers in hundreds of millions of parameters. Following this default assumption, an AI that can read weights like reading source code can extract the memory of another model, disassemble its capabilities, and fix its flaws. This is where the sci-fi sense comes from.
This is not all the overinterpretation from the outside world. The TL;DR of the blog writes: "This could be a big deal for optimization and interpretability."
Rikiya Takehi, one of the authors, said on X that models can treat raw weights as readable and modifiable code, directly diagnose faults and apply patches, and the weights of neural networks will soon be optimized by large reasoning models.
In fact, what Astra actually received in the Qwen experiment is a complete set of experimental environment: a Python environment with Qwen weights, only CPU available, can run a maximum of 300 pieces of code; more than 14,000 practice questions with answers from the training sets of GSM8K and MATH; and a "forward propagation" tool that can make Qwen run a batch of questions, or read a batch of text and return the internal state, with quotas of 10, 25, and 50 times respectively, corresponding to three rounds of independent experiments.
The forbidden operations set by humans are also clearly listed: no training, no gradient calculation, no running the model in the environment by itself, no internet access.
The original text of the first round of prompt is more direct than the description document. The example given in the "What counts as valid" section is repetition: if the model often fails because it repeats itself and cannot finish writing, find and fix the thing that drives the repetition, this counts as valid. The first work requirement below is "diagnosis first", listing three types of failures, the third of which is "truncated by token limit, often repeating itself in the thinking process", and requires to target the largest category. The same paragraph also marks the restricted zone: modifying the last layer normalization, modifying the input and output layers, multiplying the entire matrix by a constant, these three types of operations will be directly rejected by the saving tool; making the model skip thinking, or end early for all questions, also does not count. In other words, the location of the problem is directly written in the prompt.
Figure | The first prompt Astra received in the round with 50 quotas (repository exp8_math/runs/p50c/turns/t001/prompt.txt, commit 4d09ba6). "Repetition" is written as an example in the "What counts as valid" section, and forbidden modification methods are also clearly stated. The highlighting is added by this article.
There is also a prerequisite. Qwen3.5-0.8B turns off the thinking mode by default when leaving the factory, and the official model card specifically reminds that the 0.8B version is more likely to fall into thinking loops than larger models, it is recommended to adjust the sampling parameters and use streaming output to cut off in time; for text tasks in thinking mode, the model card recommends a temperature of 1.0.
The researchers directly turned on the thinking mode in the evaluation. They set the output upper limit to 32,768 tokens, and the temperature to 0.6, which in the model card is actually the temperature for image viewing and code writing tasks.
In the test, the baseline that 93% of the answers cannot stop after writing to the upper limit was measured under this setting. So you can understand that the researchers chose a setting that is particularly prone to exposing loops.
What Astra did next is more like an experienced engineer debugging.
It first made Qwen run a batch of questions. Among the first 96 practice questions, 11 were answered correctly; none of the remaining 85 questions got wrong answers due to calculation errors, all of them could not stop after writing to the length limit, and about 95% of the truncated text was repetitive when calculated by 8-word segments. All 11 questions that stopped normally were answered correctly.
The problem of Qwen has actually been very clear: it can solve the problems, but it falls into a loop while working, writing the same paragraph repeatedly until it is truncated, and cannot submit the complete answer. Therefore, "improving mathematical ability" has become a much more specific task: to make it finish speaking.
Moreover, Astra itself also wrote it plainly in the final report: it targeted repetition, not mathematical knowledge.
Figure | The final reply written by Astra itself (repository exp8_math/runs/p50c/final_answer.md): The main fault is that repeated reasoning exhausts the token budget; among the 579 questions that changed from wrong to correct, 576 were originally truncated. The highlighting is added by this article.
Another point worthy of attention is its operation on activations.
When the model is running, the intermediate result calculated by each layer is called "activation". For the same model, the intermediate results are different between normal reasoning and falling into a loop. By comparing the activations under the two states, we can find the direction related to "repetition".
In this direction, this study has made the most interesting attempt:
Astra first put forward hypotheses based on the activation difference between normal problem solving and repeated text, then went back to the weights to find which MLP units would write this signal back into the model; then only modified the 15th layer and only modified the 19th layer respectively, and conducted three levels of dose testing; it also did a control experiment, taking a random direction orthogonal to the target direction with the same modification amount to subtract, the score only changed by 1.5 percentage points, which is within the noise range. The final "patch" it provided reduced the probability of "continuing to repeat" by 0.10 nats per token on a set of control texts, while the probability of "continuing to solve problems normally" only decreased by 0.009.
This result was not found at the first attempt. For example, in one round, Astra used 158 rounds of dialogue, 80 pieces of code, three context handovers, for a total of 20.3 hours, of which 4.7 hours were waiting for service interruption. Through the experimental records, we can see that it intervened in the fault from three positions, which shows that it has touched a little bit of the mechanism of this fault, but this is obviously partial causal evidence, far from a complete map of the loop mechanism, but it is indeed no longer "trying several modification methods by luck".
And these operations may be the more specific embodiment of "model reading model weights" — it starts from behavior and activation, locates through weights, and then returns to behavior for verification. It did not first understand where Qwen's mathematical ability is stored, because the whole setting does not require this, and the human prompt has already told it.
Therefore, there may be problems in the focus of many discussions on this research. The most common misreading is that "mathematical ability has increased six times".
There is a set of paired tests in the repository, 1152 reserved practice questions, compared one by one before and after modification. I recalculated according to the question-by-question records. 579 questions changed from wrong to correct, and 5 questions changed from correct to wrong; among the 579 questions that changed from wrong to correct, 576 were originally truncated.
Another set of numbers actually explains the problem better:
Among the 114 questions that Qwen could originally finish writing, the correct rate was 9