MIT's overhyped masterpiece has been thoroughly debunked: Did Astra heavily modify Qwen's weights? That's nothing but empty boasting.

Did Astra modify the weights of small models? The viral MIT research has been publicly questioned by a top industry expert.

Astra just went viral across the AI community a couple of days ago with a stunning capability — it can directly modify the weights of another AI model without training or calculating gradients.

The MIT team claimed that Astra only modified 0.016% of the weights of Qwen3.5-0.8B, which boosted its math performance from 7.9% to 48.1%, equivalent to more than 6 times the original score.

Now, the situation seems to have reversed.

A well-known influential figure in the industry published a long blog post, directly pointing out:

This research is seriously misleading.

The questioner is Matthew Honnibal, the author of spaCy and co-founder of AI company Explosion.

The Magician's Trick

What on earth happened behind this?

Honnibal used a brilliant metaphor to satirize what the MIT research did: it was a sleight of hand.

Stage magicians often perform the same trick in multiple ways.

The first time, he might use a special prop; the next time, he will show you that the prop is not special at all, but he has hidden something himself; the third time, he might use some kind of forced choice.

When you watch the performance, you assume there is only one technique, so every time the magician performs it, it seems to overturn your assumption about how it works.

Honnibal believes that the experimental presentation of Astra's research is very similar to such a magic show.

Back to the paper, the original text actually demonstrates three distinct capabilities:

Reading weights: By only looking at the raw weight numbers, it can identify what task a small neural network is performing.

Writing weights: Without relying on environmental feedback, it can directly generate or modify the parameters of a small network from scratch, so that it can play small games like Breakout.

Repairing large models: By modifying part of Qwen's weights, it can significantly improve its performance in math tests.

But the core impression the paper's abstract gives people is —

Astra can understand and modify another model "only by observing the raw weights".

If you just skim through it, it's hard not to interpret it as:

Whether it is identifying the function of a small network or fixing the mathematical ability of a large language model, it all comes from the same "brain-reading" superpower.

But is this true?

After carefully sorting out 8 sets of experiments in the paper, Honnibal found that the experimental conditions for achieving these three things are vastly different.

In other words —

It's completely a fallacy of equivocation.

Two Technical Paths

What kind of flaw has been exposed (just a playful joke)?

First of all, in experiments such as identifying small network tasks and fixing a stuck racing car model, Astra did achieve the goal of "only looking at the raw weights".

Because the experimental conditions are very strict:

The raw float weight numbers are directly used as input and put into Astra's context.

Astra is not allowed to run Python code.

Astra is not allowed to execute the target network to observe its behavior.

In this mode, the scale of model parameters processed by Astra is very small, ranging from 17,000 to 172,000. This is indeed remarkable, but there is still a long way to go before truly understanding a modern large language model.

But when it comes to the most eye-catching and exciting Qwen math capability repair experiment, the situation is completely different.

When repairing the 800-million-parameter Qwen model, Astra did not directly read the raw weights.

Instead, Astra was authorized to use Python programming, and could perform up to 50 forward tests. Each time, the model is allowed to process a batch of practice questions, and analyze and modify the weights by observing the model's performance.

These are two completely different things...

Honnibal pointedly pointed out that packaging these two methods together makes it easy for readers to mistakenly think that:

Oh, the "brain-reading technique" implemented on small models can also be applied to large models with 800 million parameters at scale.

But the truth is, when dealing with a real large language model, Astra is more like an "off-site coach", who needs to intervene through a series of engineering methods such as programming, running, and observation.

The Real Value

So, does this mean that MIT's research is worthless?

Absolutely not.

It must be admitted that even without the aura of "mind-reading skills", the capabilities demonstrated by Astra are still very remarkable.

Honnibal also confessed that prompting an AI model to design and execute tasks that human machine learning engineers can complete — writing code snippets to analyze or even directly modify network weights — is a huge progress in itself.

AI models from a year or two ago would fail at all these tasks.

This shows that today's top large models already have fairly complex tool usage and reasoning capabilities, and can act as an "AI engineer" to diagnose and repair other models. This itself is an engineering breakthrough worthy of attention.

The real problem lies in the way the results are presented.

Two completely different capabilities are vaguely packaged into a unified "superpower" that can read and write the AI's brain.

To be honest, at first glance, it sounds like something that OpenAI would do...

This also reflects the current hype frenzy in the AI industry.

Source: https://honnibal.dev/blog/astra-read-write-misleading

This article is from the WeChat official account QbitAI, author: Jay, published with authorization from 36Kr.