HomeArticle

No training required, Opus 5.5 directly handwrites Transformer weights and achieves 100% performance on all 13 types of tasks.

新智元2026-11-19 08:54
Microsoft team rolls out iWSP, enabling large language models to handwrite transparent neural network weights.

Across 13 types of complex algorithm tasks and 26 independent runs, a brand-new neural network has achieved 100% accuracy on all hidden test sets.

Even more incredibly, the entire process did not feed it massive volumes of data, and did not even use "backpropagation" or "gradient descent" at all.

This crystal-clear, extremely compact full-mark neural network was written line by line directly in code by Claude Opus 5.5.

Recently, teams from Microsoft Research, Johns Hopkins University, Columbia University and New York University published a new study "Interpretable Weight Space Programming (iWSP)".

This study proves that cutting-edge large language models can already bypass traditional optimizers directly, precisely conceive computational logic like a surgeon based on fixed evaluation feedback, and write algorithms directly into the weights of the Transformer with code.

No Backpropagation Needed: AI Acts Directly as a "Clockmaker"

Deep learning is often jokingly referred to as "alchemy".

The whole process is like a group of blind people surrounding a huge console covered with hundreds of millions of knobs. No one can see what the inside looks like, they can only fumble to twist this and turn that based on the occasional "error beep" from the machine.

After tens of thousands of iterations, the machine actually runs normally, and even outperforms humans on certain tasks, which can be described as "ascending to the sky on the spot".

But if you ask one more question: which knob controls logic? Which knob controls memory? No one can answer.

In short, the effect is outstanding but the principle is unknown — twisting the knobs correctly is metaphysics, twisting them wrong is an accident, and there is an entire unexplainable black box in between.

In engineering applications, this is barely acceptable as long as the predictions are accurate. But in extremely rigorous scientific explorations such as neuroscience and physics, this flaw is too fatal.

As the core pain point pointed out by the research team:

Neural networks make excellent predictions but are very difficult to understand. In scientific fields where "explanation" is the ultimate goal, this is a serious problem.

Taking neuroscience as an example, deep encoding models can very accurately predict the brain's responses to natural stimuli, but they cannot tell at all exactly "which features" in the stimuli drive these responses.

To break this deadlock, the Microsoft team proposed iWSP. They decided: no backpropagation is allowed, and AI is directly tasked with setting those knobs.

In the iWSP framework, Opus 5.5 acts as a creator.

It completely subverts the traditional workflow:

Pure Reasoning: Faced with tasks, the system does not allow Opus to perform any gradient-based training. In each iteration, the agent can only see a single Validation score.

Handwritten Weights: Opus must call its existing prior knowledge to conceive a set of computational logic. Then, it directly writes Python code, explicitly writes out the features, and converts this set of logic into specific weight numbers.

Closed-loop Iteration: Opus evaluates these features, reuses known correct modules, and continuously modifies the code based on the feedback of that single score.

On average, each run takes around 39 rounds.

The final result is a "transparent" Transformer that humans can read from beginning to end like an ordinary book.

Every part of it, every line of its weights, is intentionally "written" by AI.

Counterintuitive Truth

The most stunning part of this experiment is not only that it successfully made AI create AI, but also that it revealed a counterintuitive truth.

In the traditional AI context, we firmly believe in "greater effort brings greater miracles". Hundreds of billions of parameters are just a starting point, and trillions of parameters are the future.

Everyone thought that the larger the neural network, the more capable it is.

But the data in this paper directly reverses this common sense.

In 13 types of algorithm experiments, all models "hand-crafted" by AI finally achieved 100% accuracy on the hidden test set.

But when we compare different levels of models in the Anthropic family, a magical phenomenon appears: when writing algorithm problems to get full marks, the stronger the AI, the smaller the network it writes.

The relatively basic Haiku 5.5 handwrites a network with a median parameter count of as high as 100,000.

The mid-tier Sonnet 5.5 requires a median parameter count of 41,600.

The current strongest Claude model, Opus 5.5, has a median number of handwritten network parameters that is only an extremely staggering 166!

It is recommended that everyone type "166 parameters" on the public screen. In this era of large models with hundreds of billions or trillions of parameters at every turn, 166 parameters are just like a speck of dust.

Why is there such a ridiculous gap?

The log records Opus's "mental journey": Opus's strategy is to first find a way to write correctly, then delete it to make it smaller round by round.

At the beginning, to ensure that the logic can run, Opus will also first write a network with approximately 3400 parameters.

After getting a full score of 100% on the visible set, it began to drastically streamline the code. It removed redundant calculations, compressed complex matrices, and forcibly reduced the average 3400 parameters to around 200.

This is like the difference between a top student and a poor student.

The poor student (Haiku) does not understand the principle behind the formula. When encountering problems they cannot solve, they rely on rote memorization to stuff all possible situations into their minds, so they need a "large notebook" with 100,000 parameters.

The top student (Opus) has thoroughly seen through the essence of the algorithm and directly extracted the most core calculation rules, so they only need a "small note" of 166 parameters.

This discovery is extremely shocking. It proves that if the truly correct logic can be found, there is no need to stack resources to solve complex problems.

From Mathematical Problems to the Human Brain, Saving Science from "Being Fooled"

You may ask: What is the use of going to so much trouble to let AI hand-craft several tiny models that solve math problems?

The answer is: For real scientific discoveries.

The scientific community is currently facing an awkward dilemma. Take neuroscience as an example. To understand the human brain, scientists use ready-made deep learning black-box models to predict the brain's responses.

Indeed, the model predicts very accurately, but scientists are dumbfounded: I use a black box that I cannot understand to predict another black box (the brain) that I cannot understand. What is the use of correct predictions? I still don't know how the brain actually works!

What we need is "explanation", not just "prediction".

The value of Microsoft's iWSP method comes into full play here.

Because the internal logic of the network it creates is transparent. The research team directly applied this method to extremely difficult brain data (Brain Encoding Models): In language fMRI (functional magnetic resonance imaging) experiments, iWSP has made great contributions.

Faced with complex brain datasets, if trained with traditional backpropagation, black-box models will secretly exploit flaws in the data to cheat in order to get high scores.

Scientists cannot detect it at all.

But in iWSP, Opus 4.7 explicitly writes the model step by step. When analyzing this fully transparent model, scientists immediately found something wrong: This model accurately identified a confounding loophole in the dataset!

It directly puts the problem on the table.

In addition, in the visual cortex task, iWSP not only outperforms existing interpretable models in prediction performance, but also clearly demonstrates how the brain performs different calculations in early and advanced visual regions.

On auditory ECoG (electrocorticography) data, it systematically discovered and defined those "interpretable acoustic features" that can predict neural activity.

Through iWSP, AI has truly become a microscope that helps scientists discover the truth.

The Dawn of Autonomous AI Evolution

Reading this far, many people may excitedly shout: The "alchemy era" of deep learning is over! In the future, training large models will no longer require tens of thousands of NVIDIA H100 graphics cards, just let super AI write the weights!

Hold on.

Although iWSP is extraordinary in pure algorithm tasks and brain response prediction, it hit a wall in Natural Language Processing (NLP) tasks.

In NLP tasks such as sentiment classification, human language features are highly ambiguous, unstructured, full of humor, irony, and polysemous words, and there are no absolutely clear "mathematical formulas" or "computational modules" that can be applied.

Faced with such tasks, Opus 5.5 appears quite struggling.

To achieve a certain level of accuracy, it has to desperately add features, resulting in the model's parameter count being forced to expand to around 190,000. Even more cruelly, when the parameters rise to 190,000, the sentiment classification only reaches 72%; among the four language tasks, the paraphrase judgment task does not even exceed the baseline of "always guessing the most common answer".

This points out the core limitation of the current iWSP method: It is currently mainly good at "recombining known algorithms and computing modules".

The emergence of complex representations still relies on massive data and gradient descent for now.

Therefore, it must not be concluded from this that general large models (AGI) no longer need training.

The truly underlying foundation models still require the "brute force aesthetics" of computing power.

Nevertheless, this Microsoft research is still a fascinating milestone in the history of AI.

It proves to the world that in specific scenarios, neural networks can break away from "blind evolution".

References:

https://arxiv.org/abs/2611.18114

https://github.com/csinva/weight-space-programming-autoresearch

This article is from the WeChat official account "AIEra", Author: ASI Revelation, Editor: David, published with authorization from 36Kr.