Google has open-sourced new RSI achievements, which boast an extremely high proportion of Chinese contributors.

Google's new RSI achievement has been open-sourced.

Named Harness², it enables large models to automatically modify their own plug-in code, system prompts, toolchains and skill libraries during runtime.

Yes, this is RSI at the Harness layer.

The performance is extremely impressive.

After GLM-5 is connected to Harness², its overall score on the office benchmark WorkBuddy-Bench has soared by 11.5 points directly.

It instantly surpasses the native Gemini 3.5 Flash.

The entire process does not require modifying the model weights, only optimizing the peripheral runtime scaffolding.

It is an incredibly powerful add-on: any model integrated with it will see its benchmark scores skyrocket immediately.

At present, this project has been fully open-sourced, and supports OpenCode and Codex.

Why "self-modification" keeps failing

In the past, to make Agents smarter, the industry has carried out a lot of Recursive Self-Improvement (abbreviated as RSI) attempts.

The idea sounds very appealing: let AI write its own code, modify its own prompts, and optimize its own system. But once deployed in the real open world, almost all of these attempts ended in failure.

Why is that?

The problem mainly lies in two fatal flaws:

There are no standard answers.

When doing benchmark tests in the lab, there are unit tests and Ground Truth to tell you whether the result is correct or not. But in real work scenarios, such as writing a business plan or reviewing a legal contract, there is no one online to score your output.

After the AI modifies a version of the Prompt by itself, how can it know whether the modification makes it smarter or just generates random nonsense?

Overfitting to a single task.

After you spend a lot of effort modifying the scaffolding to fix a specific task, when the next task is to write a web crawler, all the previously modified rules become a burden.

If you directly apply the modified scaffolding to the next task, the system will quickly fall into a noise spiral and become dumber with each modification.

The Harness² proposed by the Google team this time is specifically designed to solve these two fatal flaws.

They have built a set of two-layer recursive architecture.

Two-layer recursion: run experiments on one hand, take notes on the other

The "square" in the name Harness² refers to two nested recursive loops: Task-level recursion and Harnessing-level recursion.

The first layer, Task-level recursion: run controlled experiments without standard answers.

Faced with this new task, the system first uses the initial scaffolding to run a reference trajectory. It does not matter whether this result is correct or not, the core is to find out what defects the model exposes during execution.

Immediately after that, it starts a mechanism called Propose–Probe–Compose:

Propose:

A dedicated editing role proposes two sets of modification plans for the trajectory just obtained.

One set is "Invariant General Defect Repair", which is specifically designed to fix cross-domain common problems such as chaotic planning, infinite loops in tool calls, and context explosion;

The other set is "Domain-specific Repair", which is targeted at the specific knowledge and constraints of the current business.

Probe:

Put these two sets of modified candidate scaffolding back into the same task and run them separately.

It does not matter if there is no judge, the system captures their execution trajectories and performs Behavioral Contrast with the reference trajectory. It checks whether the modification solves the previously stuck step, and whether it causes new logical degradation.

Compose:

The synthesizer puts all execution evidence together, selects the modifications that are truly proven effective by behavioral evidence, and merges them into the final execution scaffolding.

That's not all. It also supports Parallel Recursion, which generates k groups of general and domain candidate solutions at a time, totaling 2k candidate scaffolding. After concurrent probing, k independent syntheses are performed to further expand the search space.

The second layer, Harnessing-level recursion: precipitate the experience of modifying scaffolding into meta-capability.

After the current task is completed and the deliverable is submitted, the lifecycle of many systems ends here.

But the real power of Harness² lies in what comes next —

Hand over the entire debugging record (reference trajectory, candidate modifications, comparison evidence, synthesis decisions) to the Experience Reflector module.

After reviewing, the reflection module will write an EXPERIENCE.md file, which clearly records why the modification was made, under what conditions it works, and what consequences it brings, and stores it in the Harness Improvement Library.

When the next new task comes in:

The execution end still gets a clean basic scaffolding.

But the "improvement brain" responsible for modifying the scaffolding already has a thick manual of historical experience. During the proposal and synthesis phase, it can directly retrieve effective components and pit avoidance guidelines from similar scenarios in the past.

What is improved is not only the current scaffolding, but also the ability to improve the scaffolding in the future.

This turns the temporary patch for a single task into a cognitive asset for lifelong learning.

Benchmark Test Results

No matter how elegant the mechanism is, it ultimately needs to be verified with real-world data.

The team directly adopted three highly authoritative hardcore professional benchmarks:

WorkBuddy-Bench: covers three high-difficulty office workflows: Code, Web, and Office Suite;

JobBench: long-range end-to-end complex professional post tasks;

LAB: the top legal agent benchmark open-sourced by Harvey.

The base models selected are Gemini 3.5 Flash and GLM-5, and the execution scaffolding selected are the mainstream open-source OpenCode and Codex.

The results are extremely remarkable.

In the WorkBuddy-Bench test, only single-step Harness² improvement (k=1) was performed:

The overall score of GLM-5 under OpenCode soared from 64.2 to 75.7, an increase of 11.5 points; under Codex, it also rose from 64.4 to 75.3.

Gemini 3.5 Flash improved from 72.4 to 77.5 under OpenCode, and from 74.5 to 77.5 under Codex.

Looking at this data:

The GLM-5 with optimized scaffolding (75.7 points / 75.3 points) directly outperforms the original unmodified Gemini 3.5 Flash (72.4 points / 74.5 points).

On the JobBench for end-to-end post tasks, single-step improvement also brings comprehensive gains: Gemini 3.5 Flash increased by 4.5 points, GLM-5 increased by 3.6 points; on the legal benchmark LAB, the partial pass rate of GLM-5 increased directly by 11.8 percentage points.

If the parallel budget is increased to k=5 (5 concurrent candidate paths), the performance will be further improved: WorkBuddy-Bench can increase by a maximum of 3.9 points, JobBench by 4.2 points, and the LAB pass rate by 6.7 percentage points.

What is even more impressive is the cross-domain transfer test of the experience library.

The team transferred the experience library accumulated on JobBench to WorkBuddy-Bench, and without increasing additional probing budget, assisted the single-step Harness improvement, which finally increased the scores of code, web and office scenarios by 0.6 points, 0.5 points and 2.9 points respectively.

In the past two years, almost all expectations for AI have been placed on "base model Scaling", and people always think that as long as the next generation of GPT is released, the application problems at hand will be solved automatically.

But the reality has poured cold water on all practitioners: the base model is getting more and more expensive, the marginal benefit is slowing down, and the long-tail bugs in real deployment scenarios are still endless.

No matter how large the context window you give the model, it will still be lazy when processing cross-file account reconciliation; no matter how high its benchmark score is, it will not actively open the file to verify the format after finishing writing the report.

These problems can never be solved by feeding more pre-training parameters.

What it needs is a set of Runtime cognitive engineering that can perform self-perception, self-repair and continuous evolution during operation.

In the past, when we built scaffolding, engineers sat in front of the computer writing Prompt, configuring tools and modifying pipelines line by line, which was extremely tiring and still full of loopholes.

Now Harness² has proven a new path:

Give the power to make rules back to the Agent itself.

Paper Authors

Finally, let's introduce the authors of the paper as usual.

There are 9 authors in total, and the proportion of Chinese researchers is extremely high, reaching 78%...

Due to the limited length of the article, three of them are introduced in order here.

Ruiyao Xu, the first author and corresponding author of the paper.

PhD student in the Department of Statistics and Data Science at Northwestern University, USA, supervised by Prof. Kaize Ding.

His main research directions include large language models, graph machine learning, anomaly detection and AI Agent systems.

This paper is the work he completed during his internship at Google.

Yanfei Chen, Senior Research Engineer at Google Cloud AI Research.

He graduated from Peking University with a bachelor's degree in Electrical Engineering and Economics, and obtained his PhD in Industrial Engineering from the University of Pittsburgh in 2018.

After that, he worked as a research engineer at machine learning company Qeexo, and joined Google in 2020. He has participated in projects such as Vertex AI Search, Gemini Enterprise Deep Research, and his main research focuses on large language model agents, retrieval-augmented generation and tool learning.

Zhongying CuiZhu, also known as Zoey CuiZhu, Research Engineer at Google Cloud AI Research.

She obtained her PhD in Electrical Engineering from Stanford University in 2019, and her main research directions are general agents, verifiable agents and agent quality evaluation.

References:

[1] https://arxiv.org/html/2610.14224v1

This article is from the WeChat official account QbitAI, author: Jay, authorized for release by 36Kr.