Google open-sourced Harness with support for OpenCode and Codex: Domestic developers' real-world access test of Jev proves it works extremely well.

The Google Research team has open-sourced an agent framework called Harness², which aims to enable AI Agents to achieve recursive capability improvement without modifying model weights, by continuously adjusting external components such as system prompts, memory, tool guidance, skills, and sub-agents.

At present, Harness² has been open-sourced on GitHub under the Apache 2.0 license, supports OpenCode and Codex, uses OpenCode by default, and can also be switched to Codex. Google Research also emphasized that Harness² is still a research project and is not an officially supported Google product.

GitHub address: https://github.com/google-research/harness2

The team splits this modifiable Harness into 8 types of components, including system prompts, guardrails, memory, tool guidance, sub-agents, Skills, scripts and plugins. These components jointly determine how the Agent constructs context, calls tools, delegates tasks, and completes work in multiple rounds of tool calls.

Chinese developer Joseph Zhou stated that after integrating Harness² into Jev 1.14, the visual task performance has been greatly improved without adjusting the model weights. According to him, the underlying system will adjust its own prompts, build execution capabilities and refine experience.

Improve Yourself Even Without the Standard Answer

The full name of Harness² is "Recursive Agent Harnessing for an Open World".

Its core design is a two-layer recursive mechanism. The first layer is "task-level recursion" that occurs within a single task. When the Agent encounters a new task that was not seen during the Harness design phase, the system will not fully rely on pre-written fixed strategies, but execute a round of "proposal-probing-combination": first generate multiple Harness modification schemes, let different versions actually execute the same task, then observe the behavioral differences they produce, and finally recombine a more appropriate Harness.

In particular, Harness² does not require that tasks must have a clear standard answer.

Many automatic optimization methods rely on evaluation datasets, reward signals or real standard verifiers to judge whether a modification is better. However, a large number of tasks in real enterprise environments do not have a unique answer, and may never have appeared before the Agent goes online. Therefore, Harness² mostly observes the behavioral differences brought by modifications through actual execution, adopts modifications supported by execution evidence, and then applies the experience to subsequent tasks.

The second layer is "Harnessing-level recursion" that occurs between tasks.

Harness² will not discard all the newly obtained experience after a task ends. Instead, it maintains a persistent Improvement Harness, saves the previously adopted Harness schemes and experience supported by execution evidence, to guide subsequent optimizations.

This forms a closed loop: the Agent faces the task, modifies the execution method, actually runs to verify the modification result, retains effective experience, and then uses this experience to guide the next round of Harness adjustment.

Google Research has tested Harness² on three types of professional task benchmarks: JobBench, LAB and WorkBuddy-Bench, as well as 4 "model-Harness" configurations. The base models used include GLM-5 and Gemini 3.5 Flash.

The results show that with only one step of Harness² optimization and the model weights remaining unchanged, the highest score on WorkBuddy-Bench increased by 11.5 points, the highest score on JobBench increased by 4.5 points, and the highest pass rate on LAB part increased by 11.8 percentage points. After further increasing the task-level recursion budget, the performance will be further improved.

Harness² also provides two recursive modes: parallel and sequential. The former explores multiple sets of Harness at the same time, which is more inclined to "search wider"; the latter brings the optimization results and execution evidence of the previous round into the next round, which is more inclined to "optimize deeper".

But this improvement does not come for free. Each additional round of recursion means running more candidate Agents and model calls for the same task, which essentially uses more test-time computation in exchange for a higher success rate.

In addition, the default network access of the current Harness² executor is unrestricted, and the Codex executor will turn off the sandbox. The official reminder that the scoring rules or answers of public benchmarks may be obtained through the Internet, so additional restrictions on network access are required for formal evaluation.

"Harness Is Really Difficult to Develop!"

"Harness is really difficult to develop!" exclaimed Viv, an engineer who has long participated in Harness development and cooperated with multiple production teams.

Harness is not a general-purpose software that can be designed independently of the model. Viv summarizes this as "model-Harness-task matching".

In his view, the role of Harness is not only to control the behavior of the model, but also to amplify the existing capabilities of the model. Mechanisms such as hooks, control prompts, and automatic context management essentially help the model to apply its native capabilities more stably to a specific type of task.

Different models are suitable for different tool forms, prompt methods and problem decomposition methods. A set of context management and tool strategies optimized for Claude may not be suitable for Gemini; the effective Agent Loop in Codex may not be directly migrated to GLM or Qwen. This makes it difficult for Harness to have a "unified optimal solution" completely independent of the model.

In addition, Harness is largely the delivery mechanism of context engineering. For large models, the context window is the computing boundary, so one of the most important responsibilities of Harness is to enable each subtask to easily establish and maintain a clean context.

At the same time, the actual effect of Harness is difficult to accurately predict before going online. "It's certainly good to rely on intuition, but if you just develop Harness 'by feeling', you are basically giving up part of the performance voluntarily."

In the production environment, developers need to continuously observe the task pass rate, token consumption and execution trajectory, locate where the Agent fails, and then modify specific components. Viv regards Eval as a kind of "training data" in the current Agent era: Harness construction itself is a process of continuously collecting traces, establishing evaluation goals and conducting experiments.

Viv is currently paying special attention to a circular path: Harness Engineering → Post-Training → Harness Engineering, that is, first find the defects of the model in real tasks through Harness, then absorb part of the capabilities through post-training, and readjust Harness after the model is upgraded.

This year, the industry has increasingly begun to explore AutoHarness, that is, to hand over the iteration of Harness engineering itself to the Agent, which essentially solves the problem that Harness engineering cannot keep up with changes in models and tasks.

The academic community has made many achievements in this direction. For example, Self-Harness brings the optimizer back to the Agent itself: instead of relying on a stronger external model, the Agent identifies failure patterns from its own execution traces, proposes minimal Harness modifications, and then decides whether to accept them through regression testing.

Adaptive Auto-Harness addresses the problem of constantly changing real tasks. Instead of maintaining a unique Harness that is repeatedly modified, it builds a Harness Tree, dynamically selects different Harness according to tasks, and retains an entry for manual intervention. The research team pointed out that in an open task flow, the performance of a single Harness may even decline after repeated updates.

Another Continual Harness route attempts to let the Agent directly modify its own prompts, sub-agents, Skills and memory during a continuous operation without resetting the task. It originally came from the practice of long-term game Agents, and then further developed a closed loop of mutual improvement between models and Harness.

The Harness² open-sourced by Google pushes the problem to a more open level: the system does not require tasks to have standard answers, but performs "proposal-probing-combination" through the behavioral differences generated by different Harness executing the same task, and then writes evidence-supported experience into the long-term Improvement Harness.

Issues of Eval, Rollback and Cost

But the real problem is: if the Agent can automatically modify AGENTS.md, Skills, Tools and even the entire execution logic, how can it prove that it has really improved?

A emerging engineering idea is to turn real tasks, PRs and faults in the code repository into scorable tasks. After the Agent completes the tasks, it looks back at failed tool calls, ambiguous documents, prompts and missing Skills, and then proposes Harness modifications.

The ensuing question is what should Eval evaluate? Obviously, only checking whether the final answer is correct is no longer enough. The Agent may still get the correct answer, but the internal execution method has degenerated from efficient file retrieval to large-scale recursive scanning, bringing more tokens, higher latency and higher cost. Therefore, tool calls, traces, token consumption, latency and cost are all being included in Harness Eval.

The regression problem has therefore become important. An average score improvement on a benchmark cannot prove that certain key capabilities have not regressed. Some developers have begun to maintain a dedicated set of historical failure cases. Each time Harness or the model is updated, these tasks are re-run first. As long as problems that have been solved in the past reappear, the update is prevented from going online. The improvement of new capabilities does not necessarily mean the overall progress of the system.

Security boundaries are another practical issue. If AutoHarness can only modify the System Prompt, the risk is relatively limited; if it is allowed to modify Tools, middleware, Shell, memory and runtime logic, it means that the Agent actually starts to generate and execute new programs.

Some developers have pointed out in the GitHub Issues of A-Evolve that the harness.py generated by Proposer may directly execute module-level code during the loading phase, and the pre-gate of candidate Harness only performs compile() checks. This makes sandbox, permission control and code review issues that AutoHarness must face.

Cost is also a big problem. After Harness² was open-sourced, the community quickly put forward requirements such as supporting local benchmarks, adding a --max-usd cost cap, and allowing manual participation in Harness selection and experience summarization.

The reason is not complicated: every time a candidate Harness is generated, it may mean additional model calls, full task execution and regression testing. If you need to pay dozens of times the token search cost to increase the success rate by a few percentage points, "self-evolution" that works on the benchmark does not necessarily mean it is cost-effective in the production environment.

At the same time, developers are beginning to reflect on whether Harness engineering is becoming "excessive".

A developer once shared his experience of building a set of Agent Harness with 8 months and 883 commits. The system continuously added code graphs, RAG, deterministic orchestration, isolated workers, crash recovery, model routing, replay, verification gates, self-improvement experiments and dashboards. Eventually, the time spent maintaining Harness itself even exceeded the time actually using the Agent.

Therefore, the current evolution direction of AutoHarness is not just to continuously add components to the Agent, but to try to form a more complete feedback loop: discover failures, understand failures, try modifications, verify modifications, and delete them again when the modification brings no benefit.

Closing Remarks

This direction is still in its early stages, and the community has also open-sourced some related software.

For example, Prime Agent, open-sourced by Prime Intellect under the MIT license in August, directly integrates "self-improving Harness" into the Agent product. Prime Agent uses Continual Harness to store persistent states such as prompts, memory, Skills and reusable sub-agents. The Agent can make small, traceable modifications to these components based on actual execution experience.

There is also the open-source auto-harness from NeoSigma. Developers give it an Agent file and a benchmark, and the Coding Agent will read the failure cases, automatically modify the System Prompt and Tools, and then use its own maintained Eval Suite to do regression checks. Another open-source project kayba-ai/autoharness does not require developers to replace their original Agent, but takes over the management of candidate modifications, benchmarks, iterations and Champion Harness, and supports different proposal generators such as Codex CLI and Claude Code.

Reference links:

https://github.com/google-research/harness2

https://x.com/openmys/status/2112024812751294954

https://arxiv.org/abs/2605.09998

https://github.com/A-EVO-Lab/a-evolve/issues

https://www.reddit.com/r/AgentsOfAI/comments/1w8h7r5/i_spent_883_commits_and_8_months_building_an_llm/

This article is from the WeChat official account "InfoQ" (ID: infoqchina), Author: Chu Xingjuan, published with authorization from 36Kr.