Google: Just like agents, environments can also have Harness

We have spent a lot of time equipping LLMs with harnesses to help LLMs as agents better complete tasks in complex environments. But what about the environment side?

Whether it is distillation, reinforcement learning or self-evolution, we need agents to interact with the environment, but we cannot control how the environment interacts with the agent. Analogous to the agent harness, the paper proposes Envharness, a brand-new component that controls how the environment interacts with agents, to help the environment provide better training signals for agents in various training scenarios.

Paper Title: EnvHarness: Awakening Static Worlds for Agent Learning

Paper Link: https://arxiv.org/abs/2608.19880

Project Code: https://github.com/google-research/envharness

Project Homepage: https://envharness.com/

Environment Harness and Agent Harness

In the general scenario of intelligent agents, agents need to complete tasks in an environment, which can be a web page, a code repository, or even a room.

Under the current general setting, the core of an agent is a static large language model. We add harnesses around it, such as memory and skill libraries, to help it complete tasks better.

Then as the other end of the framework, the environment can also have its own harness to control its performance when interacting with the agent.

Neither Envharness nor the agent harness touches the core part that interacts with the outside world (the large language model and the environment itself), but both can change the way the core content outputs to the outside only by interacting with the input and output interfaces to achieve our desired goals. For the environment, a better environment is one that can provide better training signals for agent training.

What is EnvHarness

According to the specification, Envharness follows the simplest interface protocol to ensure that different Envharnesses can be easily stacked on top of each other. In short, after an environment is added with an environment harness, its external interface will not change at all.

• Standard Environment: It follows three most standard interfaces: reset() is used to initialize the environment, step(a) executes a certain action in the environment, and obs() obtains the internal state of the environment through observation

This paper proposes three relatively simple Envharness components

• Environment Arrangement: Perform a series of operations after the environment is initialized to change the initial state of the environment
• Interaction Rules: Map the actions and observations made by the agent to new actions or observations
• Chained Environment: Automatically jump to another environment when the current environment meets certain conditions

Since all operations are based on the standard interface of the underlying environment, Envharness can be seamlessly installed on any environment that meets the conditions, regardless of whether the underlying environment is a website or Docker, for robot training or GUI training.

How to Design EnvHarness

Back to the purpose of Envharness, which is to enable the environment to provide better training signals. Referring to the self-evolving agent that reads actual trajectories to optimize the harness, the paper proposes using an agent to observe trajectories to generate Envharness.

The entire process is a cycle consisting of four simple steps

• First, read n trajectories of the agent in the original environment
• Identify whether there are any problems or areas for improvement of the agent based on these trajectories
• Then try to generate an initial Envharness
• Let the agent try to generate a new batch of trajectories again in the environment added with the above Envharness. After observation, if it is found that the Envharness can help the agent solve the problem, it will be adopted; otherwise, go back to the previous step with these trajectories

Through such loop engineering, we can generate the corresponding Envharness according to the current capability of the agent.

Experimental Results

The paper studies that in the scenarios of reinforcement learning and self-evolving agents, the environment added with Envharness can provide better training signals to help improve the performance of agents. At the same time, the performance on different benchmarks such as embodied intelligence, web navigation, and code generation also demonstrates the versatility of this method.

Other experiments discuss directions such as co-evolution with agents and targeted adaptation to long-horizon problems.

Core Limitations and Future Outlook

The paper lists many existing shortcomings and future directions for Envharness, for example:

• Just as the agent harness includes a variety of components, Envharness also requires a variety of dedicated components to solve specific problems such as long-horizon tasks, domain knowledge integration, and long context processing, which are waiting for contributions from researchers in the future
• More efficient Envharness design methods. The existing methods rely on multiple rounds of trajectory sampling, which are low in efficiency and cannot support super large-scale environment enhancement research

The first author of this paper, Chengsong Huang, is a PhD student at Washington University in St. Louis, with more than 1500 Google Scholar citations. His current research interests mainly focus on self-evolving agents and data synthesis. His previous work Lorahub and R-zero, which were reported by Machine Heart, have both gained more than 700 stars on GitHub.

This article is from the WeChat official account "Machine Heart", and is authorized for release by 36Kr.