A team of researchers from Google Cloud AI Research, Washington University in St. Louis and UNC Chapel Hill has released EnvHarness, a programmable layer that turns a static agent benchmark into one that adapts to the policy training on it. LLM agents now learn less from curated text and more from interactive environments, but those environments are hand-built and frozen: they behave identically no matter which agent is acting or how much it has improved. The usual fix is to generate new environments, which pins you to domain-specific pipelines and LLM-written verifiers that have to be over-generated and filtered. EnvHarness inverts the move. It wraps an existing environment in plug-in components that operate strictly through the standard reset() / step() interface, changing where an episode starts, what the agent may do, and what it sees, while the underlying simulator, tasks, and human-built verifier stay untouched. An LLM designer called EnvRigger writes those wrappers automatically against flaws it diagnoses in the policy’s own rollouts. Across five benchmarks in four domains, skills mined this way gain up to 9.0 points on held-out tasks with 9.8% fewer execution steps.
Is it deployable?
Yes, if you already run an agent eval loop. EnvHarness ships as Apache-2.0 Python with reproduction drivers for six environments. A new benchmark joins by implementing one interface (reset / step / observe / evaluate / get_env_state / save_state / from_state); nothing downstream changes. The hard prerequisite is a resettable environment, which rules out live user accounts and physical robots.
Environments that stop teaching
LLM agents now learn less from curated text and more from interactive environments. Those environments are hand-built and static: they behave identically no matter which agent acts or how much it has improved, so they cannot target a policy’s weakness and have nothing left to teach once solved.
The usual answer is generating more environments. The EnvHarness paper names two costs: generation pipelines are domain-specific and do not transfer, and LLM-written verifiers must be over-generated and heavily filtered without ever being fully trustworthy.
Wrapping, not authoring
The research team proposes the opposite move. An agent harness makes a frozen LLM capable through plug-in tools, memory and skills. EnvHarness applies that idea to the other side of the loop, wrapping a frozen environment in plug-in components that operate strictly through the standard reset() / step() interface.
Formally, a component is a transformation E' = w(E) that rewrites the state, action, observation and transition terms. The reward term is deliberately left out. Because no intervention reaches the simulator backend, every reshaped task keeps its original, human-built verifier, and because nothing touches benchmark-specific code, one implementation covers every domain.
Three components ship, and they compose freely:
- Stage replays a fixed action list after
reset(), so the episode starts somewhere else. Hiding the target mug in a closed drawer forces search instead of reach. - Contract installs per-step hooks on the action, transition and observation axes: block an action, rewrite a response, truncate an observation.
- Chain composes a second environment into the same episode under a shared step budget, with the composite verdict being the conjunction of both verifiers.
EnvRigger: the designer loop
Components are policy-agnostic; choosing them is not. EnvRigger treats the policy as a black box and runs four stages: it observes five baseline rollouts, diagnoses a systemic flaw, writes components as real Python, and validates on five fresh rollouts. Unsolvable and trivially solvable candidates are both rejected, with up to five revision rounds per task. Generated hooks compile in an isolated subprocess, so a bad mutation becomes a recorded trace rather than a dead run.
