Coding agents now run for hours, not minutes. Every edit, test run and log read goes back into the model’s context. A team of researchers from NVIDIA, NTU and MIT have released SoL-Pi, a set of 4 efficiency mechanisms for the open-source Pi coding agent. An AI found these mechanisms by running auto-research loops at the harness layer. On the 51-task EdgeBench evaluation, SoL-Pi cuts recorded token traffic by 44.7% to 49.0% versus Pi and cuts API cost by roughly 33%. Its scores stay close to Pi on both GPT-5.6 Sol and Opus 5.
Is it deployable? Yes. SoL-Pi ships on GitHub under NVlabs as an MIT-licensed extension that runs on an unmodified Pi release. It is tested with Pi 0.85.1 and Node.js 22.19 or newer.
Why Target the Harness
Most efficiency work lowers the cost per token through faster kernels, quantization or cheaper models. SoL-Pi instead reduces how many tokens a task consumes. The harness is the layer that handles tool calls, context, observations and delegation.
Tuning a harness by hand is slow, and its parts are coupled: a fix in one place can push cost into later steps. Meta-Harness and similar systems automate this work. However, a recent study found that evolved harnesses can overfit their search tasks and give only marginal gains on unseen ones.
How the Search Works
A research AI observes execution traces from a separate agent running base Pi. It then proposes harness changes and tests them. The search covered:
- 152 proposed directions across 6 families: context, progress, tools, delegation, prompt and policy, and improvement and evaluation
- 535 executable environments: 495 built from GitHub issue-pull request pairs and 40 synthetic tasks with executable verifiers
- More than 3,000 runs and 60,000+ agent-environment interactions
Each search is a disposable, isolated loop. It follows the autoresearch cycle, extended with a Ralph Loop implementation step and an independent reviewer.
Acceptance rules are fixed before the search starts, and the optimizer cannot change them. Every capability metric must stay within a predeclared tolerance. The candidate must also improve at least 1 efficiency metric. EdgeBench stays held out. Of its 51 public tasks, 11 are used for one-way acceptance of frozen candidates and 40 for final evaluation. Held-out results never feed back into the search.
The 4 Mechanisms That Survived
- Action Fusion: Base Pi often edits a file and then issues a separate command to test, build or run it. Action Fusion merges both into 1 tool request and returns both outcomes in 1 observation. This removes a model round trip.
- Online Context Compact: Plan steps are tracked through
update_plan. When a step completes, the harness estimates how many requests remain. It then compares the projected input savings with the extra cost of rewriting the prompt cache. It invokes Pi’s native compaction when this gate passes or when context nears the window limit. - ObservationPack: Tool outputs above 10 KiB are archived locally and sent in full for the next 2 provider requests. From the 3rd request onward, the model sees a stable handle, the original size and a short excerpt of head and tail lines. Exact pages stay retrievable through the handle.
- Evidence-Preserving Reducer: Build and test logs of at least 4 KiB go to a cheaper model, GPT-5.6 Luna at
high, which writes a compact receipt. A deterministic verifier checks the receipt’s schema, source hash, exit status, exact quotes and size. The harness falls back to the original log in 3 cases: verification fails, credentials are suspected, or the receipt is not smaller.
