Markdown edition

# SoL-Pi reduces coding-agent token traffic through four harness mechanisms

> On 51 public EdgeBench tasks, SoL-Pi's authors report 44.7–49.0% less token traffic and about one-third lower estimated API cost than Pi. At xhigh, the full stack's mean score fell from 44.833 to 42.003 with GPT-5.6 Sol and from 44.756 to 42.224 with Opus 5.

By BIGCHANGE.AI Editorial

Published: 2026-10-24T22:23:33.581Z
Updated: 2026-10-24T22:23:33.581Z
Canonical: https://bigchange.ai/blog/sol-pi-coding-agent-harness-token-traffic

![Conceptual charcoal illustration of a long fanfold diagnostic log retained in a metal tray beside a short receipt-sized sheet in its front slot.](https://bigchange.ai/api/media/file/sol-pi-retained-log-receipt-hero-v2.png)
AI-generated conceptual illustration by BIG CHANGE; physical analogy, not actual SoL-Pi equipment, output or a benchmark result.

Researchers behind [SoL-Pi](https://arxiv.org/abs/2609.20519) report that four changes to the Pi coding-agent harness cut recorded token traffic by 44.7% to 49.0% against Pi across two model backends on 51 public EdgeBench tasks. The full stack's estimated API cost was about one third lower. Its average task score fell from 44.833 to 42.003 with GPT-5.6 Sol and from 44.756 to 42.224 with Opus 5. The September 17, 2026 paper measures a research configuration; BIG CHANGE has not run the extension or independently reproduced the benchmark.

The savings come from changing the software that presents tools and prior work to a model, without changing its weights. The authors searched across hundreds of executable development environments, then tested a frozen harness on separate tasks. [The released code](https://github.com/NVlabs/SoL-Pi) packages the surviving mechanisms as a standalone, MIT-licensed extension for Pi. It is not an official Pi distribution.

## The big change

- **What changed:** A coding agent using Pi can add four harness mechanisms that join an edit with its next command, consider compaction at completed plan steps, substitute handles for older large observations, and verify short receipts from diagnostic logs.
- **Why it matters:** The paper reports lower token traffic and estimated API cost on EdgeBench. The public extension lets Pi users evaluate the approach on their own workloads without changing the underlying model.
- **What to watch:** The full stack also recorded lower mean task scores than Pi on both evaluated backends. The reducer can send eligible logs to its configured provider, and archived observations persist locally.

## Four changes around the agent loop

**Action Fusion** adds an optional follow-up command to Pi's edit and write tools. If the agent knows it should run a test or build after a file mutation, the harness performs both in one tool call and returns their outcomes together. A command that depends on inspecting the edit result still needs a separate decision. The saving is an intermediate model round trip, rather than a faster shell command.

**Online Context Compact** checks for a compaction opportunity when an `update_plan` step is completed. It estimates how many model requests remain and whether shorter future prompts would repay the cost of rewriting the cached context. It can also compact near the context-window limit. A successful compaction uses Pi's native facility and continues the task in a new turn. The paper says its gate does not separately price the summarization call. Its configured cache write/read ratio guides the decision; that ratio is an input to the rule, not a measured bill.

**ObservationPack** archives tool results larger than 10 KiB locally. The next two provider requests still receive the full result. From the third request, the model sees a stable handle, original size and short excerpt instead, with exact pages available through recall. This reduces repeated transmission of an old observation while allowing the agent to retrieve the original if the excerpt is insufficient.

**Evidence-Preserving Reducer** handles eligible build and test logs of at least 4 KiB. A lower-cost model extracts a short receipt from the archived output. A deterministic check rejects a receipt whose schema, source hash, exit status, quotations or size do not match; suspected credentials and failed reductions also leave the original log in place. File reads and search results bypass this mechanism. Reduction runs before ObservationPack, which leaves verified receipts alone.

These details matter because each mechanism has a different failure or cost boundary. A fused command could be premature when the edit needs inspection. Compaction can incur a cache rewrite. Observation handles require local archives to remain available. The reducer adds a model call and can send eligible log content to its configured provider. The [repository security policy](https://github.com/NVlabs/SoL-Pi/blob/main/SECURITY.md) says its likely-secret detector is a precaution, not a complete scanner.

## EdgeBench token, cost and score results

The authors developed mechanisms using 495 repository-derived tasks and 40 verifier-driven synthetic environments. They say EdgeBench remained outside this search. After freezing the source, configuration and acceptance rules, they used 11 of EdgeBench's 51 public tasks for one-way candidate acceptance and the other 40 for final generalization evaluation. EdgeBench has 134 tasks in all; this study uses the 51 public tasks. The [paper's tables](https://arxiv.org/html/2609.20519) report that complete public set, so all 51 should not be described as untouched final-test tasks.

With GPT-5.6 Sol at the paper's `xhigh` setting, Pi recorded 2.1538 billion total tokens, $1,339 in estimated API token cost and a 44.833 average score. The complete SoL-Pi stack recorded 1.0990 billion tokens, $894 and a 42.003 score. With Opus 5 at `xhigh`, the corresponding figures were 2.3697 billion tokens, $1,741 and 44.756 for Pi, versus 1.3101 billion tokens, $1,158 and 42.224 for SoL-Pi. The Pi baseline in each backend is the denominator for the reported 49.0% and 44.7% token reductions and 33.2% and 33.5% cost reductions.

“Recorded token traffic” combines uncached input, cache reads, cache writes and output across the evaluated runs. The paper prices those token categories at API rates dated August 17, 2026. Its token-efficiency measure is API cost divided by aggregate task score. These are paper-reported API-equivalent costs for the benchmark, not a subscription price, a measured saving for another team or a promise about today's provider rates.

The score loss is material when choosing a configuration. The paper also reports a separate “Performance” point: the highest-scoring single mechanism for each backend, selected from its add-one comparison. On GPT-5.6 Sol that was ObservationPack, with a 47.208 score and $1,271 token cost; on Opus 5 it was Action Fusion, with a 50.482 score and $1,605 cost. Neither is the four-mechanism stack. On a separate 63-task CPU-only Terminal-Bench 4 evaluation, SoL-Pi solved 15 tasks versus Pi's 18, with total API model cost of $211.12 versus $286.45. That comparison is another reason to measure completed work alongside spend.

## Installing and configuring the Pi extension

As checked on October 25, 2026, the [repository instructions](https://github.com/NVlabs/SoL-Pi#-getting-started) specify Node.js 22.19 or newer and npm. They recommend installing the tested Pi 1.3.0 release:

```bash
npm install --global @earendil-works/pi-coding-agent@1.3.0
```

Then install the SoL-Pi extension:

```bash
pi install git:github.com/NVlabs/SoL-Pi
```

For a trusted project, the repository also documents a project-local installation:

```bash
pi install git:github.com/NVlabs/SoL-Pi --local --approve
```

The [compatibility notes](https://github.com/NVlabs/SoL-Pi/blob/main/docs/compatibility.md) also list Pi 0.84.2 and 1.0.4 as tested but warn that a fresh install of earlier 1.x releases can pull mismatched internal packages. These are documented commands, not commands BIG CHANGE executed.

The input to a normal run is the coding task and repository supplied to Pi, plus a single effective `sol-pi.json` configuration. With the official Pi distribution, a trusted project's `.pi/sol-pi.json` takes precedence over `~/.pi/agent/sol-pi.json`; the files are not merged. All four features default to false. The repository's conservative example enables only Action Fusion and ObservationPack. An all-enabled setup requires explicitly enabling the reducer and Online Context Compact, and choosing the reducer provider/model route if the default is unsuitable. Pi keeps control of the main model, authentication and provider URL. The output is Pi's ordinary task work and tool observations, with session-specific local archives for large observations and reducer logs. The project provides a [full-checkout installation and validation protocol](https://github.com/NVlabs/SoL-Pi/blob/main/agents-install.md) for readers who need a reproducible all-enabled setup.

The released extension is a way to test a harness design. Its current documentation lists Pi 0.84.2, 1.0.4 and 1.3.0 as tested; other versions need verification. It also documents Pi 1.x `codemode` behavior, including that the reducer has no effect on script calls in `only` mode. Local archives persist after a session ends, and the reducer's remote path needs a data-handling decision. A useful trial would compare the same task set, model, reasoning setting and pricing assumptions under Pi and the chosen SoL-Pi configuration, recording both completed tasks and total model cost. The repository's October 13 visual and October 14 skill examples are later additions, not results from the paper's EdgeBench evaluation.

## Sources & further reading

- [SoL-Pi paper, arXiv:2609.20519v1](https://arxiv.org/html/2609.20519) describes the four mechanisms, search and held-out evaluation, benchmark scores, token categories and fixed August 17 API prices. These are the authors' reported results.
- [SoL-Pi source repository](https://github.com/NVlabs/SoL-Pi), checked October 25, documents the standalone Pi extension, current installation commands, disabled-by-default features and session archives. Its [configuration reference](https://github.com/NVlabs/SoL-Pi/blob/main/docs/configuration.md) gives the effective-file order and schema; the [compatibility notes](https://github.com/NVlabs/SoL-Pi/blob/main/docs/compatibility.md) list tested Pi versions.
- [Project page](https://nvlabs.github.io/SoL-Pi/) explains the authors' research process and figures, including the separate performance and efficiency configurations. It is a project-authored explanation, not independent replication.
- [Installation protocol](https://github.com/NVlabs/SoL-Pi/blob/main/agents-install.md) lists the full source-check and all-enabled validation procedure. The [security policy](https://github.com/NVlabs/SoL-Pi/blob/main/SECURITY.md) describes local archives, reducer data transfer and Pi's existing permissions.

## Sources

- [Liu et al., SoL-Pi, arXiv:2609.20519v1](https://arxiv.org/html/2609.20519) — September 17, 2026 primary paper for four mechanisms, 51 public EdgeBench tasks (11 acceptance, 40 final), Tables 1–4, token categories, August 17 API prices and limitations. Results are author reported; BIG CHANGE did not reproduce them.
- [NVlabs/SoL-Pi source repository](https://github.com/NVlabs/SoL-Pi) — Standalone MIT Pi extension, not official Pi. README checked October 25, 2026 for current Pi 1.3.0 installation guidance, opt-in defaults, session archives and later example dates. Earlier observed September 26 HEAD was 1559b5c; full commit ID is preserved in the internal source audit. Commands were documented, not executed.
- [SoL-Pi project page](https://nvlabs.github.io/SoL-Pi/) — Author explanation and benchmark figures; identifies xhigh backend setting. It is not independent replication.
- [SoL-Pi configuration reference](https://github.com/NVlabs/SoL-Pi/blob/main/docs/configuration.md) — Checked October 25, 2026: config lookup order, feature defaults, reducer route, cache ratio, and Pi 1.x compaction behavior. Project and user files are not merged.
- [SoL-Pi compatibility notes](https://github.com/NVlabs/SoL-Pi/blob/main/docs/compatibility.md) — Checked October 25, 2026: tested Pi versions 0.84.2, 1.0.4 and 1.3.0; fresh-install warning for earlier 1.x; codemode behavior. Documentation only.
- [SoL-Pi agent installation protocol](https://github.com/NVlabs/SoL-Pi/blob/main/agents-install.md) — Full-checkout, all-enabled source and startup validation procedure. It is documentation, not a test performed by BIG CHANGE.
- [SoL-Pi security policy](https://github.com/NVlabs/SoL-Pi/blob/main/SECURITY.md) — Local persistent archives, possible reducer log transfer, incomplete secret detection and Pi process permissions.