SoL-Pi reduces coding-agent token traffic through four harness mechanisms

Researchers behind SoL-Pi report that four changes to the Pi coding-agent harness cut recorded token traffic by 44.7% to 49.0% against Pi across two model backends on 51 public EdgeBench tasks. The full stack's estimated API cost was about one third lower. Its average task score fell from 44.833 to 42.003 with GPT-5.6 Sol and from 44.756 to 42.224 with Opus 5. The September 17, 2026 paper measures a research configuration; BIG CHANGE has not run the extension or independently reproduced the benchmark.

The savings come from changing the software that presents tools and prior work to a model, without changing its weights. The authors searched across hundreds of executable development environments, then tested a frozen harness on separate tasks. The released code packages the surviving mechanisms as a standalone, MIT-licensed extension for Pi. It is not an official Pi distribution.

The big change

• What changed: A coding agent using Pi can add four harness mechanisms that join an edit with its next command, consider compaction at completed plan steps, substitute handles for older large observations, and verify short receipts from diagnostic logs.
• Why it matters: The paper reports lower token traffic and estimated API cost on EdgeBench. The public extension lets Pi users evaluate the approach on their own workloads without changing the underlying model.
• What to watch: The full stack also recorded lower mean task scores than Pi on both evaluated backends. The reducer can send eligible logs to its configured provider, and archived observations persist locally.

Four changes around the agent loop

Action Fusion adds an optional follow-up command to Pi's edit and write tools. If the agent knows it should run a test or build after a file mutation, the harness performs both in one tool call and returns their outcomes together. A command that depends on inspecting the edit result still needs a separate decision. The saving is an intermediate model round trip, rather than a faster shell command.

Online Context Compact checks for a compaction opportunity when an update_plan step is completed. It estimates how many model requests remain and whether shorter future prompts would repay the cost of rewriting the cached context. It can also compact near the context-window limit. A successful compaction uses Pi's native facility and continues the task in a new turn. The paper says its gate does not separately price the summarization call. Its configured cache write/read ratio guides the decision; that ratio is an input to the rule, not a measured bill.

ObservationPack archives tool results larger than 10 KiB locally. The next two provider requests still receive the full result. From the third request, the model sees a stable handle, original size and short excerpt instead, with exact pages available through recall. This reduces repeated transmission of an old observation while allowing the agent to retrieve the original if the excerpt is insufficient.

Evidence-Preserving Reducer handles eligible build and test logs of at least 4 KiB. A lower-cost model extracts a short receipt from the archived output. A deterministic check rejects a receipt whose schema, source hash, exit status, quotations or size do not match; suspected credentials and failed reductions also leave the original log in place. File reads and search results bypass this mechanism. Reduction runs before ObservationPack, which leaves verified receipts alone.

These details matter because each mechanism has a different failure or cost boundary. A fused command could be premature when the edit needs inspection. Compaction can incur a cache rewrite. Observation handles require local archives to remain available. The reducer adds a model call and can send eligible log content to its configured provider. The repository security policy says its likely-secret detector is a precaution, not a complete scanner.

EdgeBench token, cost and score results

The authors developed mechanisms using 495 repository-derived tasks and 40 verifier-driven synthetic environments. They say EdgeBench remained outside this search. After freezing the source, configuration and acceptance rules, they used 11 of EdgeBench's 51 public tasks for one-way candidate acceptance and the other 40 for final generalization evaluation. EdgeBench has 134 tasks in all; this study uses the 51 public tasks. The paper's tables report that complete public set, so all 51 should not be described as untouched final-test tasks.

With GPT-5.6 Sol at the paper's xhigh setting, Pi recorded 2.1538 billion total tokens, $1,339 in estimated API token cost and a 44.833 average score. The complete SoL-Pi stack recorded 1.0990 billion tokens, $894 and a 42.003 score. With Opus 5 at xhigh, the corresponding figures were 2.3697 billion tokens, $1,741 and 44.756 for Pi, versus 1.3101 billion tokens, $1,158 and 42.224 for SoL-Pi. The Pi baseline in each backend is the denominator for the reported 49.0% and 44.7% token reductions and 33.2% and 33.5% cost reductions.

“Recorded token traffic” combines uncached input, cache reads, cache writes and output across the evaluated runs. The paper prices those token categories at API rates dated August 17, 2026. Its token-efficiency measure is API cost divided by aggregate task score. These are paper-reported API-equivalent costs for the benchmark, not a subscription price, a measured saving for another team or a promise about today's provider rates.

The score loss is material when choosing a configuration. The paper also reports a separate “Performance” point: the highest-scoring single mechanism for each backend, selected from its add-one comparison. On GPT-5.6 Sol that was ObservationPack, with a 47.208 score and $1,271 token cost; on Opus 5 it was Action Fusion, with a 50.482 score and $1,605 cost. Neither is the four-mechanism stack. On a separate 63-task CPU-only Terminal-Bench 4 evaluation, SoL-Pi solved 15 tasks versus Pi's 18, with total API model cost of $211.12 versus $286.45. That comparison is another reason to measure completed work alongside spend.

Installing and configuring the Pi extension

As checked on October 25, 2026, the repository instructions specify Node.js 22.19 or newer and npm. They recommend installing the tested Pi 1.3.0 release:

npm install --global @earendil-works/pi-coding-agent@1.3.0

Then install the SoL-Pi extension:

pi install git:github.com/NVlabs/SoL-Pi

For a trusted project, the repository also documents a project-local installation:

pi install git:github.com/NVlabs/SoL-Pi --local --approve

The compatibility notes also list Pi 0.84.2 and 1.0.4 as tested but warn that a fresh install of earlier 1.x releases can pull mismatched internal packages. These are documented commands, not commands BIG CHANGE executed.

The input to a normal run is the coding task and repository supplied to Pi, plus a single effective sol-pi.json configuration. With the official Pi distribution, a trusted project's .pi/sol-pi.json takes precedence over ~/.pi/agent/sol-pi.json; the files are not merged. All four features default to false. The repository's conservative example enables only Action Fusion and ObservationPack. An all-enabled setup requires explicitly enabling the reducer and Online Context Compact, and choosing the reducer provider/model route if the default is unsuitable. Pi keeps control of the main model, authentication and provider URL. The output is Pi's ordinary task work and tool observations, with session-specific local archives for large observations and reducer logs. The project provides a full-checkout installation and validation protocol for readers who need a reproducible all-enabled setup.

The released extension is a way to test a harness design. Its current documentation lists Pi 0.84.2, 1.0.4 and 1.3.0 as tested; other versions need verification. It also documents Pi 1.x codemode behavior, including that the reducer has no effect on script calls in only mode. Local archives persist after a session ends, and the reducer's remote path needs a data-handling decision. A useful trial would compare the same task set, model, reasoning setting and pricing assumptions under Pi and the chosen SoL-Pi configuration, recording both completed tasks and total model cost. The repository's October 13 visual and October 14 skill examples are later additions, not results from the paper's EdgeBench evaluation.

Sources & further reading

• SoL-Pi paper, arXiv:2609.20519v1 describes the four mechanisms, search and held-out evaluation, benchmark scores, token categories and fixed August 17 API prices. These are the authors' reported results.
• SoL-Pi source repository, checked October 25, documents the standalone Pi extension, current installation commands, disabled-by-default features and session archives. Its configuration reference gives the effective-file order and schema; the compatibility notes list tested Pi versions.
• Project page explains the authors' research process and figures, including the separate performance and efficiency configurations. It is a project-authored explanation, not independent replication.
• Installation protocol lists the full source-check and all-enabled validation procedure. The security policy describes local archives, reducer data transfer and Pi's existing permissions.