An October 20 preprint reports SpecEvo results on 176 program-search tasks.

The big change

  • What changed: Four parallel cheap-model Speculators propose programs; a cheap-model Advisor distills history. A frontier Navigator chooses checkpoint interventions based on stagnation.
  • Why it matters: Authors report 50–80% lower API charges than frontier-heavy baselines in three-hour tests. Separate tests matched spend or evaluations.
  • What to watch: API savings exclude local compute. Model-written archive descriptions can misstate code.

What the tests measure

An evaluator scores programs. The archive retains alternatives; history records failures. Across six tasks, 2.4% of 500 calls used the frontier model.

Tests used two 16-core Xeon CPUs and lightweight evaluators. They did not establish total compute costs or results with expensive evaluators. BIG CHANGE did not run SpecEvo.