Markdown edition

# ThinkingBox Training improves repeated success as task coverage narrows

> Repeated benchmark success rose after training, while at-least-once task coverage fell.

By BIGCHANGE.AI Editorial

Published: 2026-10-25T19:42:38.983Z
Updated: 2026-10-25T19:42:38.983Z
Canonical: https://bigchange.ai/blog/thinkingbox-training-repeated-success-task-coverage

![Microsoft logo on ivory.](https://bigchange.ai/api/media/file/microsoft-official-press-logo-hero-v1.png)
Microsoft News Center press image; layout by BIG CHANGE.

On October 22, Microsoft and Hugging Face released ThinkingBox Training for database-state rewards.

## The big change

- **What changed:** Database-state checks became agent training rewards.
- **Why it matters:** The authors report 107 of 507 tasks passed all 20 attempts, up from 38. Engineers can weigh consistency.
- **What to watch:** At-least-once coverage fell from 453 to 446. Check breadth; live deployment performance remains unmeasured.

After 50 GRPO steps with Qwen3.8-27B, the authors counted 6,163 successes in 10,140 attempts, up from 5,242.

Base-model probes fixed a private 187-task pool; the public 39-task sample differs. Most rewards used executable checks, some a judge.

Reference run targets Linux, Python 3.12, patched VeRL 0.9.0 and 24 H100 GPUs, with simulated-user and judge endpoints. Training cost is unreported.

## Sources

- [ThinkingBox Training blog](https://huggingface.co/blog/microsoft/thinkingbox-training) — Methods, results, limits.
- [Training code](https://github.com/microsoft/thinkingbox-training) — Implementation, requirements.
- [Data preparation](https://github.com/microsoft/thinkingbox-training/blob/main/docs/data-preparation.md) — Sample versus training pool.