# UserSWE tests coding agents in conversations

> UserSWE shows how turn budgets alter coding-agent rankings.

By BIGCHANGE.AI Editorial

Published: 2026-10-25T14:14:38.541Z
Updated: 2026-10-25T14:14:38.541Z
Canonical: https://bigchange.ai/blog/userswe-coding-agent-dynamic-user-evaluation

![Angled monitor showing alternating empty message shapes.](https://bigchange.ai/api/media/file/userswe-conversation-monitor-hero-v1.png)
AI-generated conceptual illustration by BIG CHANGE.

[UserSWE](https://arxiv.org/html/2610.29822v1) is an October 21 preprint testing 11 coding agents on 72 repository tasks with simulated users.

## The big change

- **What changed:** The user’s clarity and proposed cause can change across turns.
- **How it works:** Each turn triggers tests and cross-family verdict review.
- **Why it matters:** The leader changes with the exchange budget.

## What the score captures

A fixed pool constrains disclosure. Messages react to the agent and tests; a persona chain calibrated on 20,859 private sessions shapes their wording. The users were simulated.

Claude Fable 5.1 leads at one turn (31.9%); GPT-6-Astra at eight (81.9%), one task above Claude Opus 5. Opus leads on the score that rewards faster completion. Different scaffolds complicate model-only comparisons. The authors promise a benchmark and framework release; v1 links none. Tasks cover one or two issues.

## Sources

- [UserSWE preprint v1 (2026)](https://arxiv.org/html/2610.29822v1) — Author-run preprint.