The November 4 VABAR preprint introduces a simulation benchmark for robot manipulation under changing conditions.
The big change
- Six tasks include variations in pose, layout and clutter.
- Generated and human-authored programs share robot skills.
- Simulated results do not establish real-world reliability.
Why variation matters
VABAR specifies permitted variations and success conditions. In gear meshing, a simulated arm assembles gears on axles; the harder tier adds a third gear.
VABAR provides skills for perception, grasping, planning and verification. The authors’ earlier Graph-as-Policy paper describes a system that composes such skills into executable graphs and rehearses them in simulation. In that design, simulation rehearsal can expose failures and guide revisions to a graph before deployment. It does not independently validate VABAR.
What the baseline shows
The study evaluated 30 instances per task and difficulty condition. On easier gear meshing, an agent-generated graph succeeded in 70% of simulated trials versus 100% for a human-authored graph with the same perception, skill, planning and execution stack. The comparison concerns graph construction within that setup.
VABAR records observations, actions and simulated contacts to help diagnose failures. These simulations do not establish physical-robot performance or independent replication.
