A robot policy trained to copy successful demonstrations can struggle after a grasp fails or an arm lands somewhere the recorded path did not cover. In an October 9 paper, researchers from the Chinese Academy of Sciences and Peking University report a way to train vision-language-action (VLA) policies on task progress using existing robot trajectories. Their method, called Offline Goal-conditioned Policy Optimization (OGPO), improved success rates in simulated manipulation tests that included mid-execution disturbances. The paper reports no physical-robot deployment or independent replication.

The big change

  • What changed: OGPO labels stages of recorded robot trajectories as reachable task-progress states, then uses those labels to reward actions that move a policy toward the next stage during offline training.
  • Why it matters: A policy that has learned the value of reaching an intermediate state may have more options when a grasp or motion goes off the demonstrated path. The paper tests that proposition in simulation; it does not establish reliability on a physical robot.
  • What to watch: Independent reproduction, results on physical manipulators, and tasks whose progress cannot be divided by gripper events would clarify the method's reach.

Learning progress from recorded attempts

The authors start with successful demonstrations and failed trajectories. OGPO splits each trajectory where the gripper changes state, such as opening or closing, and assigns a local goal to each segment. Completed segments receive a positive label. A vision-language model annotates whether segments in failed trajectories achieved their local goals, so a run that eventually fails can still supply evidence about intermediate progress. The authors' method section describes this labeling process and the training objectives.

The labels create a training reward for movement toward completed states, a bonus for reaching one, and a penalty when a segment misses its goal. A value model estimates which observed states are followed by progress. OGPO then places more weight on recorded actions with a higher estimated advantage while retaining supervised fine-tuning on successful demonstrations as an action prior. The local goal is used to construct rewards during training; it is not an extra input to the deployed policy or its value function. The value model is used during training, according to the paper's implementation details.

This design addresses a narrow problem in imitation training. Copying an expert action at each recorded step tells a model what happened along that path. OGPO tries to make intermediate outcomes informative when the policy encounters a feasible deviation. Its ability to recover still depends on what the fixed dataset contains. The authors explicitly warn that severe departures beyond that data may remain out of reach.

What the benchmarks show

The authors tested OGPO on the LIBERO and MetaWorld simulated manipulation suites, using the π0 and π0.5 VLA backbones. In the paper's LIBERO comparison, the π0.5 few-shot supervised baseline averaged 77.1% task success; the same starting policy with OGPO averaged 97.1%. On LIBERO-Long, the reported result rose from 43.9% to 93.2%. These are task-success rates for the named benchmark and starting policy, not a general robot success rate.

Recovery is more directly tested in two perturbation settings. LIBERO-Plus changes camera view, robot initial state, visual noise or object layout. The authors' LIBERO-ReAct extension disturbs execution with action noise, a flipped gripper command or an end-effector position change during a selected 20-step interval. Against the π0.5 full-shot supervised baseline in their robustness table, OGPO's combined score was 81.8% versus 76.7%. Under a flipped gripper command, success was 87.7% versus 76.8%. Camera-view performance went the other way, from 62.8% for the baseline to 61.5% for OGPO. The aggregate improvement therefore does not cover every disturbance.

The researchers also removed parts of the training recipe. Their ablations report lower performance when they drop the intermediate-goal reward or the offline reinforcement term; removing the supervised term also hurts. That helps locate the gain within the authors' setup, though it is not an independent test of the method.

The remaining boundary

Gripper changes provide OGPO's trajectory boundaries. The authors acknowledge that this may fit poorly when task progress has no natural gripper event. They also identify dependence on the quality and range of offline data. The reported tests are simulated manipulation benchmarks, and the paper supplies no measured physical-robot completion rate or operational safety result.

For teams evaluating offline VLA training, the paper supplies a concrete comparison: train on the same recorded trajectories, then test task completion after controlled deviations. Its strongest evidence comes from simulated manipulation, especially the mid-execution gripper disturbance. BIG CHANGE reviewed the paper and its reported tables; we did not train a model or test a robot.

Sources & further reading