A September arXiv preprint, “In-Context Robot Learning with VLM Agents”, introduces GPT-Policy, which connects a fixed vision-language model to robot tools using demonstrations, images and interaction history. Its experiments include robot-arm tasks and mobile exploration. They do not validate the separate Reddit post’s claim about GPT-6 Astra and a Unitree G1.

The big change

  • What changed: GPT-Policy gives a general vision-language model a structured way to turn demonstrations and current camera views into robot-tool requests, then use execution feedback to choose another action without updating the model’s task-specific weights.
  • Why it matters: The approach separates broad visual reasoning from motion checks and execution. In the authors’ limited trials, added context helped on some tasks, while contact-sensitive actions sometimes needed aligned robot-action references.
  • What to watch: The paper reports only three trials per condition and describes slow, failure-prone runs, including arm collisions. Reproduction, larger evaluations and independent safety controls matter before these results say much about robots working around people.

What GPT-Policy does

The paper was submitted to arXiv on September 16. It asks whether a general vision-language model can use examples and feedback to perform a task from a new initial state without fine-tuning or changing task-specific model parameters. The authors call their framework GPT-Policy and name GPT-6 Astra among the models they evaluated.

The system assembles several kinds of context: a task instruction, current camera views and state, plus optional human demonstration videos, robot demonstrations with action references, a target image, earlier robot attempts or human interaction. A context compiler selects task-relevant visual transitions and combines them with instructions, constraints and tool schemas. The VLM then proposes a structured robot-tool request.

That request passes to an embodiment-specific controller. The authors describe checking inverse-kinematics solutions, sampling Cartesian poses and timing joint references before executing or rejecting a movement. The controller returns observations and execution feedback for the next model decision. The model proposes actions; robot-specific code validates and carries them out.

This loop is the paper’s main contribution. It lets a fixed VLM adapt its next action to examples and recent outcomes without a gradient update. “In-context learning” here describes how the system uses information supplied during a task. It does not mean a model permanently learned a new skill or that every robot can run the same tool interface.

What the trials measured

The authors report five experiment families across ten robot tasks, with three trials per condition. Their project page lists real-robot runs and publishes task outcomes, decisions and elapsed time. In towel pickup, GPT-6 Astra succeeded in two of three trials with a human video and none of three without it. Notebook pickup also moved from zero of three without a demonstration to two of three with one.

Contact tasks show why the added context is not a general solution. For unscrewing a bottle cap, robot video produced two successes in three trials; adding aligned robot actions produced three of three. For removing and reinserting a plug, the video-only condition had no successes in three trials, while video plus action references reached two. These are small condition-level counts, not reliability estimates.

The project also reports three of three successes for target-image block and fruit arrangements and two human-interaction tasks. Under self-interaction history, the paper reports three of three successes on both “Lemon to Pink Plate” and “Movable Exploration.” In the latter, the robot actively avoided obstacles while searching for the target; its mean elapsed time was 25.53 minutes. Figure 1 also depicts mobile object retrieval. These results broaden the paper beyond tabletop manipulation, while remaining selected tasks with three trials per condition. Runs took minutes: the project page reports an average of 18.9 minutes for towel pickup with a human video, and 17.9 minutes for the bottle-cap task with video and action references. The work therefore leaves latency and operating cost as practical questions, not solved details.

Appendix B lists Morphi Kino alongside YAM and ARX X5 among the paper’s described robot configurations. It identifies Morphi Kino as having a mobile base, waist and head, as well as two seven-joint arms. The paper therefore includes a mobile platform configuration in addition to arm platforms; the appendix does not identify a Unitree G1.

The paper’s mobile task does not establish the Reddit scenario

The Reddit post claims GPT-6 Astra controls a Unitree G1 in an unfamiliar room, remembers where objects are and retrieves them later from vague requests. The paper reports the distinct “Movable Exploration” task and describes Morphi Kino as a mobile-base platform, but the paper, project materials and public GitHub repository do not identify the Reddit post’s G1 scenario as a GPT-Policy experiment. The post remains a separate, unverified claim; it is not disproved by this comparison.

The authors’ project page contains videos of their experiments, which are evidence of the tasks they report, not independent validation. The public code repository currently lists adapters for ARX X5 and I2RT/YAM and says deployment hosts, private prompts, run recordings, site-specific calibration and evaluation history are not included in the public tree. That adapter list describes the released repository; it does not define the full scope of the paper, which also reports mobile exploration and lists Morphi Kino in Appendix B. The available materials do not provide enough detail for BIG CHANGE to reproduce the reported runs, and we did not test a robot.

The control boundary still matters

The project page reports lengthy action sequences and notes execution difficulties. Its discussion says observed inter-arm collisions showed that the existing safeguards were insufficient for safe autonomous deployment. It calls for independent monitoring of physical separation and contact, alongside faster low-level control and safer recovery. A controller that rejects some invalid movements is not, by that fact alone, a complete safety system.

The study offers a concrete research result: context can help a general VLM guide robot tools through certain tasks, and the type of context matters. The size of the evaluation, time per run, incomplete public reproduction materials and reported collisions keep the result far from evidence of a household humanoid that reliably understands and carries out open-ended requests.

Sources & further reading