A Caltech and Stanford team has shown HomeBody, a research system in which GPT Astra directs a Unitree G1 through kitchen tasks by calling navigation and manipulation skills. The robot first explores the room and stores where it saw objects. In the team’s demonstrations, it gathers coffee bags, discards specified cartons and retrieves a medicine bottle from a drawer after the object has left its camera view.

The big change

  • What changed: HomeBody gives a vision-language model access to a spatial record of an explored room and a common interface for robot skills. The model selects a skill and target; local software plans and executes the movement, then reports the outcome.
  • Why it matters: The model can plan across locations and use feedback without a newly trained action policy for this kitchen. The team’s demonstrations show how memory and existing motor skills can be assembled for longer tasks.
  • What remains open: The public release shows selected demonstrations, not a controlled success-rate study. A paper and the robot code are not publicly available as of September 27, so the reported performance cannot yet be independently reproduced from the release.

How the robot remembers the room

HomeBody starts with exploration. According to the project’s deployment description, the G1 collects wide-angle iPhone video, D435i camera observations, LiDAR scans, joint poses and model-chosen waypoints. A LiDAR-based SLAM map supplies measured room geometry. Astra uses that material to build a digital twin in Nvidia Isaac Sim. The team aligns the map and reconstruction so recorded camera views and the robot’s current position share a coordinate frame.

That preparation matters when the instruction refers to an object the robot cannot currently see. For the medicine request, the team says HomeBody recalls stored camera keyframes to find the drawer, travels there, opens it and picks up the bottle. The room model supports navigation and location recall; the live camera still guides the final physical interaction.

The interactive kitchen rollout on the project page is explicitly labelled illustrative and takes place in the reconstructed room. The page separately presents videos of the G1 performing the kitchen tasks and labels its navigation, pick, place and drawer skill clips as real robot footage. Those videos document the team’s selected runs, while the simulated replay explains the planning sequence.

From model decision to movement

The model receives the task, its current camera view, map context, gripper state, recalled observations and the preceding skill result. It sends a structured call naming a skill and target. For a pick, the target is an image point on a 0–1000 scale plus a choice of hand. Local perception segments the object, estimates its depth from stereo images and turns the selection into a 3D grasp. An arm planner then computes and checks a motion path. Navigation instead takes a 2D goal and facing point in map coordinates; placement takes a 3D release point. Drawer opening packages handle alignment, hooking and backward walking into one action.

Skills make small corrections locally. The team describes visual tracking during an approach and bounded retries when a grasp misses. If recovery fails, the skill returns a reason to Astra for another decision. Walking and reaching rely on a pretrained whole-body controller, AMO. The page says arm and hand commands run at 250 Hz and the AMO policy updates at 50 Hz. Astra runs remotely; perception, motion planning and the skills run on a laptop with an RTX 4090 GPU.

Astra chooses what action to try and where; the skill library and controllers determine how the G1 moves. The team says the kitchen demonstrations used no environment-specific training data or additional policy learning. Existing pretrained control and substantial room reconstruction are still part of the setup.

What the demonstrations establish

The project page describes two physical task sequences in a previously unseen kitchen. One consolidates coffee bags and throws away specified milk and orange-juice cartons. In the other, the G1 locates a medicine bottle from an underspecified request, retrieves it from a drawer, hands it over and discards a carton. Its skill clips show individual actions and recorded retries; the page says most previews are sped up and identifies a drawer-pick sequence with continuous reattempts.

The release does not give trial counts, a task success rate, a comparison against a learned vision-language-action policy or the time and cost of each completed kitchen run. It therefore supports an architectural demonstration, not a measured claim that this design works reliably in other kitchens. The linked public repository currently says “Code coming soon,” and the project page’s Paper label has no linked manuscript. The repository holds the website, illustrations and videos, but no robot implementation or evaluation files. BIG CHANGE did not run HomeBody or test a robot.

BIG CHANGE’s earlier GPT-Policy report examined a different team’s paper on context for robot actions. The HomeBody materials do not identify that paper as part of this system.

The team lists practical constraints: building the digital twin takes setup time and API spending; Astra’s remote reasoning creates pauses between skills; reach and manipulation limit what the G1 can do; and finger servos can overheat during extended operation. The local stack requires the RTX 4090 laptop, and heavier skills may need more compute.

Sources & further reading

  • Caltech and Stanford researchers’ HomeBody project page — Primary account of the system, kitchen demonstrations, simulated replay, skill interfaces, control stack and stated limitations. The page is dated September 2026; its selected videos and descriptions are the team’s evidence, not an independent evaluation.
  • Stanford TML’s public HomeBody repository — Linked by the project page. As checked September 27, 2026, the README says code is coming soon; the public tree contains website assets rather than the robot implementation or a full paper.