A November 4 preprint compares robot touch cues.

The big change

  • One fingertip yields multiple touch cues.
  • Matched demonstrations support comparison.
  • Findings concern tabletop trials.

What the fingertip can tell a policy

An internal camera observes marker-embedded elastomer. Its images yield contact geometry and marker flow, which tracks sideways deformation. Diffusion Policy combines these cues with cameras and robot state to predict actions.

Earlier GelSight research describes the optical principle: an object presses on a soft surface and an internal camera records the deformation. Markers on that surface shift as it is pulled sideways. The resulting pattern gives information about contact, even when an outside camera cannot see the interface.

In controlled shear experiments, the GelSight authors examined how marker displacement changed under load and as partial slip appeared. They treated the patterns as a way to assess contact conditions, while noting that more testing under different shapes and pressures was needed.

The original Diffusion Policy research describes a way to learn robot behavior from demonstrations and generate actions through an observation-conditioned denoising process. Rather than executing one fixed sequence, the system predicts an action sequence, carries out part of it, then observes again and replans. The original authors tested it on several manipulation benchmarks.

The experiment and its limits

Each task used the same demonstrations. A tube was repositioned before insertion; wiping required sustained contact. Peg insertion was the third task. Eight configurations attempted each task 30 times, totaling 720 trials. Vision alone succeeded 52/90 times (57.8%); vision plus full tactile input succeeded 71/90 (78.9%), and vision with marker flow 67/90 (74.4%).

The authors’ qualitative analysis found normal/depth inputs often established contact but failed during tube rotation or sustained wiping. Marker flow had less reliable initial grasps, yet better regulated established contact. The finding suggests complementary cues may help at different stages in this setup. One sensor and gripper, differing frontends and no direct force measurements limit generalization.