Hongkun Yu, Francisco M. López and Francisco Cruz have described HOC2R in an arXiv paper, a tool that extracts where a hand touches an object in human videos and turns that into trajectories a robot can follow. It combines frozen DINOv3 and SAM 2.1 features with trained parts that link hands to objects and locate contacts.
On a 30-task benchmark, its automatic mode raised contact-count accuracy from 58.1% to 71.4% and cut location error from 19.81 to 10.46 pixels compared with an adapted VRB2 baseline. Across six robot evaluation sequences, it produced four feasible trajectories and succeeded on three tasks, against two feasible trajectories and one success for each baseline.
Background
HOC2R
Robot motion transfer method
A tool that extracts hand-object contact trajectories from human videos for robots to follow.
Description from arXiv Robotics (cs.RO)
