Researchers led by Chen Liu have turned recordings from egocentric glasses worn by people into training data for a humanoid robot, according to a paper on arXiv. Teleoperated robot demonstrations are costly because a trained operator has to steer a real robot for every episode, while copying human motion directly fails because robots differ in height, reach and hands, the authors write.
An agent rebuilds the recorded scene in simulation from the glasses' colour, depth and head-tracking data, fills in unseen surfaces and predicts where the robot should touch objects. Planners then work out feasible grasps, standing positions and whole-body movement, and a reinforcement-learning policy trained in physics simulation checks the motions, producing images from the robot's camera, its body state and verified action labels.
Fine-tuned only on this converted data, without any real-robot demonstrations, the GR00T N1.7 model carried out tasks that combine walking and handling objects on a Unitree G1 on its own.
Background
Unitree G1
Humanoid robot
The Unitree G1 is a small humanoid robot from China's Unitree Robotics, and research teams, including one at Apple, use it to test new ways of training and controlling humanoids.



