Researchers led by Yimin Han have presented MoViC, a framework that uses live video from an ordinary colour camera to control a humanoid robot's whole body, without wearable tracking devices, according to a paper on arXiv. It combines reconstruction of human motion from a single camera, retargeting to the robot and a transformer-based behaviour model.
The policy gets the robot's recent body states, one explicit next-step target and a compact summary of the motion's timing, which keeps the network small. A method that extrapolates between frames links the 10 Hz video estimates to the 50 Hz controller. In hardware experiments, the robot tracked fast, dynamic motions, and operators controlled two-armed and one-armed manipulation tasks from live video without any devices.



