Augmentron varies the scenery in robot training videos consistently across cameras

Tanay Tandon and co-authors have described Augmentron in an arXiv paper, a pipeline that adds visual variety to robot demonstrations so trained policies are less thrown by features unrelated to the task. Instead of editing every frame and camera view separately, which is costly and can give inconsistent images, it builds reusable 3D scene representations and renders them through the recorded camera poses, keeping the robot, objects, actions and labels unchanged.

On three physical manipulation tasks, replacing half of the real training data with Augmentron data improved success when the robot faced unfamiliar distractor objects and changed workspaces. Compared with other augmentation methods, it achieved the highest pooled success in all three test conditions, the authors report.