Researchers led by Yiming Dou have built DreamFAR, a world model that can serve as a simulator for robot manipulation, according to a paper on arXiv. Its dynamics model is trained from scratch on a large dataset of teleoperated robot work, with no pretraining on internet video.

Existing world models either fit a single environment or cannot simulate long tasks with a lot of contact, the authors write. DreamFAR is much smaller than high-performance video generation models but shows a better grasp of contact and cause and effect, and it runs at 10 frames per second on a commercial graphics card.

In experiments it predicted the long-term future better than existing models. The team also used it to evaluate robot policies and to train them on a mix of real and generated data.

Source: arXiv Robotics (cs.RO)

Background

Yiming Dou

Robotics researcher

Robotics researcher.