ObjectWorld world model predicts how whole objects move, trained on a 1,210-hour dataset

Researchers led by Haonan Chen have presented ObjectWorld, an approach to world models for robot manipulation built on what they call 3D full-object flow, in a paper posted on arXiv.

Full-object flow gives dense 3D trajectories to points across an object's whole surface, including parts hidden from view, and works for rigid and deformable objects alike. To train it at scale, the team built a 1,210-hour dataset from real-world videos and physical simulations, with a pipeline that recovers full-object motion from real footage and metrics that check annotation quality.

The resulting world model takes an image and a task description and predicts the flow over the object's reconstructed shape. Experiments showed reliable annotations, flow prediction that improved with more data and successful manipulation in new scenes and on different robots, the authors report.