Researchers led by Xinheng Zhang have presented DiffWAM, a world action model that learns not only to predict future camera frames and robot actions but also the differences between consecutive frames, according to a paper on arXiv. Predicted future frames mix the changes that matter for a task with much that does not, such as a static background, the authors write.
DiffWAM uses three experts: one predicts future frames, one generates actions and one predicts the differences between frames. Because the other two only see the current frame, they can be dropped when the robot runs, adding no extra computing time.
It reached average success rates of 98.8% on LIBERO and 92.65% on RoboTwin 2.0, and 88.3% on real-world manipulation tasks, beating the FastWAM model by 1.75, 1.55 and 15.0 percentage points.



