Researchers led by Ran Qi have presented RoboCoEvo, a framework in which an agent built on a multimodal language model and a robot foundation model, such as a vision-language-action model, improve together, according to a paper on arXiv. Improvements to one part of such combined systems usually do not help the other, the authors write.

The agent writes and refines programs that detect failures and recover from them, which produces targeted corrective experience. Further training then folds those corrections into the robot model's own policy, and the way the updated policy fails guides the next revision of the agent's programs. In the BEHAVIOR-1K, RoboCasa and RoboTwin simulation benchmarks, the robot model's autonomous task performance kept improving over successive rounds.

Source: arXiv Robotics (cs.RO)

Background

Ran Qi

Robotics researcher

Robotics researcher.