Researchers led by Wei Dai have built EmbodyAgent, a post-training framework that turns an open-weight vision-language model into an agent that controls a robot by writing and running code, according to a paper on arXiv. A strong closed teacher model generates examples, which are filtered by task success.

The examples are distilled into a 9-billion-parameter model, which is then trained further with online reinforcement learning. On ten LIBERO Spatial tasks, held-out success on 100 test states rose from 0 for the base model to 14 after supervised fine-tuning and 84 after reinforcement learning. The median number of model calls per episode fell from 36 to 11.

Source: arXiv Robotics (cs.RO)

Background

Wei Dai

Robotics researcher

Robotics researcher.