Researchers led by Xianzhe Fan, with Linxi Fan and Jan Kautz among the authors, have built MAWAM, a model that predicts the actions of several robots together with what each will see next, according to a paper on arXiv. Every move by one robot changes the scene for the others, so the model generates them all in one sample while each robot keeps its own control interface.

On three real-robot tasks, collecting laundry bags, folding bath towels and handing over an umbrella, MAWAM averaged 67.8% success. A multi-agent vision-language-action model trained on the same demonstrations and cameras did 18.9 points worse. In simulation MAWAM averaged 90.3% on eight tasks with up to four arms, and without the attention that links the robots a four-arm relay dropped from 95.0% to 39.0%.

Source: arXiv Robotics (cs.RO)

Background

Xianzhe Fan

Robotics researcher

Robotics researcher.