Researchers at MIT's Computer Science and Artificial Intelligence Laboratory and Carnegie Mellon University's Machine Learning Department have published Workspace Models, a way to give robots a compact memory of their recent observations, on the arXiv preprint server.
Instead of having a large vision-language model search through a long history of camera frames at runtime, the reasoning happens offline: the large models pick out the salient details, and a small module learns to compress the observation history into latent vectors. A small policy then predicts the robot's actions from those vectors, reacting to changes in milliseconds.
In virtual and real-world tests on a robotic arm, the system picked up and put away cubes 92% of the time, roughly 25 percentage points better than the next-best approach, which used an agent looking up keyframes. It was also the fastest of the tested systems that can reason over long-term memory.
"If we want robots to surpass human performance one day, we might want to look toward a lighter, more compact system to do tasks," said lead author Nitish Dashora, a CSAIL researcher. The team sees the method suiting dynamic settings such as kitchens, where a robot must recall the right details quickly.
Background
Nitish Dashora
CSAIL researcher
Workspace Models
Robot memory method
A method from MIT CSAIL and Carnegie Mellon that compresses a robot's observation history into latent vectors, letting it react in milliseconds.
