Robot policies that remember by time mistake a pause for progress

Memory-augmented vision-language-action policies are supposed to remember what happened, but many just stamp each frame with its time step, according to a paper on arXiv. Chubin Zhang and Bo An show such memory cannot tell waiting from working.

On the counting tasks of the RoboMME benchmark, making the robot pause or fidget briefly caused a released memory policy to count the interval as work and stop early, a failure the authors name phantom progress. Success fell from 47 to 16 out of 50 on one task and from 44 to 10 on another.

Withholding the paused frames from memory restored success, and the authors trace the cause to a temporal shortcut: in demonstrations, elapsed time predicts progress.