ActPrune speeds up robot vision-language-action models by up to 1.61 times

Researchers led by Sungwon Moon have described ActPrune in an arXiv paper, a method that needs no training and makes vision-language-action robot models respond faster by dropping parts of camera images. Earlier pruning methods decide which image tokens are redundant only after the vision encoder has processed the whole picture; ActPrune decides before, using the motion the robot planned at the previous step.

In the fixed third-person camera view it keeps patches along the expected path of the gripper plus key reference points, and in the wrist camera it drops patches that have not changed once the camera's own motion is accounted for. In two simulation environments and four real-robot tasks, it ran up to 1.61 times faster with 59.5% fewer computations and little loss in success rate, the authors report.