Researchers led by Arusa Kanwal have built VoxelSBFL, a way to tell which runs of a robot policy are likely to fail from the path of its gripper alone, according to a paper on arXiv. It borrows spectrum-based fault localisation from software testing, and divides the workspace into small cubes, called voxels, that take the place of lines of code.
Recorded paths of the gripper show which voxels each run passed through. Together with whether each run succeeded, this gives a map of the regions linked to failure, which can then score new paths without access to the inside of the model.
The team compared 34 such techniques on four vision-language-action models, using 96,000 simulated pick-up runs. VoxelSBFL reached a macro ROC-AUC of 0.704, clearly better than the eight action- and path-based measures it was compared with. How well it worked depended on the size of the voxels and the amount of training data.



