MIND looks inside a robot's control policy to detect and explain its failures

Researchers led by Riad Ahmed have presented MIND (Mechanistic Interpretability for Neural Diagnosis), a method that inspects a trained robot manipulation policy while it runs to catch failures and point to the internal concepts behind them, in a paper posted on arXiv.

MIND uses a sparse autoencoder to split the policy's internal activations into separate features. The researchers tie each feature to the robot's position, motion and surroundings, and change feature activations to see how the robot's actions respond. From successful and failed runs, they learn how the features behave as a failure approaches.

During a task, MIND flags departures from those patterns and describes the problem with the features that changed most. Tests in simulation and on a physical robot showed that shifts in these features helped detect and explain failures, the authors report.