Researchers led by Xinpei Ni have built a monitor that checks from camera images whether a robot manipulation mission meets timed and conditional requirements written in signal temporal logic, according to a paper on arXiv. Such checks depend on physical margins like distances and clearances that the images do not show directly, the authors write.

Instead of a separate visual monitor for each rule, they learn one shared representation from several camera views and decode physically meaningful margins for each basic condition, which can then be combined into any rule built from them without retraining. Training the encoder together with the decoders raised the certification rate from 58.7% to 96.3%, and calibrated error bounds certify both whether a rule is met and whether it is broken.

In simulated LIBERO kitchen missions, the monitor judged satisfaction or violation correctly in 98.9% of 4,662 evaluations across 18 unseen formulas. For missions of up to 17 minutes, the calibrated bounds settled 96.9% of missions without a wrong conclusion.

Source: arXiv Robotics (cs.RO)

Background

Xinpei Ni

Robotics researcher

Robotics researcher.