Researchers led by Akul Santhosh have built FailureGen, a method that generates synthetic videos of robot failures, with time labels for events, by reusing the motion from recorded failures, according to a paper on arXiv. Robot failures are hard to collect at scale.
The method fine-tunes an action-conditioned Cosmos world model on real successful and failed runs, finds the failure segments in recordings, moves their motion into the setting of successful runs to generate new videos, and filters the results by events and visual quality.
In a preliminary comparison, a failure detector trained on real and generated data reached an average precision of 0.51, against 0.41 with real data alone, and found 33 of 36 failure intervals instead of 28.



