Training on crash videos helps a driving world model foresee collisions

Researchers led by Zihao Sheng have identified what they call survivorship bias in world models for autonomous driving, in a paper posted on arXiv. Because recorded drives in common datasets almost always end safely, such models often imagine harmless outcomes even when a collision is about to happen, they write.

Their neutral world model is trained on both everyday and safety-critical futures, mixing ordinary driving footage with real accident videos and an objective guided by bounding boxes. It anticipates imminent collisions more faithfully while keeping false collision predictions low and generating routine scenes at a competitive quality, the authors report.

Used to compare several candidate paths for the car, the model predicts the future of each and a reward function scores them before an action is chosen. On hard NAVSIM scenarios this picked the better path more reliably than the baseline world model, and zero-shot tests with real data of a pedestrian dummy crossing in front of a research vehicle showed it transfers to unseen sensor data.