Vision-based estimator helps a simulated race car adapt to unknown surface friction

Researchers led by Alexander Borca-Tasciuc have described a real-time estimator that lets robots infer physical properties such as friction from vision before contact, in a paper posted on arXiv.

Instead of fixed semantic classes, it keeps a Gaussian-mixture estimate of physical parameters tied to continuous visual embeddings, drawing on past interactions with similar-looking regions. When there is little history, a generative prior model fills in.

In a detailed simulation of an autonomous race car on a track with unknown slippery and grippy patches, the controller adapted over repeated laps as evidence built up and improved its performance.