Researchers led by Iana Zhura, with Dzmitry Tsetserukou among the authors, have built BEV-Diffusion, a navigation model that uses only cameras to learn a bird's-eye map of where a robot can travel, how sure it is of that map and the path to take, all in one network, according to a paper on arXiv. Current methods often rely on expensive lidar, long mapping and pipelines of separate modules.

A frozen DINOv2 backbone encodes successive camera frames, which are projected onto a map seen from above, and a ranking loss grades outdoor surfaces rather than marking them as passable or not, so asphalt scores higher than grass. A transformer then generates waypoints in a few steps, guided away from parts of the map it is unsure about, and a memory keeps obstacles after they leave the camera's view.

In a courtyard it had not seen before, a wheeled-legged robot travelled 120 m and completed 96% of the route without help, against 75 to 88% for other methods, and it also finished the route after sunset. Retraining only about 3% of the model adapted it to a small legged robot in 2.3 minutes, and indoors the memory helped that robot get out of maze-like traps.

Source: arXiv Robotics (cs.RO)

Background

Iana Zhura

Robotics researcher

Robotics researcher.