Researchers led by Javad Amirian have released HubEgo-100K, a dataset of video recorded from the wearer's point of view to train robots, according to a paper on arXiv. Robot policies trained by imitation do poorly outside the places and objects they were trained on, and robot data is costly to collect in many places.

The first release has 10,124 hours of 1080p video with 100 Hz motion sensor data, each recorded with one head-mounted phone camera. In it, 12,227 contributors filmed their own unscripted work in 15,548 settings of 11 types, such as kitchens and workshops, with 83% of the hours in workplaces.

It contains 72,754 episodes with machine-generated action and language labels, in LeRobot and MCAP formats under a CC BY 4.0 licence. Later versions will add benchmarks and grow towards 100,000 hours, the authors write.

Source: arXiv Robotics (cs.RO)

Background

Javad Amirian

Robotics researcher

Robotics researcher.