Researchers led by Pol Mestres have extended a reinforcement-learning method so that robots can learn skills while keeping to safety constraints with a guaranteed probability, according to a paper on arXiv. Standard reinforcement learning does not guarantee that learned policies are safe, and earlier safe methods with proofs covered only limited kinds of constraints and needed unbiased estimates, which kept them from working with common algorithms such as PPO, the authors write.

Their version of the RSGF-RL framework handles constraints over whole trajectories, such as avoiding collisions with a set probability, and works with biased estimates. The authors show that constraint violations during training stay within a bound that shrinks as more trial runs are collected.

Implemented with PPO-style estimates, the method produced whole-body control policies that ran on a Unitree G1 humanoid straight from simulation, and collision-free waypoint navigation on a Crazyflie drone.

Source: arXiv Robotics (cs.RO)

Background

Pol Mestres

Robotics researcher

Robotics researcher.

Unitree G1

Humanoid robot

The Unitree G1 is a small humanoid robot from China's Unitree Robotics, and research teams, including one at Apple, use it to test new ways of training and controlling humanoids.