Koopman learning method controls a quadruped standing on two rounded feet

Researchers have proposed K-MBRL, a Koopman model-based reinforcement learning algorithm for controlling nonlinear robotic systems with unknown dynamics, according to a paper on arXiv.

The method approximates the unknown dynamics in a linear representation using a stochastic Koopman operator, which reduces the model-dependent policy gradient computation to a constant matrix product, while a critic estimates the remaining future cost.

In simulations on HalfCheetah and quadruped locomotion tasks, the method showed strong sample efficiency and better performance than classical model-free and model-based baselines, the authors reported.

On a physical quadruped robot, the learned policy carried out a sequence of manoeuvres: rising to an upright posture supported by two rounded rear feet, keeping balance during bipedal walking, and returning safely to the ground.