Researchers at the Moscow Institute of Physics and Technology (MIPT) have developed a modular control system for humanoid robots working in warehouses, MIPT told CNews. Called SG-RAPL, it lets a robot keep an updated 3D scene graph of its surroundings and plan a sequence of actions from commands in natural language. The results are published in the journal Engineering Applications of Artificial Intelligence.
The system builds the graph in real time from a single onboard camera, combining object masks in camera images with depth maps and ArUco visual markers, and records shelves, containers, boxes and people and how they relate. It can break a task such as taking a box from the top shelf into steps: going to the rack, grasping the load and carrying it.
The team built and hand-labelled a dataset of warehouse scenes with dozens of voice-command scenarios. In simulation and on a real robot from the university's robotics lab, the approach did better than systems without 3D scene graphs, said co-author Alexander Panov, director of MIPT's Center for Cognitive Modeling. The code and dataset are on GitHub, and Russia's Ministry of Economic Development supported the work.
Background
Moscow Institute of Physics and Technology
University
A Russian technical university whose Center for Cognitive Modeling works on robot planning.
Description from CNews
SG-RAPL
Robot planning system
A modular control system that lets warehouse humanoid robots plan actions from spoken commands using 3D scene graphs.
Description from CNews
Alexander Panov
Director of MIPT's Center for Cognitive Modeling
Director of the Center for Cognitive Modeling at the Moscow Institute of Physics and Technology.
Description from CNews
