XYZ has built a training pipeline for its semi-humanoid robot DEUX that feeds the failures the robot meets at work back into its training, the AI robotics company announced. The loop links collecting demonstrations, training the model, deploying it on the robot and checking it on site, and adds demonstrations where the model is weak instead of adding data across the board.

XYZ applied it to a conveyor task in which two DEUX robots pick bread bags from a moving tray, scan their barcodes and move them to the next tray. After training on 1,200 demonstrations, the robots often could not pick a bag up again after missing it and could not grasp bags turned by about 90 degrees. With about 200 extra demonstrations for each of these failures, recovery after a missed grasp rose from 0% to about 85% and grasps of turned bags from about 10% to about 90%, the company said. Bags lying diagonally, which the training data covered poorly, showed no clear gain in offline tests.

Operators teach DEUX by steering it with a Meta Quest 3 headset or a leader arm, and the joint commands sent to the robot are recorded as they are, so a trained model can later take the operator's place without changes to the robot's lower-level controls. XYZ fine-tuned Nvidia's Isaac GR00T N1.7 vision-language-action model and stores the tasks DEUX learns as skills on its BrainX robot intelligence platform.

Chief executive Hwang Sung-jae said XYZ would widen the work DEUX can do and bring it to retail and office sites. “The competitiveness of physical AI depends on how quickly the failures a robot meets in the field are turned into its next round of learning,” he said.

Source: iRobotNews

Background

DEUX

Semi-humanoid robot

XYZ's semi-humanoid robot, developed for everyday interaction with gestures, gaze and responses to people.

Hwang Sung-jae

Chief executive of XYZ

Chief executive of the AI robotics company XYZ.