Researchers led by Achyuthan Unni Krishnan have built a loop in which people show a robot how to grasp objects it keeps failing to pick up, according to a paper on arXiv. A grasping policy that works directly on point clouds from a depth sensor can handle new objects, and reinforcement learning makes it better on average, but some grasps stay unsolved, the authors write.
Their system finds those cases automatically and sends them to a virtual-reality app, where a person lifts the object with their own hands. Only the two grasp points go back into training; the robot works out its own approach.
In simulation, reinforcement learning raised success on unseen objects from 55% with the Contact-GraspNet generator to 83%. Adding 58 human demonstrations nearly doubled success on the hard cases from 24% to 45%, also helped similar cases nobody demonstrated, and did not lower success on unseen objects.



