Researchers led by Junwei Li describe a way for an autonomous excavator to work out the positions of its arm joints from a camera inside the cab and its recent joystick commands. The aim is to avoid fitting extra joint sensors, outside cameras or visual markers to existing machines.
The method keeps a rolling record of the bucket joint to cope with moments when the bucket is hidden, and it limits estimates to what the machine's joints can physically do. The estimates and the camera image feed a control policy that decides the next action.
In a simulation, across 20 autonomous digging trials, the system completed the task 95% of the time. Each step of estimating and controlling took on average 10.11 milliseconds, and the bucket-joint estimate was off by 0.965 degrees on average. The tests were in simulation, not on a real machine.
Background
Junwei Li
Researcher
The first author of a paper on estimating excavator joint positions from a cab camera.
