Johns Hopkins researchers describe ActOR, a system that merges a wall-mounted camera and a humanoid robot's head camera into one shared model of an operating room. A language-model agent plans with pre-programmed movement skills and retries or re-plans when something fails.
In simulated operating rooms rebuilt from real rooms, the shared view meant the robot needed far less searching to find objects, and feedback roughly doubled success when tasks were disrupted. The paper also reports a test in a physical mock operating room.
Most results are from simulation. The work is a preprint and reports no clinical use.
Source: arxiv.org
Limits
Mainly simulated results; physical test was a mock room; not peer reviewed.