Researchers led by Wei Liu have built LoRN, Low-Rank Noise Steering, a reinforcement learning method that improves large pretrained robot policies by choosing their starting noise without changing the model's weights, according to a paper on arXiv. The team found that the actions of world-action models depend more on the video noise than on the action noise, which makes video noise a strong handle for steering.
Existing noise-steering methods struggle with the very large noise inputs of world-action models. In LoRN, a small actor picks a direction through a fixed low-rank random projection, and the result is kept near the usual scale of Gaussian noise, so both video and action inputs can be steered.
Across three world-action models and ten pairs of model and task, steering the video lifted mean success 26.2 percentage points above the base policy, against 7.1 points for steering actions with DSRL. Actors trained in simulation also worked on three physical two-armed manipulation tasks without extra training in the real world.



