TLDR: We introduce CLAW, which uses a hypernetwork to generate LoRA adapters for a world model from a few seconds of data, adapting it to a new environment in a single forward pass.
Link: paper
The problem
World models trained on one set of environments break when the dynamics or embodiment change at test time. We want to adapt them to a new environment using only a handful of interaction episodes.
Our solution
During pretraining, we jointly train a base world model and a hypernetwork across many environments. The hypernetwork takes a short context of data and outputs low-rank (LoRA) adapters for the world model. At test time, the base model stays frozen. Adapting is just one forward pass through the hypernetwork, with no gradient steps.
On locomotion and manipulation tasks with varying dynamics and embodiments, CLAW beats gradient-based adaptation and in-context learning with only seconds of test-time data. It also avoids overfitting when data is scarce.