DualWorld

Current Vision-Language-Action (VLA) models face two major bottlenecks: scarce action-labeled data and reliance on manual task segmentation, which limit scalability and policy transfer. While world models trained on internet-scale video data offer a promising alternative, they often lack semantic structure in their representations, and the small diffusion backbones used for real-time control weaken physical reasoning. To overcome these limitations, we introduce DualWorld, a dual-world model system that operates asynchronously in visual space: A “slow” planning model​ performs full-frame rollouts over ~6-second horizons using a medium-sized video diffusion model, preserving strong generative priors for long-term reasoning. A “fast” reactive model​ encodes both predicted and real-time visual observations into actionable representations using a state-of-the-art video encoder, enabling real-time control. This design allows learning from large-scale, diverse-quality datasets—including human videos and cross-embodiment robot trajectories—without manual annotation or curation. Across manipulation and loco-manipulation benchmarks, DualWorld matches or surpasses leading VLA and compact world-model policies in success rate and generalization, demonstrating a viable path toward practical embodied AI.

Key Innovation: The world model provides consistent visual guidance throughout the entire manipulation sequence, while the fast encoder ensures real-time responsiveness (30Hz). This dual-system design enables humanoid robots to execute complex whole-body movements with both long-horizon coherence and immediate reactivity.