World Model for Robot Learning: A Comprehensive Survey

World models are predictive representations of how environments evolve under actions. In robot learning, they support policy learning, planning, simulation, evaluation, and data generation. This survey reviews how world models are coupled with robot policies, how they serve as learned simulators, and how robotic video world models progress toward controllable, structured, and foundation-scale formulations.

Recent world-model-based robot learning has progressed along two connected tracks: world models become increasingly integrated with policy generation, while learned simulators move from validation and ranking toward reinforcement learning, post-training, and co-evolving optimization.

Timeline coverage: through March 31, 2026. The timeline will be continuously maintained.

The survey separates how predictive generation interacts with action production, ranging from modular predict-then-act pipelines to tightly integrated end-to-end and latent-space formulations.

IDM-style policies

A world model predicts future observations, then an inverse-dynamics-style policy recovers actions.

DecoupledPredict then act

Single-backbone policies

Future visual evolution and actions are modeled jointly inside one shared generative backbone.

Shared backboneVideo-action

MoE / MoT-style policies

Specialized video, action, and language experts interact through fusion or shared attention.

Expert fusionJoint attention

Unified VLA models

Future-oriented prediction is internalized inside a multimodal VLA policy through visual foresight or structured world knowledge.

VLAForesightWorld knowledge

Latent-space world modeling

Future prediction is represented in compact latent space, reducing the need for explicit pixel-level decoding.

Latent dynamicsJEPA-styleEfficient control

The survey also treats world models as learned environments. In this view, a world model can generate imagined transitions for policy improvement, or roll out candidate actions to validate, rank, and evaluate likely outcomes before execution.

For reinforcement learning

The world model serves as a learned simulator that produces imagined observations, rewards, and termination signals for policy updates.

Imagined rolloutsPolicy updatePost-training

For evaluation

The world model evaluates candidate actions by predicting their consequences, enabling decision-time validation, ranking, and rollout-based scoring.

Action rankingValidationRollout scoring

Beyond policy coupling and simulator-style usage, the survey also reviews video-based world models by their modeling capability: from imagination-based generation for policy learning to action-controllable, structure-aware, and foundation-scale formulations.

A compact visual summary of robotic video world models organized by world-modeling capability.

For a compact homepage summary, we highlight two dataset/resource tables from the survey: core data attributes and relevance to embodied world-modeling capabilities.

Table 3: core attributes of representative embodied world-model training datasets/resources.

Table 4: relevance of representative datasets/resources to embodied world-modeling capabilities.

X-Emb.: cross-embodiment coverage. Act.: explicit action supervision or aligned action proxy. Obs./3D: strong observation support beyond basic monocular RGB, e.g., multi-view, depth, LiDAR, or 3D annotations. Lang.: language/task conditioning. M/C: multimodal or contact-rich signals such as force, tactile, audio, or dense proprioceptive/contact cues. ✓ denotes strong support, – denotes partial/moderate support, and ✗ denotes limited or no support.

@article{hou2026worldmodelrobotlearning,
  title   = {World Model for Robot Learning: A Comprehensive Survey},
  author  = {Hou, Bohan and Li, Gen and Jia, Jindou and An, Tuo and Guo, Xinying and Leng, Sicong and Geng, Haoran and Ze, Yanjie and Harada, Tatsuya and Torr, Philip and Mees, Oier and Pollefeys, Marc and Liu, Zhuang and Wu, Jiajun and Abbeel, Pieter and Malik, Jitendra and Du, Yilun and Yang, Jianfei},
  year    = {2026},
}