DreamerV3

World models · 2025

The idea

The JEPA pages ask: can you predict the next latent well enough to plan? Dreamer asks a sibling question: can you train the policy entirely inside a world model you learned, so the agent practices in imagination instead of burning real environment steps?

That loop is older than the 2025 paper - Ha and Schmidhuber’s 2018 World Models, then DreamerV1/V2. DreamerV3 (Hafner et al., Nature, 2025) is the edition that made the recipe boring in the good sense: one fixed hyperparameter configuration across more than 150 diverse control tasks, from continuous robotics-style benchmarks to Atari to Minecraft.

The Minecraft result is the headline people remember. Collecting a diamond from scratch - no human demonstrations, no task-specific curriculum - had been treated as a hard open problem; prior strong approaches leaned on expert data (VPT) or adaptive curricula. DreamerV3 agents discover diamonds within 100 million environment steps, out of the box.

What the loop actually is:

1. Learn a world model from the agent’s experience (encode pixels → latent dynamics + reward).
2. Train the actor-critic only on imagined rollouts of that model.
3. Deploy the actor in the real environment; feed new experience back.

Judge the world model on whether plans and policies survive contact with the real game - not on how pretty a decoded dream looks.

The misconception to name

It is tempting to collapse Dreamer into “just another video model.” The point of the Dreamer program is imagination training: the policy never needs the real env during its gradient updates. That is the same economic motive as offline robot learning - and the bridge to Dreamer 4.

Also: DreamerV3 still collects experience online. Diamonds from scratch ≠ diamonds from a fixed offline dataset. That distinction is the next page.

Verdict

What it fixedOne fixed hyperparameter set across 150+ control tasks; first agent to collect Minecraft diamonds from scratch with no human demonstrations, by training the policy inside imagined rollouts.

What it still costsLearns from online environment interaction. The world model is accurate for its training domains, but the architecture was not built to fit open internet video the way later generative simulators attempt.

The story, animated

A real closed loop: learn the model, train the policy purely in imagination, act for real, and feed that back in.

Where it sits

Worth having first: Three programs · What is a world model

Unlocks: Dreamer 4

Contrast with the JEPA planner spine: V-JEPA 2-AC plans by searching in a frozen encoder’s latents; Dreamer trains a policy by RL inside its own model.

Source-checked against Hafner, Pasukonis, Ba et al., “Mastering diverse control tasks through world models,” Nature 640, 647–653 (2025).