Dreamer 4

World models · 2025

The idea

DreamerV3 still had to play Minecraft while learning - online interaction, a hundred million steps. What if interacting with a half-trained agent is unsafe or too slow, as it is for many robots? Then you need the same imagination loop on a fixed offline dataset.

Dreamer 4 (Hafner, Yan, Lillicrap, 2025, arXiv:2509.24527) is that challenge made concrete: obtain diamonds in Minecraft from only offline data - sequences of over 20,000 mouse and keyboard actions from raw pixels - with no environment interaction during training. The paper reports it as the first agent to do so, substantially improving on OpenAI’s VPT offline agent while using about 100× less labeled data.

The world-model side is where the engineering moved. Prior Dreamer models were fast in narrow domains but not built to fit complex interaction distributions; Genie-class video models scale but often need many GPUs for real-time rollouts and still struggle at precise object mechanics. Dreamer 4’s answer:

• a high-capacity transformer world model that predicts object interactions and game mechanics well enough for RL inside the dream
• a shortcut forcing objective (shortcut models + flow matching) so generation needs only a few forward passes - real-time interactive inference on a single GPU
• learn most knowledge from unlabeled video , and action conditioning from only a small aligned subset

Decode the imagined training sequences and you see breaking blocks, using tools, crafting - the model has to simulate mechanics, not just pretty frames, or the policy cannot find diamonds in imagination.

The misconception to name

Two slides people blur:

1. DreamerV3 diamonds ≠ Dreamer 4 diamonds. V3: online, from scratch. V4: offline dataset only. Same item, different constraint.
2. Not JEPA. Dreamer 4 is in the generative-latent family (diffusion / flow / shortcut). It does generate observations you can decode. That sits in productive tension with the JEPA claim that pixel prediction is wasted capacity - both programs now have strong agent results; the fork is unresolved, not settled by a demo.

The paper is also honest about leftovers: short memory, imprecise inventory predictions - not a full game clone.

Verdict

What it fixedFirst agent to obtain Minecraft diamonds purely from offline data - no environment interaction during training - with real-time single-GPU imagination via shortcut forcing.

What it still costsStill far from a full Minecraft clone (short memory, imprecise inventory). Generative latent predictions, not JEPA-style embedding-only forecasts - the pixels-vs-latents tension stays open.

The story, animated

DreamerV3’s loop, with the live environment connection severed - fed only from a fixed tape, decoding to real pixels.

Where it sits

Worth having first: DreamerV3

Contrast: LeWorldModel and V-JEPA 2-AC plan in embedding space without this generative stack; Three programs is where that fork was named.

Next on this track: Genie 3 and NVIDIA Cosmos - the other column that keeps the pixels.

Source-checked against Hafner, Yan, Lillicrap, “Training Agents Inside of Scalable World Models” (arXiv:2509.24527).