What is a world model

World models

The idea

Suppose you are about to tip a glass. You do not need to tip it to know the water will spill. Something in your head predicted the next state of the world given an action. That is the bet a world model makes in silicon.

Strip the branding and the definition is narrow: a learned function that maps (state, action) → next state (and often a reward). It answers what happens if I do X without doing X. That is different from a policy (what should I do) and different from a plain generative model (what usually comes next, with no commitment to a persistent state underneath).

Why build one?

1. Sample efficiency - real robot time and user sessions are expensive; practice inside a model you carry in your head.
2. Planning - roll out candidate action sequences, score the imagined futures, pick the best. No transition function, no lookahead.
3. Counterfactuals - evaluate irreversible actions before committing.

The modern starting point is Ha and Schmidhuber’s 2018 World Models: compress pixels with a VAE, learn latent dynamics with a recurrent net, train the controller entirely inside the dream, then transfer back to the real environment. Dreamer (later pages) is that loop grown up. JEPA (next pages) keeps the “predict the next state” part and moves the prediction into representation space so you do not spend capacity painting leaves.

The misconception to name

It is tempting to call any impressive video model a world model because it “understands physics.” That misses the test. Sequence prediction can be nearly flawless while the implied world is incoherent - the Manhattan taxi experiment (Vafa et al., NeurIPS 2024) is the cautionary picture: perfect next-turn guesses, a reconstructed street map that could not exist, and collapse under detours. A world model is graded on whether plans made inside it survive contact with the real environment, not on how pretty the rollouts look.

The story, animated

The loop, named - then two near-neighbors each missing one piece - then the taxi-map diagnostic made visible.

Where it sits

Worth having first: The next-token objective

Unlocks: Three programs

Framing drawn from Ha & Schmidhuber (2018); the taxi-map diagnostic is Vafa et al., NeurIPS 2024.