The planner's path integral
World models · 2026 · Lab: Two Rooms
Light takes every path and adds them up; so does a sampling planner. The difference is one sign in the exponent, and this page lets you watch what that sign does: a light wave through the door, Feynman’s mirror, an MPPI planner you can run round by round, and the arrival-time field that turns out to be the ideal planning cost.
A photon and a planner both try every path
Feynman described light as taking every route at once and adding them all up. The planner in our Two Rooms world model does something very close: before it moves, it imagines hundreds of routes and adds those up too. For a class of noisy control problems the two sums are mathematically related, and the main difference is one sign: i in the exponent for light, −1 for the planner. That sign decides whether routes can cancel each other.
How light picks a route
In Feynman's picture, light going from a lamp to your eye by way of a mirror takes every route, one for each point of the mirror. A screen blocks the direct route. Each route carries a small arrow that turns as the light travels, one full turn per wavelength. To find how much light arrives, you lay all the arrows head to tail and measure from the first tail to the last head.
Routes that bounce near the middle of the mirror take almost the same time, so their arrows point the same way and add up. Routes near the edges differ quickly in time; their arrows curl into spirals and cancel. What survives is the route of least time, where the angle in equals the angle out.
- Total that arrives
- 100
- What the strips show
- Colour = arrow direction
Try this. Pick Left edge only: its arrows curl and cancel, so little light arrives (12, against 100 for the whole mirror). Now pick etched, which scrapes away the strips whose arrows point backwards. The arrows that remain all lean the same way, and the light comes back (63). That is a diffraction grating. Switch to the planner and repeat: its arrows all point one way, so removing routes can only make the total smaller.
How our planner picks a route
Our planner never moves while it thinks. It imagines a sequence of pushes, plays it forward inside the world model, and gives it a cost: how far the imagined dot is from the goal, added up along the way. It does this for many random sequences, then averages them, giving each the weight e−cost/λ. It carries out the first few pushes of the average plan, looks again and repeats. This is MPPI, model predictive path integral control.
The planner in our film uses a hard version of the same idea, the cross-entropy method: of 512 imagined 50-step routes it keeps the best 48, redraws new routes around them, and repeats eight times. In its first round only 3% of the imagined routes ended in the goal room. By the sixth round every route it kept went through the door. Kappen saw the same thing in his control version of a double slit: with plain random sampling, "most of the trajectories are killed at the infinite potential".
- Round
- 0
- Imagined routes reaching the goal room
- –
- Routes that carry the weight
- –
- Average plan reaches the goal
- –
Try this. Raise λ and bad routes start to count, so the average plan grows timid. In 30 random draws at λ = 0.3, plans scored by walking distance all crossed the wall, while plans scored by straight-line distance all stayed stuck on the left: the straight line looks shortest, but it runs into the wall. Here the planner scores the dot's real position so you can watch it. Our world model scores distances between its 32-number codes, which is why the shape of that code space matters (see wave packets, below).
What changes when i becomes −1
Replacing i by −1 in the exponent is what physicists call moving to imaginary time. It turns the mathematics of waves into the mathematics of heat and random walks. In 2005 Bert Kappen showed that for a class of noisy control problems, writing the cost-to-go as an exponent turns the optimal-control equation into a linear one, and the best action becomes an average over noisy routes weighted by e−S/λ. He points out that this same log transform "was first used by Schrödinger to relate the Hamilton-Jacobi formalism to the Schrödinger equation". The link is exact only for that class of problems, where actions enter the motion linearly and the noise is tied to the cost of acting. MPPI borrows the same weighting as a practical method beyond it, and runs the average inside model predictive control. CEM is a cousin that keeps the best routes and refits, so its climb from 3% to 100% is the sampler learning where to look; nothing in it interferes. Meta's JEPA example on Two Rooms, with the wall redrawn for every episode, reaches 97% with MPPI and 96% with CEM.
| Light (Feynman) | Planner (path-integral control) | |
|---|---|---|
| Each route adds | an arrow of fixed length that turns: eiS/ħ | an arrow that points one way and fades: e−S/λ |
| Routes far from the best | cancel each other | get weights too small to matter |
| The scale | ħ, fixed by nature | λ, a setting we choose |
| Shrink the scale | classical physics: only the least-action route survives | a hard choice: only the cheapest routes survive (CEM keeps 48 of 512) |
| Remove some routes | the total can grow (a grating) | the total can only shrink |
| Name of the sum | path integral, in real time | Feynman–Kac formula, in imaginary time |
A double slit for a planner
Open a second door in the light simulation at the top. Light goes through both doors at once and the two waves interfere, leaving bright and dark bands in the right room. A planner cannot do that. It has to pick a door.
Kappen worked this out as a control problem: a particle must pass a barrier with two openings. Far from the barrier, the best noisy controller steers between the openings and delays its choice. Once the time left is short compared with the noise (when νT < 1), it commits to one. Physicists call this symmetry breaking. In the same paper a drunken spider takes the narrow bridge across a lake when it is steady, and walks the long way around when it is drunk.
Our planner shows a version of this, set by its temperature. In 30 random draws of the simulation above with two doors at λ = 0.1, every plan reached the goal: 20 through the upper door and 10 through the lower, decided by chance. At λ = 1, plans with one door crossed the wall in 29 of 30 draws; with two doors, none did. The average of an upper-door plan and a lower-door plan heads straight into the wall between them. Kappen's delayed choice belongs to the best possible noisy controller. MPPI and CEM see only a finite sample of routes, so they can commit too early or, as here at high temperature, average two good routes into a bad one.
The best cost is light's arrival time
The cost of the best route from any point to the goal is called the cost-to-go. In mechanics it is Hamilton's principal function, and it obeys the Hamilton–Jacobi equation. For a dot that moves at a steady speed around walls, that equation becomes |∇T| = 1, the eikonal equation of geometric optics, which describes how a light front spreads. So the ideal planning cost is the time light would take to arrive from the goal.
Each cell gets its arrival time from its neighbours, the way every point of a wavefront sends out new wavelets (Huygens). The rule is local, so it works for any door. Following the arrival time downhill gives the shortest route, which is Fermat's principle. This walking distance is the cost our deep-research review recommended for the spatial world model, and it is the default cost in the planner above.
- Walking distance, start to goal
- –
- Straight line (blocked)
- 0.700
Why one door was enough for the spatial model
Physics generalises because its laws are local and the same everywhere. A wall or a slit enters only as a boundary condition. Our first world model squeezed the whole room into 32 numbers, so it learned answers for the doors it had seen. The spatial model keeps a 16 × 16 grid and updates it with 3 × 3 rules, the same kind of object as a discretised derivative. To that model, a moved door is a new boundary condition for the same local law.
Trained on the middle door only, the spatial model predicted correctly whether the dot gets through at 89–100% of heights at every door from 0.1 to 0.9, in two training runs. With time-similarity added, it also plans at doors it never saw:
| Door height | Spatial model, trained on door 0.5 only | Vector model, trained on door 0.5 only |
|---|
Trips that reached the goal, out of 24 per door, with the same reduced planner for both models (128 imagined routes, 6 rounds, 40 steps ahead). Every vector model we trained, including those trained on several doors, reached the goal in only 2–6 of 24 trips at doors 0.1 and 0.9. A second training run of the spatial model reached 18, 24 and 18 of 24 at doors 0.1, 0.5 and 0.9. A direct test then separated predictions from the measure of progress: scored with the true walking distance instead of its own code distance, the original spatial model, without time-similarity, reached the goal in 199 of 216 trips instead of 7. Its predictions were right at every door; only its sense of progress was flat.
Why the spatial model could not plan at first
The spatial model stores the dot as a blob on its grid, much like a wave packet. The distance between two codes depends on how much the two blobs overlap: distance² = 2 − 2 × overlap. Once two blobs stop overlapping, every separation looks equally far, and the planner has no slope to follow. Widen the blob and the distance keeps growing over a longer range.
| Spatial model (trained on door 0.5) | 90% of a dot's code lies within | Code distance stops growing at | Planning at doors 0.2, 0.5, 0.8 |
|---|---|---|---|
| Raw pixels, for reference | 0.066 of the room | – | – |
| Original | 0.099 | 0.10 | 1 of 72 |
| Time-similarity, weight 1 | 0.133 | 0.15 | 26 of 72 |
| Time-similarity, weight 10 | 0.159 | still rising at 0.20 | 64 of 72 |
Measured on our models. Real codes are not perfect Gaussians, so the numbers differ from the toy above, but the pattern is the same: time-similarity spread each dot's code over a wider patch, the distance kept growing for longer, and planning went from 1 to 64 successful trips. This is a correlation across three models. Time-similarity may also reshape the code in other ways; training a model whose blobs are equally wide without time-similarity would separate the two causes.
Where the analogy stops
- No interference. The planner's weights are all positive, so it can never make a grating effect or a dark fringe. It is the classical, thermal relative of Feynman's sum.
- Least or stationary. Nature picks a route of stationary action, which can be a saddle. A planner always wants the true minimum.
- A constant or a setting. ħ is fixed by nature. λ is a number we tune, and it changes what the planner does.
- Real or imagined routes. Light's routes are the physics itself. The planner adds up a few hundred routes imagined inside a learned model. When that model is wrong, as with the moved door, the sum is taken over the wrong world.
- R. P. Feynman, QED: The Strange Theory of Light and Matter (1985): the mirror, the arrows and the grating.
- H. J. Kappen, Path integrals and symmetry breaking for optimal control theory (2005): the double slit, delayed choice and the drunken spider. arXiv physics/0505066
- H. J. Kappen, Linear theory for control of nonlinear stochastic systems (2005). arXiv physics/0411119
- G. Williams et al., Information-theoretic model predictive control. PDF
- G. Williams, A. Aldrich, E. Theodorou, Model predictive path integral control using covariance variable importance sampling. arXiv 1509.01149
- Meta FAIR, eb_jepa action-conditioned Two Rooms example. README
- Sobal, Zhang et al., PLDM: planning with latent dynamics models (time-similarity). arXiv 2502.14819
- Our experiments: jepa_two_rooms/video3 (door_rules.py, spatial_check.py, spatial_ts_full.py) and the deep-research review deep_research_extrapolation.md.
Back to the experiment summary: Lab: Two Rooms