Tokens to think

Inside the blocks

The idea

Every token gets exactly one pass through the same fixed stack of blocks - the network spends roughly the same compute whether the next token completes a greeting or a proof. So a hard answer crammed into one token fails: force a word problem to be answered immediately and large numbers break the model, while the same model solves it when allowed to work through intermediate tokens first (Karpathy’s demo at ~1:46-1:55).

This is the bridge from architecture to reasoning models: chains of thought are not decoration, they are where the computation lives. Reinforcement learning later discovers that spending tokens wins - this page is why.

Try it

Where it sits

Worth having first: Rows ask questions · Sampling

Unlocks: RL & reasoning models

Source-checked against Karpathy, “Deep Dive into LLMs like ChatGPT” (2025), ch. 1:46-2:01.