The machine underneath
Running the model & the world
The idea
Section titled “The idea”The ideas that won are the ideas the hardware liked. Memory bandwidth, not arithmetic, decides what is fast - the story of why FlashAttention mattered and what GPUs rewarded next.
The economics, in one line: an 8xH100 node rents for roughly $3 per GPU-hour, and GPUs fit this workload because giant matrix multiplications parallelise cleanly across their compute units (ch. 0:39-0:40). That demand curve is why Nvidia’s valuation sat around $3.4T when the talk was filmed (ch. 0:41).
The story, animated
Section titled “The story, animated”Matrix-shaped work fills the chip; scattered work starves it - the economics of why FlashAttention was the trick the hardware rewarded.
Where it sits
Section titled “Where it sits”Worth having first: FlashAttention
Episode 4 · The Machine Underneath
Source-checked against Karpathy, “Deep Dive into LLMs like ChatGPT” (2025).