MQA

The efficiency lineage · 2019

The idea

Multi-Query Attention. The first move in the shrink-the-cache branch: keep the mechanism, share the memory.

PaperFast Transformer Decoding: One Write-Head is All You NeedShazeer · Google · 2019arXiv:1911.02150 →

Verdict

What it fixedOne K/V shared by all heads - the cache shrinks by the head count.

What it still costsQuality drops when you push it hard.

The story, animated

One drawer, four heads - watch the pile shrink by exactly the head count.

Watch

Episode 1, beat 7 · 64s, from the film series.

Where it sits

Worth having first: The memory wall

Unlocks: GQA