inference efficiency

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is understanding they share key/value heads across query heads to trim memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

The 2026 ML coding round has moved past plain multi-head attention: labs now chain it into KV-cached decoding and GQA in a single sitting. The locked answer walks the full NumPy implementation, the one equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.