inference efficiency

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is understanding they share key/value heads across query heads to trim memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

Plain multi-head attention is rarely the whole exercise: the usual extension is to chain it into KV-cached decoding and GQA in one sitting. The locked answer walks the NumPy implementation, the equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.