← 🧠 Foundations of LLMs & GenAINEXT IN FOUNDATIONS OF LLMS & GENAILoRA and Parameter-Efficient Fine-Tuning→
Core
The KV Cache
In autoregressive decoding a model would recompute attention over the whole history at every step; the KV cache keeps each token's key and value vectors so a new token only attends and never recomputes. Compute is saved, but the cost shifts to memory: the cache grows with sequence length times batch size and usually turns into the binding constraint in serving. AI, ML, and GenAI interviews probe it because it explains why long contexts are costly to serve, why throughput (not model speed) is often the limit, and why MQA/GQA and PagedAttention exist.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
ML Infrastructure & GPUsServe a 70B-parameter model with high throughput. Do the memory math and name the optimizations.→ML Infrastructure & GPUsExplain the KV cache: prefill vs decode, why it grows, and how MQA/GQA and PagedAttention help.→ML Infrastructure & GPUsExplain speculative decoding and the other main levers for cutting LLM generation latency.→LLM & GenAI FundamentalsWhat are Multi-Query (MQA) and Grouped-Query Attention (GQA), and why do they exist?→ML Infrastructure & GPUsYour inference p50 is fine but p99 latency spikes under load. How do you fix tail latency?→ML Infrastructure & GPUsHow do you load-test and capacity-plan an LLM inference service before launch?→
COMPANIES THAT ASSUME THIS