memory
AI, ML & GenAI interview questions tagged memory, across every topic.
9 questions · 0 unlocked for you
Concepts behind "memory"
The curriculum that explains the ideas these questions test.
Foundational
GPU Memory and the Serving StackServing an LLM is largely a memory problem: the GPU has to hold the model weights along with a KV cache that scales with sequence length and batch size, and inference divides into a compute-bound prefill and a memory-bandwidth-bound decode. Understanding the memory math (weights plus KV cache), why decode is bandwidth-bound, and the levers (quantization, batching, paged attention) is the bedrock of LLM serving. AI, ML, and GenAI engineer interviews probe it because 'will this model fit and how fast will it run?' is a recurring production question.🖥️ ML Infrastructure & Serving
Core
Agent Memory: Short-Term, Long-Term, and Memory StoresAgent memory is how an agent holds onto and recalls information across steps and sessions. Short-term (working) memory sits in the context window for the current task; long-term memory is durable information (facts, user preferences, past outcomes) kept outside the window and retrieved when it matters. The skill lies in choosing what deserves to be remembered, where to keep it, and when to read it back. Applied AI interviews test it because durable memory is what turns a one-shot chatbot into an agent that gets better over time.🤖 Retrieval & AgentsSign in
Core
Context Engineering for AgentsContext engineering is the discipline of designing the entire information payload that enters an agent's context window each turn: system instructions, memory, retrieved data, tool definitions and results, and conversation history. Most agent failures are context failures, where the right information is missing, buried, stale, or squeezing out the rest of the budget. Applied AI interviews test it because it is the highest-leverage lever on agent reliability and cost, and it sorts people who tune prompts from people who manage state.🤖 Retrieval & AgentsSign in
Core
Quantization and Low PrecisionQuantization holds and runs model weights (and activations) at fewer bits, FP16/BF16, FP8, INT8, INT4, rather than FP32, shrinking memory and accelerating inference for some accuracy cost. It is the primary way to fit a large model onto a given GPU and serve it cheaply, and it sits behind QLoRA fine-tuning and KV-cache compression. AI, ML, and GenAI engineer interviews probe it because 'how do you serve a 70B model affordably?' typically opens with quantization, so the precision ladder and its trade-offs are must-know material.🖥️ ML Infrastructure & ServingSign in
Core
PagedAttentionThe KV cache is the memory bottleneck in LLM serving, and naively reserving a contiguous block per request (sized for the maximum length) loses most of it to fragmentation and over-allocation. PagedAttention adapts virtual-memory paging: keep the KV cache in fixed-size non-contiguous pages allocated on demand, so memory is consumed only as tokens are generated. This fits far more concurrent requests onto a GPU, raising throughput. AI, ML, and GenAI engineer interviews probe it because it is the key memory innovation behind modern serving (vLLM).🖥️ ML Infrastructure & ServingSign in
Core
Streaming and BackpressureWhen data is too large to hold in memory or keeps arriving without end, you handle it as a stream, one piece at a time, with bounded memory, rather than pulling it all in. Backpressure is the mechanism that keeps a fast producer from swamping a slow consumer, by signaling 'slow down' instead of buffering without limit until memory runs out. Applied-AI interviews test it because AI pipelines chew through huge datasets and token streams, and the naive load-everything approach OOMs while unbounded buffering crashes under load.💻 Coding & Engineering CraftSign in