vllm
AI, ML & GenAI interview questions tagged vllm, across every topic.
6 questions · 2 unlocked for you
Concepts behind "vllm"
The curriculum that explains the ideas these questions test.
Core
PagedAttentionThe KV cache is the memory bottleneck in LLM serving, and naively reserving a contiguous block per request (sized for the maximum length) loses most of it to fragmentation and over-allocation. PagedAttention adapts virtual-memory paging: keep the KV cache in fixed-size non-contiguous pages allocated on demand, so memory is consumed only as tokens are generated. This fits far more concurrent requests onto a GPU, raising throughput. AI, ML, and GenAI engineer interviews probe it because it is the key memory innovation behind modern serving (vLLM).🖥️ ML Infrastructure & ServingSign in
Core
Model Serving FrameworksYou seldom build a serving stack from scratch; frameworks take care of the production plumbing. General servers (Triton, TorchServe, KServe) host many model types with dynamic batching, multi-model hosting, and versioning. LLM-specific servers (vLLM, TGI, TensorRT-LLM) add the essentials general servers miss: continuous batching, paged KV cache, and token streaming. AI, ML, and GenAI engineer interviews probe it because knowing what these provide, and that LLM serving needs the specialized ones, is practical deployment knowledge.🖥️ ML Infrastructure & ServingSign in