Core

Retrieval vs Long Context

If a whole document fits in a model's large context window, should you paste it, or retrieve only the relevant chunks? Long context is simpler but costly (quadratic attention), slower, and used unevenly (lost in the middle); retrieval is cheaper, faster, refreshes without retraining, and surfaces only what matters. The usual answer is retrieval for large, changing, or partially-relevant corpora, and long context for small, cohesive inputs. AI, ML, and GenAI engineer interviews test it because 'just use the big context window' is a common, costly oversimplification.

a free account unlocks the core curriculum tier · no card

RELATED CONCEPTS

The Context Window The RAG Pipeline Vector Search and ANN Indexes LLM Cost Optimization

PRACTICE THIS IN REAL QUESTIONS

RAG & Agent System DesignContext windows are now huge. When do you just stuff everything in context instead of building RAG?→RAG & Agent System DesignHow do you choose top-k and the context budget for RAG, given recall, noise, and cost all pull against each other?→RAG & Agent System DesignDesign a production RAG system over 10M documents serving ~1,000 QPS at sub-second latency.→System Design for AI in ProductionDesign a text-to-SQL feature: let users ask questions in natural language over a real database.→RAG & Agent System DesignWhat is query transformation in RAG (HyDE, decomposition, step-back), and when does each help?→RAG & Agent System DesignWhat is GraphRAG, and when does it beat traditional vector RAG?→

COMPANIES THAT ASSUME THIS

OpenAI Anthropic Google Microsoft Scale AI Cognition

NEXT IN RETRIEVAL & AGENTSAgent State, Checkpointing, and Durable Execution→