RAG & Agent System Design

One of the most common AI, ML, and GenAI design rounds. Sketching embed-retrieve-generate is easy; the signal lives in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents get reached for too often. A strong answer pushes past the hype: most tasks are better served by a single structured call, and agents pay off only under particular conditions. Here is when one is worth it and how to stop it from compounding errors.

The two knobs that decide whether a RAG system works, and the two candidates tend to gloss over. The signal is tuning chunking against recall and knowing precisely what dense retrieval drops that BM25 recovers. Here is the reasoning, not the rules of thumb.

Most RAG systems ship without genuine evaluation, which is why most of them quietly degrade. The signal is breaking evaluation into retrieval and generation, and scoring faithfulness apart from relevance. Here is the framework that makes RAG improvements measurable.

Enterprise RAG breaks on isolation, not on retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user is not allowed to see. Here is the design that survives a security review.

A 2025-2026 question that trips up people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for deciding.

Agents and integrations depend on the model returning valid, schema-conforming output. The signal is stacking constrained decoding, schema validation, and retries, rather than trusting a prompt to do the job. Here is the production-reliability answer.

Every RAG system rests on a vector index, and the interviewer wants proof you grasp the recall/latency/memory tradeoff, not just 'use a vector DB.' The signal is why exact search will not scale and how HNSW and IVF place different bets.

Multi-agent is the pattern most often reached for without cause in AI today. The signal is holding back unless the task genuinely calls for specialization or parallelism, knowing the supervisor and handoff patterns, and grasping why coordination magnifies failure.

Conversations and agent tasks grow past the context window, and the naive 'stuff the whole history' approach breaks on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieval quality is bounded by the query, and raw user queries are frequently poor for search. The signal is knowing the techniques that rewrite the query ahead of retrieval and which failure each one fixes.

Vector RAG quietly breaks on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing precisely what the graph gives you and the construction cost it demands.

Retrieving every time is wasteful and occasionally harmful; retrieving never leads to hallucination. Self-RAG turns retrieval into a decision the model controls, then critiques what came back. The signal is the retrieve-on-demand plus self-critique loop.

MCP standardized how agents connect to tools and data, but tool design is what determines reliability. The signal is knowing what MCP actually standardizes and the principles behind a tool the model can call correctly from its description alone.

Reflection is the pattern in which an agent critiques and revises its own work. The signal is knowing when it truly helps (external feedback grounds the critique) versus when self-critique on its own is theater.

Chunking quietly caps RAG quality, and 'split every 500 tokens' leaves recall on the table. The signal is knowing the strategy families and the parent-child trick that separates retrieval size from context size.

Citations turn a RAG answer from 'trust me' into something checkable, and they are a genuine engineering problem. The signal is grounding each claim to a specific passage and verifying the attribution, not tacking a bag of links onto the end.

Real corpora contradict themselves (old versus new policy, different teams), and naive RAG quietly picks one or fuses them into a wrong answer. The signal is spotting conflict and resolving it through recency, authority, and transparency.

Plan-and-Execute and ReAct are the two leading agent control patterns. The signal is knowing the upfront-plan versus step-by-step tradeoff, why one costs less, and why most production agents wind up blending them.

Agents need memory beyond the context window, and 'just store the history' is the wrong answer. The signal is telling the memory types apart and mapping each to a storage and retrieval strategy. Here is the answer.

Context engineering is the discipline that took over from 'prompt engineering' for serious agent work. The signal is grasping that what you place in the context window (and what you omit) shapes behavior more than clever wording. Here is the answer.

The embedding model is the base of retrieval, and swapping it is deceptively risky. The signal is choosing on domain-relevant retrieval quality (not a leaderboard) and knowing that a new model means re-embedding everything. Here is the answer.

Code-execution agents are powerful and risky: arbitrary model-generated code runs on your infrastructure. The signal is genuine isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Tool selection and parameter extraction are where agents quietly break. The strongest answer treats the tool spec as the only thing the model sees, then repairs the design, not the model. Here is how.

Dense versus sparse is the core retrieval tradeoff, and 'just use embeddings' quietly loses every exact-match query. The signal is knowing what each representation captures and why production blends them. Here is the answer.

Agents fail in the middle, not only the end, so final-answer-only scoring conceals the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and pinpointing where it broke. Here is the answer.

Sub-agents are how you scale a complex agent task without one huge, noisy context. The signal is context isolation and clean delegation: a focused subtask in, a compact result out. Here is the answer.

Multi-agent systems succeed or fail on how agents share information and hand off work. The signal is naming the three communication patterns and the coordination failures that sink the naive version. Here is the answer.

Garbage parsing means garbage retrieval, and naive PDF text extraction wrecks tables and reading order. The signal is fixing the parser, not the retriever. Here is the answer.

Real documents carry images, charts, and tables, not only text, and text-only RAG quietly drops them. The signal is knowing the two retrieval approaches and why the generator has to change too. Here is the answer.

One model can behave completely differently depending on the harness wrapped around it. The signal is recognizing that the scaffolding frequently sets agent quality more than the underlying model does. Here is the answer.

A computer-use agent operates a real browser or desktop via screenshots and clicks. The signal is the perceive-decide-act loop together with the containment for an agent that can click, buy, or delete anything. Here is the answer.

RAG and agents turn expensive and slow quickly: retrieval plus reranking plus big-model calls, multiplied across agent steps. The signal is naming the dominant cost first, then the levers that genuinely move it. Here is the playbook.

An agent that performs actions is far riskier than one that only talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

Reranking is the cheapest large win in RAG quality: pull a broad set, then reorder it precisely. The signal is the bi-encoder-versus-cross-encoder distinction and the retrieve-then-rerank two-stage design. Here is the answer.

Pure vector search drops exact terms; pure keyword search drops meaning. The signal is understanding why each fails on its own and how to merge two rankings on incomparable scales without hand-tuning weights. Here is the answer.

Single-shot retrieval breaks on questions that chain facts ('who directed the highest-grossing film of 2019?'). The signal is decomposing or iterating retrieval rather than retrieving once, and guarding against errors compounding across hops. Here is the answer.

A RAG system is only as current as its index, and stale or duplicated content quietly degrades answers. The signal is an incremental pipeline with deletes, updates, and recency signals, rather than a one-time bulk load. Here is the answer.

Late interaction reaches cross-encoder-like quality with near-bi-encoder scalability. The signal is the per-token embeddings plus MaxSim matching that earns you the middle ground, and understanding what it costs. Here is the answer.

Pushing every query through one pipeline wastes latency, money, and precision. A router decides what each query actually needs. The signal is naming the four decisions a router makes and how you stop a misroute from sinking the answer.

A short question and the passage that answers it sit in different neighborhoods of embedding space. HyDE bridges that gap by embedding a fake answer rather than the question. The signal is understanding why a wrong hypothetical still retrieves the right docs.

Small chunks match precisely but starve the LLM of context; big chunks carry context but match badly. Parent-child retrieval declines to choose. The signal is separating what you match on from what you feed the model.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time rather than query time.

Fixed-size chunking cuts a coherent idea mid-thought; semantic chunking splits on meaning. The signal is naming the methods (structure-aware, embedding-similarity, LLM-based) and the tradeoffs that decide which one to ship.

Standard RAG retrieves once and generates; agentic RAG makes retrieval an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

Standard RAG trusts whatever it retrieves, even when it's irrelevant. Corrective RAG scores retrieval quality and acts on it. The signal is the grade-then-correct loop: re-retrieve, fall back to web search, or discard.

RAG failures are usually retrieval failures, yet most teams measure only the final answer. The signal is scoring retrieval on its own (recall@k, precision@k, MRR/nDCG) to pin down exactly where the system breaks.

Late chunking reverses the usual order of operations to solve the lost-context problem with zero additional LLM calls. Everything hinges on which step runs first. Here is the answer.

At a billion vectors the index choice is a memory budget question before it's a recall question. Flat search is off the table, HNSW may not fit, and PQ swaps recall for RAM. Here is the call a staff engineer makes on the whiteboard.

A model that leads MTEB can still stumble on your jargon-heavy corpus, and most teams reach straight for the costly fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

Top-k by raw similarity often feeds the model five paraphrases of one paragraph, denying it the other facts the question needs. The fix is ranking for relevance and novelty together. Here is how.

An agent that retries the same failed action 40 times means a runaway bill and a stuck user. Termination is something you build into the harness, not something the model reliably decides. Here is the control layer.

Real tools time out, rate-limit, and hand back malformed JSON. An agent that assumes every call succeeds is a demo, not a product. Here is the error-handling layer that keeps it running in production.

Real users type 'what about the new policy?' with no context. Retrieving on those four words yields noise. Here is how strong systems disambiguate before they ever reach the retriever.

When the right answer isn't even in the top-50, the generator can't rescue you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

More chunks means higher recall but also more noise, more cost, and more lost-in-the-middle. The right k is an empirical tradeoff, not a default of 5. Here is how to find it.

When a chain of LLM calls and tools yields a wrong final answer, 'the model was bad' is not a diagnosis. You need visibility into every step. Here is the tracing layer that turns a black box into something debuggable.

A 90% eval score alongside angry users means your eval set doesn't resemble reality. The fix is to make evaluation follow production, not the reverse. Here is how.

Frameworks promise speed and hand you abstraction you'll eventually fight. Rolling your own is more code but full control. Here is the honest tradeoff, and why many production teams end up thinner than they started.

The industry uses 'agent' for anything that calls an LLM. The distinction that matters is how much control you hand to the model, and more autonomy is not better. Here is the spectrum and the decision rule.

A live-coding round that tells apart engineers who have shipped agents from those who have only read about them. Interviewers want a genuine plan-act-observe loop, tools that query a real SQL database, guardrails that block the obvious failures, and a concrete evaluation plan, rather than one prompt masquerading as an agent.

A bridge connecting classic DSA and modern RAG. Interviewers want to see you build a working inverted index and a correct BM25 score by hand, reason about its complexity, and then know precisely when you would switch to embeddings and an ANN index instead.

Late interaction keeps one vector per token, so a corpus that fit in a few GB as single vectors can swell 100x. Interviewers want to see you know the compression and indexing tricks (centroids, residuals, PLAID) that make multi-vector retrieval shippable.

A single embedding cannot capture a question with three independent parts. Interviewers want to see you know when to split a query, how to retrieve and recombine per sub-query, and the latency and drift costs that make decomposition a net loss on simple questions.

HyDE produces a fake answer to embed, which helps on vague queries but actively hurts on factual or out-of-domain ones. Interviewers want to hear you name precisely when the hypothetical document misleads retrieval and which multi-draft and hybrid variants recover.

Vector search over rows is the wrong tool when the user asks for a count or an aggregate. Interviewers want to see you retrieve the right schema, generate validated SQL, and know when to query the database instead of embedding it.

Splitting source files every 500 characters slices functions in half and wrecks retrieval. Interviewers want to see you chunk on syntax, retrieve by symbol and dependency, and blend lexical exact-match with semantic search the way code search actually needs.

Re-embedding 10M documents nightly is wasteful when only 0.5% changed. Interviewers want to see you upsert by stable id, handle deletes and tombstones in an ANN index, and compact before fragmentation tanks recall and latency.

An LLM can cite a source that does not say what the answer claims. The signal is verifying claim-by-claim entailment against the cited text, not merely that a citation marker exists, and knowing what to do when grounding fails.

An answer can be correct yet still unfaithful, asserting things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Pushing top-k higher raises recall but buries the answer in noise, and a tight reranker raises precision but can discard the one chunk you needed. The signal is knowing which metric caps the system and how to move both with a retrieve-wide-then-rerank shape.

Hybrid retrieval only beats either method when the fusion is tuned. The signal is knowing why you cannot simply add BM25 and cosine scores, how RRF's k constant behaves, and when a learned weight beats rank fusion.

An off-the-shelf cross-encoder is general; your domain carries jargon and relevance rules it never encountered. The signal is knowing when fine-tuning pays off, how to mine hard negatives, and how to avoid training a reranker that merely memorizes your retriever's mistakes.

A better embedding model is worthless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

Linear agents lock into one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from decaying.

Tools fail: timeouts, rate limits, bad arguments, garbage output. A naive agent retries blindly or quits. The signal is classifying errors and matching each to the right recovery, with hard caps.

Multi-agent systems coordinate through handoffs and shared state. The signal is knowing the orchestration topologies, what a clean handoff protocol carries, and why naive multi-agent often loses to a single good agent.

Most people know MCP exposes tools. The deeper signal is the complete primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy conceals how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is grasping compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Agents can critique and retry their own work. The signal is knowing how Reflexion's verbal feedback loop operates, when self-critique genuinely helps versus when it is theater, and what makes it real.

Autonomous agents still need humans at the right moments. The signal is designing checkpoints: where to interrupt, what to surface, how to persist and resume state, without reducing the agent to a click-through rubber stamp.

Any content an agent reads can smuggle in instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Agents re-send growing context every step, so cost scales worse than linearly. The signal is knowing where the tokens go and the levers (caching, model routing, context trimming, step caps) that shrink the bill.

Screen-driving and browser agents break in ways chat agents never hit: stale DOM, pages that keep shifting, misplaced clicks. What interviewers watch for is the grounding and reliability methods that move a flaky demo toward something dependable.

A 0.91 cosine score and a useless passage are not a contradiction. The candidates who score here name the encoding mismatch, the topical-but-not-answering chunk, and why an absolute score cutoff was never going to work.

Every RAG design says 'just filter by tenant_id.' Almost nobody can explain why that one line can quietly cut recall in half. The answer is in what the filter does to the ANN graph.

The CRM says the balance is $0 and the ledger says $412. An agent left to its own devices will average them into something confident and wrong. Designing the precedence before the model has to guess is the whole answer.

Six stages, six isolating experiments, six metrics. The candidates who score do not guess at knobs, they bisect the pipeline, and they instrument the rungs nobody else does: ranking, context assembly, generation.

Retrieval is exonerated, so the bug lives in generation. There are four usual causes and they have different fixes, and the one nobody names is that the model was never given permission to say it does not know.

Freshness and reproducibility pull in opposite directions, and almost every team builds only for freshness. A pointer into a mutable index is not a record. Here is what an auditable knowledge base actually requires.

A dense encoder given 'refund policy' has almost nothing to work with. The fix is counterintuitive and it is not a better embedding model, and the reason this bug survives your eval set is worse than the bug.

float32 is far more precision than ranking needs. int8 is close to free, binary is 32x smaller, and the difference between a working binary index and a recall disaster is one word: rescoring.

An hour-long agent run is a distributed workflow wearing an LLM hat. The signal is designing serializable state with step-level checkpoints, and knowing why resume is easy while replaying a side effect is not.

Tool calling costs one LLM round trip per action. Code costs one round trip for a loop over 500 records. The signal is knowing exactly what you trade away when you let the model write the program instead of the call.

Interviewers ask about these by name and listen for whether you have read the source or only the README. The signal is explaining the runnable interface, why an agent needs a cyclic graph rather than a chain, and what the layers hide.

Agent latency is a step-count problem disguised as a speed problem. The winning move is removing round trips, not shaving milliseconds off each one, and knowing which step you can delete is the whole interview.

A proactive agent runs on a trigger, with no human watching at the moment of action. That inverts the risk model, and the interviewer is listening for whether you separate notifying from acting.

The agent loop assumes an observation is a short string, and images break that assumption in both directions. The signal is a context policy that keeps pixels out of the transcript and a verification step for the files the agent produces.

Everyone says 'cap the budget'. The signal is the enforcement mechanism: a budget object that travels with the run, checked before each call rather than after, with reserved headroom for a final answer and correct accounting across sub-agents.

The runaway loop gets all the attention. The failure that actually ships is the agent that declares victory on step 4 with half the work missing. The signal is a finish tool the harness can reject, and a completion check that reads the artifact instead of taking the model's word for it.

Agent PR volume scales with compute; human review capacity does not. Most candidates answer 'add an AI reviewer' and miss the real design problem: deciding which changes deserve human judgment at all. Interviewers want a risk-tiered pipeline you would trust in production.

Two frontier coding agents made opposite bets on the same problem, and both published evidence. Candidates who pick a side on vibes miss what the interviewer wants: the axes that actually decide it, and what each choice costs in staleness, latency, tokens, and infrastructure.