Anthropic AI Engineer interview questions

Nearly every candidate can recite softmax(QKᵀ/√d_k)V. What the interviewer is after is the variance derivation and the precise training failure the scale heads off. This is the response that tells rote recall apart from real grasp.

The most frequently asked applied GenAI question, and the one most candidates turn into a definition dump. What the interviewer wants is a decision framework with a default and the conditions that override it. Here is the call that earns points.

Alignment has become a baseline expectation even beyond 'safety' roles. The interviewer is looking for the three-stage RLHF pipeline and a sharp account of why DPO threw out the reward model completely. Here is the answer that leads with mechanism, not the buzzwords.

Every customer wants to know how to stop the model inventing things. The weak reply is 'better prompts.' The signal is understanding why models hallucinate and stacking grounding, abstention, and measurement into a system you can defend.

Every LLM product tunes these knobs, and the interviewer wants to hear what each one does to the probability distribution and which setting suits which task. That includes why factual and creative tasks call for opposite settings.

Evaluation is the toughest and most underrated part of shipping LLMs. The signal is understanding why public benchmarks mislead, why LLM-as-judge carries bias, and how to build a task-specific eval you can genuinely trust.

In-context learning is the property that turned prompting into a paradigm; chain-of-thought is its most useful trick. What matters is knowing what 'learning' means here (no weight updates) and why making the model reason step by step actually helps. Here is the answer.

Two prompting strategies that purchase accuracy with extra compute. What matters is knowing how each explores the reasoning space, that they trade tokens and latency for reliability, and how to judge when the problem actually warrants it. Here is that answer.

ReAct is the pattern behind most tool-using agents. What matters is why interleaving reasoning with actions (and observing results) beats reasoning alone or acting alone. Here is the answer that ties the prompt pattern to real agent behavior.

RL alignment shifted from PPO to leaner methods, and DeepSeek-R1 put GRPO on the map. What matters is knowing what the value/critic model does in PPO and how GRPO replaces it. Here is the mechanism, not the acronyms.

What separates a demo prompt from a production prompt is structure, versioning, and defense. What matters is treating the system prompt as a contract and templates as parameterized, testable artifacts. Here is the answer.

Jailbreaking is the adversarial sport of steering a model around its own safety training. What matters is naming the technique families and treating defense as a layered, never-finished arms race, not a one-time patch.

Instruction tuning converts a bare next-token predictor into a model that obeys instructions. What you are being tested on is slotting it correctly into the pretrain to SFT to alignment pipeline and stating precisely what it does and does not repair.

Plenty of people cite benchmark numbers; almost nobody reads them carefully. What shows depth is explaining what each one measures and why leaderboard figures outrun real-world skill. The question the interviewer is saving is how you would spot contamination.

Fine-tuning lives or dies on data, not on hyperparameters. What shows depth is spelling out what makes a set good (quality, diversity, format, dedup) and defending why a few thousand clean examples outperform a million noisy ones.

Synthetic data is how teams manufacture training examples when real data runs short, and it comes with sharp failure modes. What shows depth is pinning down when it helps, how to keep quality up, and why training on model output over successive generations narrows the distribution.

'Eval is the new system design' for LLM apps, and starting with zero data is the real test. What shows depth is bootstrapping a golden set, choosing task-specific metrics, and running evals continuously as a regression gate.

Attention gets all the attention, but the feed-forward layers hold most of a transformer's weights and handle much of the per-token 'knowledge' work. What shows depth is knowing what the FFN computes and why it dominates the parameter count.

Context windows are not arbitrary figures; they follow from attention's quadratic cost, KV-cache memory, and how the model was trained. What shows depth is naming all three causes and matching each to the technique that extends it.

RLAIF and Constitutional AI trade human preference labels for AI-generated ones to scale alignment. What shows depth is naming exactly what they substitute for human feedback, and the consistency-versus-bias tradeoff that swap introduces.

Reasoning models are the 2025-2026 frontier. The signal is understanding they are trained (largely via RL on verifiable rewards) to generate long internal reasoning, and that they scale along a second axis: spending more compute at inference.

Tool calling powers every agent. The signal is grasping that the model executes nothing: it emits a structured request your code runs, then you hand the result back. Here is the loop and the details that determine reliability.

Modern LLMs abandoned learned absolute positions for RoPE and ALiBi to cope with long context. The signal is recognizing that both encode RELATIVE position, and precisely how that permits length extrapolation past the trained window.

DPO simplified RLHF, and a family of variants now trade off what data and reference model they require. The signal is knowing what each variant drops or alters (reference model, paired data, separate SFT stage). Here is the answer.

Scaling laws explain why bigger models trained on more data predictably improve, and Chinchilla changed how we allocate compute. The signal is the model-size-vs-data tradeoff and the fact that many models were under-trained. Here is the answer.

Prompting for JSON works until it doesn't, and at scale the tail breaks your parser. Constrained decoding makes malformed output literally impossible to sample. The signal is knowing exactly where in the loop the constraint takes hold. Here is the answer.

Chain-of-thought looks like the model showing its work, but it may not mirror the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is a separate one. The signal is the detection toolkit, why each signal is imperfect, and how you merge them into an action.

Both are conditional compute, but one scales parameters and the other scales depth. The signal is knowing which axis each routes along and why that shifts the FLOP story. Here is the sharp version.

A model that claims a million-token window may not actually exploit it. The signal is knowing needle-in-a-haystack, where it fails, and how to probe genuine multi-fact reasoning across the entire context. Here is the answer.

The senior signal is fitting the method to what is genuinely missing rather than reaching for your favorite. Knowledge, behavior, or actions each steer you to a different tool. Here is the decision framework that scores highest.

Models are trained to be helpful, which quietly teaches them to never refuse. Restoring honest abstention is a system problem, not a prompt tweak. Here is the stack that actually shifts the refuse-when-unsure rate.

Alignment that piles on refusals and politeness often quietly erodes reasoning and coding. The alignment tax is real and measurable. Here is how to hold onto the safety gains without paying for them in capability.

Push any proxy hard enough and the model discovers the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

The model's training cutoff is frozen, but your dependencies shift weekly. Retraining is the wrong reflex. Here is how teams keep code suggestions current without touching the weights.

Generation has to end somewhere, and the wrong stop rule either truncates answers or wastes tokens on trailing garbage. Here is how the EOS token, stop strings, and max-token caps actually interact.

CoT is not a blanket upgrade. On some tasks it does nothing or backfires, and the reason points to what to try instead. Here is the escalation ladder past 'let's think step by step'.

A model that scores 92% on your eval can still ship a product users hate, because the model is one component in a system. Teams that run only model evals get blindsided. Here is the distinction that matters.

A few thousand carefully chosen examples can beat a million scraped ones. The signal is knowing which filters matter, how you gauge example quality, and why diversity beats raw volume.

Push PPO hard enough and true quality peaks then declines while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

DPO is simple and stable, but it can drive down the probability of chosen responses, overfit the preference margin, and magnify length bias. The signal is naming these and the variants that fix each.

A reward model can only match the quality of its labels, and human preference labels arrive noisy and inconsistent. The signal is measuring inter-annotator agreement and the concrete steps that raise label quality.

A reward model converts pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

By swapping costly human labels for an LLM's preferences, RLAIF scales cheaply but carries over the labeler model's biases. What matters is how AI feedback gets gathered and where it silently breaks down.

Speculative decoding accelerates generation, and with the right verification rule it leaves the output distribution untouched. What matters is understanding how draft-model, self-drafting (Medusa/EAGLE), and lookahead approaches differ in where the guesses originate, and which of them are still exact.

Candidates who cite one leaderboard number fail the follow-up. The interviewer wants you to name what SWE-bench measures, what it cannot (long-horizon multi-file refactors, real tool use, business logic), and which complementary benchmarks close those gaps.

Temperature 0 is not the same as deterministic, and the reason lives in the GPU kernels, not the sampler. What gets scored is naming the batch-invariance problem and knowing which fixes are real versus placebo.

Your assistant scores 4.6 out of 5 on every individual turn and still loses the user by turn eight. Per-turn averages hide the failures that actually cancel subscriptions, and here is the session-level eval that catches them.

Most teams reach for a token-dropping compressor and pay an accuracy tax for tokens they could have had for almost nothing. The ranking of levers matters more than any single technique, and the top of the list is not compression at all.

Appending every turn is the default in tutorial code and it fails twice: on the context window and on cost. The signal is the layered assembly, the constraints you refuse to let the summarizer touch, and what you do the moment the user changes subject.

The instinct is to blame the vision encoder. It is a token-budget problem, and the fix is to retrieve pages before you look at them. What separates a strong answer is knowing which question types page retrieval quietly cannot answer.

An emerging alternative to stuffing the window or building an index: keep the context outside the prompt and let the model write code to inspect it, recursing into the pieces. Knowing when it does not pay is most of the score.

Every LLM is one function: tokens in, a distribution over the next token out. The signal is what you say next, because the distance from that function to a useful assistant is three training stages, and almost every production complaint lands in exactly one of them.

The shift from one model per task to one model adapted many ways rewrote the job description. The interviewer is checking whether you know which half of the work got nearly free, which half did not move at all, and which problems classical ML still owns outright.

The whiteboard question that spans the entire stack, from BPE to the sampler. What separates a strong answer is knowing that prefill and decode stress opposite hardware resources, and that nearly every serving decision you will ever make falls out of that one fact.

One triangle of negative infinity, added before the softmax, is what makes transformer training parallel and generation stubbornly serial. Interviewers want the mechanism and the asymmetry it creates, not a definition of attention read back to them.

Two defaults that every Llama-class model shares and almost no candidate can justify. One is a compute win that turned out to cost nothing in quality; the other is what makes a 60-layer stack converge at all.

Most candidates answer with the 2015 ResNet answer and stop. The senior answer treats the residual stream as a shared communication bus, and that reframing is what makes logit attribution, the logit lens, and activation steering possible at all.

Definitions take thirty seconds. The interview is about the four things log-probs buy you in production: a confidence signal, banning tokens, guaranteeing valid output, and classification you can calibrate. Plus the tokenization gotcha that quietly breaks all of them.

The follow-up every interviewer holds in reserve after you say you fine-tuned something. A task-metric win proves almost nothing on its own. Here is the five-part protocol that separates a shipped model from a demo.

The staged recipe is the easy half. The half that sinks real projects is data licensing, an evaluation bar that needs actual domain experts, and knowing which parts of the problem fine-tuning cannot solve at all.

Summarization is the one task where the model was handed everything it needed and still made something up. The signal is separating intrinsic from extrinsic hallucination, knowing why ROUGE cannot see either, and gating on entailment before you ship.

Truncate, map-reduce, refine, retrieve, or stuff a long-context model. Each buys you something different and breaks differently. The candidates who score are the ones who pick based on the question being asked, not the document being read.

Letting a model write your prompts works, right up until the optimizer memorizes your eval set and hands you an unreadable prompt that scores better and generalizes worse. The bounds are the whole answer.

The most repeated trick in prompt engineering, and the evidence for it is much thinner than the folklore. A screening question for whether you measure your prompts or inherit them from a thread of screenshots.

Every request pays for the prompt and almost nobody has looked at where the tokens actually go. It is rarely the part you can read: thirty tool schemas and seven stale examples usually outweigh the system prompt, and the output side costs several times more per token.

Naming RLHF is not an answer. The frame that scores is outer versus inner alignment, why Goodhart breaks any proxy objective you optimize hard enough, and why a classifier sitting at the boundary can constrain a model's behavior without changing what it is trying to do.

Most candidates answer 'use the reasoning model for hard problems' and stop. The interviewer wants it framed as an eval and a budget problem: how you prove the extra thinking tokens paid off, and what you do when the model talks itself out of a correct answer.

One of the most common AI, ML, and GenAI design rounds. Sketching embed-retrieve-generate is easy; the signal lives in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents get reached for too often. A strong answer pushes past the hype: most tasks are better served by a single structured call, and agents pay off only under particular conditions. Here is when one is worth it and how to stop it from compounding errors.

A 2025-2026 question that trips up people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for deciding.

Agents and integrations depend on the model returning valid, schema-conforming output. The signal is stacking constrained decoding, schema validation, and retries, rather than trusting a prompt to do the job. Here is the production-reliability answer.

Multi-agent is the pattern most often reached for without cause in AI today. The signal is holding back unless the task genuinely calls for specialization or parallelism, knowing the supervisor and handoff patterns, and grasping why coordination magnifies failure.

Conversations and agent tasks grow past the context window, and the naive 'stuff the whole history' approach breaks on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieving every time is wasteful and occasionally harmful; retrieving never leads to hallucination. Self-RAG turns retrieval into a decision the model controls, then critiques what came back. The signal is the retrieve-on-demand plus self-critique loop.

MCP standardized how agents connect to tools and data, but tool design is what determines reliability. The signal is knowing what MCP actually standardizes and the principles behind a tool the model can call correctly from its description alone.

Reflection is the pattern in which an agent critiques and revises its own work. The signal is knowing when it truly helps (external feedback grounds the critique) versus when self-critique on its own is theater.

Plan-and-Execute and ReAct are the two leading agent control patterns. The signal is knowing the upfront-plan versus step-by-step tradeoff, why one costs less, and why most production agents wind up blending them.

Agents need memory beyond the context window, and 'just store the history' is the wrong answer. The signal is telling the memory types apart and mapping each to a storage and retrieval strategy. Here is the answer.

Context engineering is the discipline that took over from 'prompt engineering' for serious agent work. The signal is grasping that what you place in the context window (and what you omit) shapes behavior more than clever wording. Here is the answer.

Code-execution agents are powerful and risky: arbitrary model-generated code runs on your infrastructure. The signal is genuine isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Tool selection and parameter extraction are where agents quietly break. The strongest answer treats the tool spec as the only thing the model sees, then repairs the design, not the model. Here is how.

Agents fail in the middle, not only the end, so final-answer-only scoring conceals the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and pinpointing where it broke. Here is the answer.

Sub-agents are how you scale a complex agent task without one huge, noisy context. The signal is context isolation and clean delegation: a focused subtask in, a compact result out. Here is the answer.

Multi-agent systems succeed or fail on how agents share information and hand off work. The signal is naming the three communication patterns and the coordination failures that sink the naive version. Here is the answer.

One model can behave completely differently depending on the harness wrapped around it. The signal is recognizing that the scaffolding frequently sets agent quality more than the underlying model does. Here is the answer.

A computer-use agent operates a real browser or desktop via screenshots and clicks. The signal is the perceive-decide-act loop together with the containment for an agent that can click, buy, or delete anything. Here is the answer.

An agent that performs actions is far riskier than one that only talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

A chunk embedded in isolation forgets which document, section, and entity it came from, so it retrieves poorly. Contextual retrieval writes that context back in before indexing. The signal is fixing retrieval at index time rather than query time.

Standard RAG retrieves once and generates; agentic RAG makes retrieval an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

The industry uses 'agent' for anything that calls an LLM. The distinction that matters is how much control you hand to the model, and more autonomy is not better. Here is the spectrum and the decision rule.

A live-coding round that tells apart engineers who have shipped agents from those who have only read about them. Interviewers want a genuine plan-act-observe loop, tools that query a real SQL database, guardrails that block the obvious failures, and a concrete evaluation plan, rather than one prompt masquerading as an agent.

A bridge connecting classic DSA and modern RAG. Interviewers want to see you build a working inverted index and a correct BM25 score by hand, reason about its complexity, and then know precisely when you would switch to embeddings and an ANN index instead.

Splitting source files every 500 characters slices functions in half and wrecks retrieval. Interviewers want to see you chunk on syntax, retrieve by symbol and dependency, and blend lexical exact-match with semantic search the way code search actually needs.

An LLM can cite a source that does not say what the answer claims. The signal is verifying claim-by-claim entailment against the cited text, not merely that a citation marker exists, and knowing what to do when grounding fails.

An answer can be correct yet still unfaithful, asserting things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Linear agents lock into one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from decaying.

Tools fail: timeouts, rate limits, bad arguments, garbage output. A naive agent retries blindly or quits. The signal is classifying errors and matching each to the right recovery, with hard caps.

Multi-agent systems coordinate through handoffs and shared state. The signal is knowing the orchestration topologies, what a clean handoff protocol carries, and why naive multi-agent often loses to a single good agent.

Most people know MCP exposes tools. The deeper signal is the complete primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy conceals how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is grasping compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Agents can critique and retry their own work. The signal is knowing how Reflexion's verbal feedback loop operates, when self-critique genuinely helps versus when it is theater, and what makes it real.

Autonomous agents still need humans at the right moments. The signal is designing checkpoints: where to interrupt, what to surface, how to persist and resume state, without reducing the agent to a click-through rubber stamp.

Any content an agent reads can smuggle in instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Agents re-send growing context every step, so cost scales worse than linearly. The signal is knowing where the tokens go and the levers (caching, model routing, context trimming, step caps) that shrink the bill.

Screen-driving and browser agents break in ways chat agents never hit: stale DOM, pages that keep shifting, misplaced clicks. What interviewers watch for is the grounding and reliability methods that move a flaky demo toward something dependable.

The CRM says the balance is $0 and the ledger says $412. An agent left to its own devices will average them into something confident and wrong. Designing the precedence before the model has to guess is the whole answer.

Six stages, six isolating experiments, six metrics. The candidates who score do not guess at knobs, they bisect the pipeline, and they instrument the rungs nobody else does: ranking, context assembly, generation.

Retrieval is exonerated, so the bug lives in generation. There are four usual causes and they have different fixes, and the one nobody names is that the model was never given permission to say it does not know.

An hour-long agent run is a distributed workflow wearing an LLM hat. The signal is designing serializable state with step-level checkpoints, and knowing why resume is easy while replaying a side effect is not.

Tool calling costs one LLM round trip per action. Code costs one round trip for a loop over 500 records. The signal is knowing exactly what you trade away when you let the model write the program instead of the call.

Agent latency is a step-count problem disguised as a speed problem. The winning move is removing round trips, not shaving milliseconds off each one, and knowing which step you can delete is the whole interview.

A proactive agent runs on a trigger, with no human watching at the moment of action. That inverts the risk model, and the interviewer is listening for whether you separate notifying from acting.

The agent loop assumes an observation is a short string, and images break that assumption in both directions. The signal is a context policy that keeps pixels out of the transcript and a verification step for the files the agent produces.

Everyone says 'cap the budget'. The signal is the enforcement mechanism: a budget object that travels with the run, checked before each call rather than after, with reserved headroom for a final answer and correct accounting across sub-agents.

The runaway loop gets all the attention. The failure that actually ships is the agent that declares victory on step 4 with half the work missing. The signal is a finish tool the harness can reject, and a completion check that reads the artifact instead of taking the model's word for it.

Two frontier coding agents made opposite bets on the same problem, and both published evidence. Candidates who pick a side on vibes miss what the interviewer wants: the axes that actually decide it, and what each choice costs in staleness, latency, tokens, and infrastructure.

Labs reach for this practical screen often, since it probes concurrency, time handling, and judgment inside 20 lines. The pitfall is a background thread that burns CPU. Below is the lazy-refill approach interviewers expect, along with the follow-ups.

The from-scratch implementation that frontier labs and NVIDIA really do ask. Writing it shows you grasp shapes, the scale factor, the causal mask, and how heads split and recombine. Here is a correct, readable implementation plus the follow-ups.

Coding the KV cache shows you understand why decode is efficient: you append the new token's K/V and reuse the rest rather than recomputing. Interviewers look for the append-and-attend-over-cache logic. The implementation follows.

Single-head attention shows up often; multi-head layers on the split-into-heads, attend-per-head, concatenate pattern. What gets scored is the reshape into heads plus a sharp explanation of why several heads help. Here is the implementation.

The classic multi-round build screen: a plain key-value store that gains new requirements at each stage (TTL, transactions, scans). The signal is not stage 1, it is whether your code takes on stage 4 without a rewrite. Here is how to design for it.

OpenAI's signature build screen: a credit ledger that issues GPU-hours, spends them, and lets unused grants lapse in issue order. The trap is charging against the right grant first. Here is the FIFO-by-expiry design that holds up through every follow-up.

A classic build screen: a key-value store whose writes inside a transaction can be committed or discarded, with transactions that nest. The trap is mutating the base store directly. Here is the overlay-stack design that keeps rollback O(1).

A practical build screen: transform a stream of sampled call stacks into a flame tree, then report self-time against total-time per function. The trap is mixing up the two times. Here is how to aggregate the tree and rank the hot functions.

Most candidates start rewriting. The interviewer is grading whether you isolate pure logic from side effects before you touch a line, because that is the difference between a safe refactor and a silent behavior change.

Plain multi-head attention is rarely the whole exercise: the usual extension is to chain it into KV-cached decoding and GQA in one sitting. The locked answer walks the NumPy implementation, the equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.

Sierra's signature build screen: cells hold values or formulas that reference other cells, and editing one has to recompute its dependents without infinite loops. The signal is the dependency graph plus cycle detection. Here is the topological-eval design.

Google's 'vibe coding' screen: the spec is intentionally vague and shifts mid-interview. The signal isn't the algorithm, it's whether you clarify before coding, structure for change, and keep tests green as the requirements move. Here is how to run that loop.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams stays truncated mid-token for the whole generation. What matters is a clean strict parser plus a small repair layer, not a second parser. The code follows.

The most job-shaped coding screen in AI, ML, and GenAI engineering: fan out N LLM calls without melting the rate limit or losing the batch to one bad request. What matters is the retry policy, not the async syntax. Below is the version that passes.

The most-asked hands-on LLM exercise: a chat history that outgrows the context window. What gets scored is not the sliding window, it is the region eviction is never allowed to touch. Below is the version that passes.

Every RAG system packs a prompt, and the packing bug is always the same one: the input fits the window exactly, so the model has nowhere left to answer. The arithmetic here is what separates a candidate who has shipped from one who has read about it.

The lookup is three lines of linear algebra. What separates a cache from an incident is where the similarity threshold came from, and the one-token queries ('2023' vs '2024', 'not') that no threshold can catch because the embedding barely moves.

Asking a model which answer is better is one line of code. Getting a number you would let block a deploy takes position-bias control, a confidence interval, and a human-labeled set the judge is measured against. Here is the harness.

Every agent is a loop around this function. The candidates who fail it raise an exception on a bad tool call; the ones who pass hand the error back to the model as a tool result and let it fix itself on the next turn.

Writing the regexes takes ten minutes. The half of the question that separates candidates is the evaluation: a detector that blocks 'ignore the noise in the data' has shipped a bug to every analyst using your product. Here is the layered detector and the harness that proves it works.

This code runs on every single response, so it sits on the critical path and a heavyweight check here doubles your p95. The signal is in the ordering, the fail-open decision when the guardrail times out, and the fact that you cannot un-send a token you already streamed.

The naive version splits on newlines and works right up until a TCP chunk lands mid-line. Then there is the error that arrives after a 200 OK, the user who closes the tab while you keep paying for tokens, and JSON you cannot parse until it closes.

Asking a model 'is this answer hallucinated?' barely beats a coin flip. This is the build: claim extraction, a judge forbidden from using world knowledge, a denominator that excludes hedges, and one model call per claim, which decides where the check can run at all.

Everyone can write the happy path. The screen is decided by the two branches most candidates skip: what the system does when retrieval finds nothing good, and what it does when the model cites a chunk you never sent.

A prompt is code, but most teams edit it in a dashboard with no history and then cannot explain last Tuesday's quality drop. This is the small piece of infrastructure that makes prompts reviewable, pinnable, and instantly reversible.

Most RAG projects do not die at the retriever, they die at ingestion. Naive PDF extraction interleaves columns into nonsense, shreds tables, and stamps the footer into all 4,000 chunks. Here is the parser that survives real documents.

Bayesian inference requires an intractable normalizing constant, and MCMC works around it. The signal is explaining the acceptance ratio, detailed balance, and why you can drop the constant entirely. Here is the answer.

Most teams run human eval as a vibe check with ten examples and a 1-5 slider. The signal is treating it as a designed experiment: pairwise, blinded, powered, and agreement-measured. Here is the protocol that survives scrutiny.

Accuracy against a held-out label is the wrong instrument for a system with no single right answer, an open failure space, and a metric that is itself a model. Here is what breaks, what survives, and what replaces it.

The applied data-wrangling screen: here is a grubby database, turn out a clean analysis-ready table. The signal is profiling before transforming, doing set-based cleaning in SQL and row-level fixes in Python, joining without fanning out rows, and validating the output rather than trusting it.

Anyone can say 'filter and dedup.' The signal is the funnel arranged by cost, the MinHash/LSH banding math, and recognizing that the shuffle across billions of documents is what actually runs up the bill, plus the benchmark decontamination people forget until their eval numbers get challenged.

The canonical AI system-design question. The signal is covering both the model lifecycle and the serving stack at real scale, without rambling. Most candidates design only half and lose the points.

The modern agentic system-design question. The signal is the plan, search, read, synthesize, verify loop with per-claim citations, plus keeping cost and latency bounded. The hard parts are not the writing.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem rather than a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

AI code review succeeds or fails on precision: a handful of noisy comments and the team silences the bot for good. The signal is grounding in the diff plus repo context, unforgiving false-positive control, and winning developer trust one accepted comment at a time.

Summarize, draft, and triage, all across the most sensitive PII a person holds. The signal is grounding in the real thread, matching the user's voice, and a firm human-in-the-loop rule on anything that gets sent.

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation carries real consequences, and lawyers have been sanctioned over fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Code migration is a high-value agentic task with a strict correctness bar: the migrated code has to still work. The signal is grounding in the real repo, transforming in verifiable chunks, and gating on tests, never one huge LLM rewrite.

A Devin-style agent that carries an issue all the way to a merged PR is the hardest kind to make reliable, since every step can fail and errors pile up. Here is the architecture and the loop that keeps it honest.

Exact-match caching does little when no two prompts match. Semantic caching reuses answers across similar queries, and its entire danger is handing back a near-match that is subtly wrong. Here is how to build it safely.

When every team evaluates prompts in one-off notebooks, an org ships regressions and argues about vibes. A shared eval platform turns quality into something measurable and comparable. Here is what it must provide.

An agent that clicks through a real website is slow, brittle, and a single wrong click away from a costly mistake. The architecture comes down to perception, action, and guardrails around irreversible steps. Here is how to build it.

RLHF and evals can only match the preference data behind them, and that data comes from humans whose quality swings wildly. The platform that yields trustworthy labels is a serious system in its own right. Here is its design.

Anthropic's signature opener. The naive answer (one server pushes to N clients) is capped by the source uplink and scales linearly. The answer interviewers reward turns receivers into senders so the swarm's capacity grows with its size. Here is that reasoning.

The Dynamo question. Interviewers want three decisions made cleanly: how you partition keys so adding nodes does not reshuffle everything, how you replicate for durability, and where you sit on the CAP spectrum. Here is how to reason through all three with concrete quorum math.

Every API platform needs one. Interviewers reward the candidate who picks the right algorithm (token bucket vs sliding window) for the burst behavior, then cracks the genuinely hard part: holding one shared limit across many servers without a per-request round trip to a central store.

The distributed ordering problem that sits at the heart of Kafka, replicated logs, and consensus systems. Interviewers look for you to face the uncomfortable fact that physical clocks lie, then pick the right instruments: a single sequencer, consensus, or logical clocks. This walks that progression.

Inline completion lives or dies on tail latency: a suggestion that lands after the developer keeps typing is useless. Learn how to assemble the context window, hide model latency behind speculation and caching, and keep a tight feedback loop on acceptance rate.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context wins out, and how to evaluate summaries at scale.

Moderating billions of items a day with an LLM on each one is too slow and too expensive. Learn the tiered funnel (cheap classifier then LLM then human), how to tune thresholds for precision versus recall, and how to stay ahead of adversaries.

Agents that loop for minutes, invoke tools, and spawn sub-tasks demand durable execution rather than one request handler. See how to checkpoint state, recover after failures, cap runaway loops, and trace every move a non-deterministic agent makes.

SSE versus WebSocket is the easy half. The half that separates candidates: a buffer-everything output filter throws away the time-to-first-token you paid a GPU to deliver, and you cannot send an HTTP error after the 200 has already flushed.

Every AI system spends quality to buy speed, whether or not the team admits it. What scores is knowing the exchange rate for each knob, taking the free wins first, and having already decided what you will sell when the system saturates.

The root cause is not a weak system prompt. A model trained to be agreeable is dangerous to a user in crisis, because the objective and the safety requirement point in different directions. The fix has to live outside the model. Here is what that architecture looks like.

Every AI product that runs generated code (notebooks, agents, code interpreters) ends up building this, and the interview turns on two numbers: how strong your isolation is and how fast a session starts. The answer is microVMs plus snapshot restore, and knowing why the VM boot is not the slow part.

You cannot improve an LLM app you cannot see into, and LLM observability is not service monitoring. What counts is tracing multi-step chains, capturing inputs/outputs/tokens/cost, and online quality signals, not just latency and errors.

Caching is one of the biggest LLM cost levers, but 'cache the response' is naive for a non-deterministic system. What counts is telling prompt (prefix) caching apart from semantic caching and knowing when each is safe.

Editing a prompt is a code change that ships without a compiler or a unit test by default, so quality regressions slip out invisibly. Treating prompts and models as testable artifacts is what divides a toy from a product. Here is the harness.

A green build says nothing about model quality. The signal is an eval harness that runs deterministically in CI, checks against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

'Just set a seed' misses the point. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism together. Here is the full checklist and where it springs leaks.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline pairs a versioned dataset, layered scorers, and a CI gate. Here is the architecture.

A provider swap does not fail with a 500. It fails by staying up and quietly getting worse. The signal is knowing what actually changes under you on a model swap, and the shadow-plus-staged-ramp protocol that catches it before your users do.

AI ships behavior, not just code, and behavior is what you need to be able to switch off. The cheapest safety control you will ever build is a toggle that takes effect in seconds, and most teams discover theirs is broken during the incident.

Your eval set is clean, well-punctuated, and phrased the way you think about the problem. Your users are not. The move that turns this from a vibe check into an engineering artifact is measuring invariance rather than accuracy, and gating on the number it produces.

Migrating models is a project you plan. Being migrated is a Tuesday. When the weights move under a stable API and your prompts were tuned against the old ones, every dashboard stays green while quality quietly rots.

Interviewers here want concrete figures, not 'grab a bigger GPU.' Weight memory stays constant, the KV cache scales with traffic, and the order you pull levers in settles the outcome. This walks through the napkin math and the serving stack.

The interviewer is checking that you understand how a model too large for any one GPU still gets trained, and can run the optimizer-state memory math that justifies sharding. This covers the parallelism taxonomy and the 16-bytes-per-parameter figure.

A staple at hardware-aware teams. What they look for is grasping that attention's cost is memory traffic rather than FLOPs, and that FlashAttention is an exact, IO-aware reordering rather than an approximation. This is the answer that shows you reason about the memory hierarchy.

The KV cache is what makes LLM serving hard, and the interviewer wants the mechanics: what it holds, why it caps concurrency, and the tricks that shrink it. This is the answer that shows you grasp decode-time economics.

Decode runs sequentially and is memory-bound, so latency tricks count. What they grade is whether you can explain speculative decoding's draft-and-verify mechanism (and why it stays exact) along with the other levers and when each fits. This is the latency toolkit.

Continuous batching is the biggest single throughput lever in modern LLM serving. The signal is explaining why static batching leaves the GPU idle on variable-length generation, and why PagedAttention is what makes the fix practical.

Users experience the p99, not the median, and the tail is where serving systems quietly break. The causes are queuing and batching effects, not a slow model. Here is how to flatten it.

A shared system prompt is re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks align exactly. Here is the hashing and eviction machinery underneath.

Round-robin routing throws away a feature that doubles throughput: cache locality. Designing the router means picking a replica by KV state, not just least-loaded. Here is the two-layer design.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to strip it out.

Autoscaling reacts in minutes, but a burst lands in seconds, and the gap is where your tail latency dies. Absorbing bursts is about buffers and shedding, not just adding replicas. Here is the playbook.

RLHF with PPO is not one model training, it is four models in the loop at once, three of them on the GPU during every step. The memory math and the generation bottleneck are what catch people out.

Data, tensor, and pipeline parallelism each leave a single sequence's activations on one device, so training past 200k tokens hits a wall none of them can clear. The fourth axis shards the sequence itself, and the interview turns on the communication math.

When each request thinks for 30,000 tokens, serving swings from compute-bound prefill to memory-bound decode, and the KV cache turns into the resource you genuinely schedule. The levers that governed chat traffic stop being the ones that count.

Raw tokens per second is gameable: crank the batch size and the dashboard looks great while every request misses its latency target. Goodput is the throughput that actually meets your SLOs, and it is the number you size and autoscale on.

An image is not one input, it is a large prompt. Once you internalize that image tokens dominate prefill, the cost levers stop being model choices and start being resolution, tiling, and cache placement.

There is no one-shot patch for prompt injection, and answering 'sanitize the input' loses the round. What lands is defense in depth: privilege boundaries, handling retrieved content as data rather than instructions, and a human gate on irreversible actions. Here is the layered answer.

A CISO-facing question that trips up engineers who focus only on model quality. What matters is treating governance as architecture rather than a checkbox: minimization, isolation, auditability, and never training on customer data by default. Here is the framework.

Getting an LLM feature to production safely is an evaluation problem first and a model problem second. What lands is a layered eval-plus-guardrail design with honest, segmented metrics rather than a single 'safety classifier.' Here is how to measure and defend each failure mode.

Fairness questions catch engineers who treat it as a vibe. What lands is knowing the formal fairness metrics conflict mathematically, that you have to choose one deliberately for the context, and where bias enters the pipeline. Here is the rigorous, honest answer.

A classic ML-security question that trips up people who only know clean-data accuracy. What lands is grasping why models are brittle, the realistic threat model, and the fact that no single defense holds. Here is the honest answer.

LLM endpoints are unusually exposed: every call can spend real money and consume scarce GPU capacity, so abuse is a security and a financial problem at once. What lands is layered controls on tokens and cost, not just request count. Here is the defense.

Most ML security centers on inference-time attacks; this one targets the training pipeline, where a poisoned dataset or a tampered dependency can hide a backdoor that clean-data evaluation never catches. What lands is naming the attack classes and recognizing that defense means provenance, not a single model fix.

Privacy starts as a governance requirement and turns into a training-time technique. What lands is knowing what differential privacy actually promises (and what it costs), plus how federated learning and DP fit together. Here is the rigorous, honest answer.

Two attacks aimed at a deployed model's confidentiality: copying its functionality through the API, and deducing who was in its training data. What lands is naming the precise mechanism each exploits and recognizing that every defense trades against utility.

Enterprise AI deals turn on governance, and engineers who can speak it stand out. What lands is treating it as documentation, accountability, and risk-tiered controls across the lifecycle rather than a legal afterthought.

Red teaming is how you uncover an AI system's failures before users or attackers do. What lands is a structured program covering the real attack surface (jailbreaks, harmful content, leakage, bias) and routing every finding back into defenses.

AI fails in ways traditional software does not: bias, hallucination, harmful output, silent quality regression. The strong answer is a concrete playbook (detect, contain, communicate) plus a blameless post-mortem that ships systemic fixes. Here is the answer.

Because detection never catches everything, the strong answer works in layers: checksummed regex for structured PII, ML/NER for the unstructured PII regex misses, and an honest explanation of why redaction alone is never enough. Here is the answer.

Model cards and datasheets are the standard responsible-AI documentation artifacts, and regulators and buyers increasingly demand them. The strong answer knows what each one covers and grasps that the whole purpose is honest disclosure of limitations, not marketing. Here is the answer.

'Add guardrails' says nothing concrete. The strong answer names the specific input and output checks, the mechanisms that enforce them, the fail-safe behavior when one trips, and the honest admission that they are imperfect. Here is the implementation answer.

GDPR erasure and copyright takedowns require pulling a user's influence out of trained weights, not just deleting a dataset row. The signal is knowing why deletion falls short and where retrain, SISA, and approximate scrubbing each sit on the cost-versus-proof curve.

Attackers coax models into spilling the hidden system prompt, exposing IP, business logic, or worse, embedded secrets. The signal is treating the prompt as non-secret and defending in layers, rather than betting on 'never reveal your instructions.'

The EU AI Act sorts obligations by risk, making that classification a first-order design decision rather than a legal footnote. The signal is knowing the four tiers and what high-risk systems must prove before they ship.

Safety is not a single number. The candidates who pass name the axes, run benchmarks as a gate, and then explain why benchmarks by themselves certify nothing. Here is the framing interviewers score highest.

With direct injection the user attacks the prompt. With indirect injection the payload sits inside a page, doc, or email the model reads. The signal is recognizing that retrieved and tool data is untrusted and can hijack the model. Here is the answer.

Every third-party model, dataset, and library widens the attack surface: backdoors, poisoning, arbitrary code on load, license landmines. The signal is treating models and data as supply-chain artifacts that need provenance and vetting. Here is the answer.

Large models memorize and can be coaxed into emitting training examples verbatim, a genuine privacy and copyright liability. The defenses reach across data, training, and output. Here is the layered answer.

When an agent holds private data, reads untrusted text, and can talk to the outside, injected instructions can steal data. This 'lethal trifecta' is the defining agent vulnerability. The fix is architectural, not a better prompt.

Before naming defenses, a security engineer maps the attack surface systematically. LLM apps present a wider, stranger surface than classic software. Here is how to threat-model one end to end.

A request that gets refused in one message often lands when broken across ten. The signal is understanding that conversational state is part of the attack surface, and that turn-by-turn classifiers see only fragments.

base64, leetspeak, invisible Unicode, and foreign scripts each let a payload look like gibberish to your filter yet read as a clear instruction to the model. The signal is knowing why the model decodes what the classifier cannot.

Roleplay, obfuscation, optimization-based suffixes, and multi-turn escalation are distinct attack classes that call for distinct defenses. The signal is organizing the space and pairing each class with a control instead of hoping one filter covers all.

A backdoored model acts perfectly until it meets a secret trigger, then flips. The signal is explaining why clean test accuracy never exposes it, and what detection actually buys you when the trigger is unknown.

Gibberish-looking token strings tacked onto a prompt can reliably break refusals, and they carry over between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.

An agent that trusts its tools and its own memory can be redirected by one poisoned record that resurfaces turns or sessions later. The signal is treating persistent state as an attack surface, not just the live prompt.

A hijacked agent does not need a 'send email' tool to leak secrets. A rendered markdown image, a URL parameter, or a DNS lookup will do. The signal is enumerating the covert channels and locking down egress, not just tools.

Asking the model to keep secrets or enforce permissions puts the job on the wrong component. The signal is enforcing access control before retrieval, in code you trust, and treating the LLM as untrusted compute over data that is already authorized.

An agent running on a single over-privileged service account is a confused deputy waiting to happen. The signal is per-user delegated credentials, token exchange, per-tool scopes, and an audit trail that names the human. Here is the answer.

Human-in-the-loop is the mitigation everyone writes into the risk register and almost nobody measures. When the reviewer anchors on a confident model output, your oversight control fails exactly on the cases it existed to catch.

Single-axis fairness audits are just another form of averaging, and they average away the exact group that is being harmed. The hard part is not noticing that: it is handling the combinatorial blowup, the tiny cells, and the multiple-testing problem without chasing ghosts.

Most teams stop their governance work at 'we can explain the decision.' The obligation, and the interview signal, is what happens after the user reads the explanation and says it is wrong.

Rate limits stop a user who costs you too much. They do nothing about a user doing something you forbid. The enforcement system is a different design, and interviewers can tell within a minute whether you have built one.

Your guardrail reads text. The attack is not in the text. That single sentence is the whole vulnerability, and the red team you build from it looks nothing like the one you already have.

The technique works and everybody knows it works. The interview is not testing whether you can generate the training set; it is testing whether you know what you are agreeing to when you do.

The point is not whether you failed; everyone has. Panels are probing for ownership, debugging discipline, and candor under stress. Here is the frame that turns a failure story into a hire signal, plus the pitfalls that turn it into a flag.

This is an entire interview round (OpenAI's project deep dive, Amazon's Tech Talk), not small talk. It is where they check whether you genuinely understand the work you claim. The signal is depth, your specific contribution, and the tradeoffs you can defend. Here is how to prepare and deliver it.

Applied AI work is ambiguous by nature, and panels (Meta's 'embracing ambiguity,' Anthropic's deliberately under-specified problems) check whether you freeze or drive. The signal is structured progress under uncertainty. Here is the arc that connects.

AI moves fast enough that learning speed is itself a core competency, and several companies ask about it directly. The signal is a concrete method for ramping fast plus a real, recent example of putting something new to use. Here is how to show learning agility rather than merely claim it.

This question screens for ego and growth, not perfection. The interviewer wants to watch you take hard feedback without getting defensive, own a real mistake in first person, and change as a result. The signal is humility plus a durable, concrete change.

At the frontier labs this is a genuine gate, not a pleasantry. They turn away strong engineers who cannot articulate real motivation. The signal is specific, honest alignment between what you want to build and what this company actually does.

Mentoring questions check whether you scale past your own output, a senior and leadership signal. The interviewer wants a real example with a tailored approach and a measurable outcome, not 'I helped a junior.'

A soft-sounding question that quietly filters out low-effort candidates and tests real fit. The signal is specific, researched reasons tied to your trajectory, not flattery. Here is the structure that lands.

The field advances quickly, requirements move, and tooling turns over month to month. The question tests adaptability. What scores is welcoming the change, ramping fast, and steering course productively rather than pushing back. Particularly relevant for applied AI.

AI features are probabilistic, yet stakeholders hear 'it works.' Setting honest expectations without draining enthusiasm is a core applied-AI skill. Here is how to frame reliability so trust holds through the first mistake.

At an AI company, arguing against AI is a strong signal: it shows judgment ahead of hype. Interviewers use it to spot engineers who solve problems instead of reaching for a favorite hammer. Here is how to tell it.

Every AI team feels the tug between velocity and doing it responsibly. How you work through that tension, with judgment rather than dogma in either direction, is what this question screens. Here is the answer.

Chasing every new model is as harmful as ignoring them all. Interviewers want a repeatable filter for signal versus hype, plus the discipline to test on your own problem. Here is that filter.

Ethical pushback probes integrity and judgment under pressure. Interviewers look for principled, specific action, not a generic statement of values. Here is the arc that signals real backbone.

The 2025-2026 AI-fluency screen. Interviewers look for a concrete daily workflow, a real verification discipline, and a story where the AI was wrong. Vague enthusiasm fails; here is what passes.

The opener in half of all Applied AI screens, and most candidates answer it with a definition nobody scores. The distinction that actually earns points is about where you start and what your bottleneck turns out to be.

Both of the obvious moves cost you credibility: block the launch on principle, or ship quietly and hope. The answer that scores turns a scary percentage into a number of bad outputs per week and then changes what the argument is about.

The number is bait. Interviewers use this to see whether you treat a benchmark delta as a decision or as a claim to be audited, and whether you can price latency, cost, and on-call burden against accuracy. Here is the answer that scores.

The classic closer, and most candidates answer it with a horoscope or a list of tool names. The signal is whether you hold an actual view, can justify it from what is already happening, and know what it means for the job.

Most candidates answer this with a speech about tradeoffs. The ones who get hired describe a mechanism: the teams that ship fastest in AI are not the ones with fewer guardrails, they are the ones whose guardrails make a mistake cheap.

The ordering is the answer, and one move separates the people who have shipped from the people who have prototyped: where evaluation sits in the sequence. Most candidates put it in the wrong place and lose the question in the first sentence.