Hard AI, ML & GenAI interview questions

Nearly every candidate can recite softmax(QKᵀ/√d_k)V. What the interviewer is after is the variance derivation and the precise training failure the scale heads off. This is the response that tells rote recall apart from real grasp.

Alignment has become a baseline expectation even beyond 'safety' roles. The interviewer is looking for the three-stage RLHF pipeline and a sharp account of why DPO threw out the reward model completely. Here is the answer that leads with mechanism, not the buzzwords.

MoE explains why some frontier models carry huge parameter counts yet serve cheaply. The signal is the sparse-activation idea, the routing mechanism, and the operational cost (memory, load balancing) that the FLOP savings quietly conceal.

Attention is permutation-invariant, so with no position information a transformer cannot detect word order. The signal is knowing why, and why the field shifted from absolute sinusoidal encodings to relative ones like RoPE and ALiBi that extrapolate to longer contexts.

Evaluation is the toughest and most underrated part of shipping LLMs. The signal is understanding why public benchmarks mislead, why LLM-as-judge carries bias, and how to build a task-specific eval you can genuinely trust.

PEFT is how everyone fine-tunes large models now. What matters is the low-rank insight behind LoRA, why it cuts memory so sharply, and what 4-bit QLoRA adds. Here is the answer that goes past 'it's efficient fine-tuning.'

RL alignment shifted from PPO to leaner methods, and DeepSeek-R1 put GRPO on the map. What matters is knowing what the value/critic model does in PPO and how GRPO replaces it. Here is the mechanism, not the acronyms.

Fine-tune a model on your domain and it may lose the ability to do everything else. What earns the signal is explaining why shared weights cause it and listing the concrete mitigations that all boil down to one principle.

PEFT is a family, not only LoRA, and interviewers test whether you know how each one injects trainable parameters. The signal is where each method adds capacity and the latency and quality tradeoffs that result.

Synthetic data is how teams manufacture training examples when real data runs short, and it comes with sharp failure modes. What shows depth is pinning down when it helps, how to keep quality up, and why training on model output over successive generations narrows the distribution.

'Eval is the new system design' for LLM apps, and starting with zero data is the real test. What shows depth is bootstrapping a golden set, choosing task-specific metrics, and running evals continuously as a regression gate.

Full attention is O(n^2), so long context demands cheaper attention. What shows depth is naming the families (sparse, sliding-window, linear), what each surrenders, and why FlashAttention is not one of them.

RLAIF and Constitutional AI trade human preference labels for AI-generated ones to scale alignment. What shows depth is naming exactly what they substitute for human feedback, and the consistency-versus-bias tradeoff that swap introduces.

Reasoning models are the 2025-2026 frontier. The signal is understanding they are trained (largely via RL on verifiable rewards) to generate long internal reasoning, and that they scale along a second axis: spending more compute at inference.

Modern LLMs abandoned learned absolute positions for RoPE and ALiBi to cope with long context. The signal is recognizing that both encode RELATIVE position, and precisely how that permits length extrapolation past the trained window.

MQA and GQA shrink the KV cache, the thing that bottlenecks LLM serving. The signal is understanding they share key/value heads across query heads to trim memory and bandwidth, with GQA as the quality-preserving middle ground. Here is the answer.

DPO simplified RLHF, and a family of variants now trade off what data and reference model they require. The signal is knowing what each variant drops or alters (reference model, paired data, separate SFT stage). Here is the answer.

Fine-tuning a 65B model once demanded a node of A100s. QLoRA compresses it onto one card with three specific tricks. The signal is knowing what gets quantized, what stays trainable, and why quality barely shifts. Here is the answer.

Fuse several fine-tunes into one model by doing arithmetic on their weights, with no retraining and no data. The signal is knowing why it works and which method to pick when merges interfere.

Chain-of-thought looks like the model showing its work, but it may not mirror the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is a separate one. The signal is the detection toolkit, why each signal is imperfect, and how you merge them into an action.

Both are conditional compute, but one scales parameters and the other scales depth. The signal is knowing which axis each routes along and why that shifts the FLOP story. Here is the sharp version.

RAG and fine-tuning get cast as either/or. RAFT trains the model to be good at RAG itself. The tell is what goes into the training set: relevant docs mixed with distractors. Here is the answer.

Distillation gives you most of a frontier model's quality for a fraction of the serving cost. The signal is naming the three LLM-specific variants and knowing which one holds up against a closed API. Here is the answer.

Models are trained to be helpful, which quietly teaches them to never refuse. Restoring honest abstention is a system problem, not a prompt tweak. Here is the stack that actually shifts the refuse-when-unsure rate.

Alignment that piles on refusals and politeness often quietly erodes reasoning and coding. The alignment tax is real and measurable. Here is how to hold onto the safety gains without paying for them in capability.

Push any proxy hard enough and the model discovers the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

Few-shot accuracy can swing on example order by itself. If your prompt is fragile to things that shouldn't matter, the fix is structural, not lucky example-hunting. Here is what genuinely reduces variance.

The per-token sticker price masks the real decision. Self-hosting only pays off past a volume break-even most teams misjudge. Here is the back-of-envelope a staff engineer does on the whiteboard.

CoT is not a blanket upgrade. On some tasks it does nothing or backfires, and the reason points to what to try instead. Here is the escalation ladder past 'let's think step by step'.

When a drug name or a Hindi word splits into nine subword tokens, you pay in cost, context, and accuracy. Adding tokens is easy and adding them well is not. Here is the tradeoff.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field relies on a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

A few thousand carefully chosen examples can beat a million scraped ones. The signal is knowing which filters matter, how you gauge example quality, and why diversity beats raw volume.

Push PPO hard enough and true quality peaks then declines while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

DPO is simple and stable, but it can drive down the probability of chosen responses, overfit the preference margin, and magnify length bias. The signal is naming these and the variants that fix each.

Swap a learned reward model for a checker that returns right-or-wrong, and reward hacking largely vanishes. The signal is why verifiable rewards beat learned ones for math and code, and where they break.

Distillation is not a single method. The signal is knowing when to match logits versus train on generated text, why on-policy distillation beats off-policy, and how reasoning models are distilled.

A reward model can only match the quality of its labels, and human preference labels arrive noisy and inconsistent. The signal is measuring inter-annotator agreement and the concrete steps that raise label quality.

The proportions of web, code, books, and math in pretraining quietly set downstream skills. The signal is knowing how mixtures get chosen, why upsampling helps, and how methods like DoReMi automate it.

Continued pretraining adds domain knowledge that fine-tuning cannot, yet done carelessly it wrecks general ability. The signal is the replay ratio, learning-rate rewarming, and how you measure forgetting.

When a benchmark leaks into pretraining, the score measures memorization rather than ability. The signal is the detection methods (n-gram overlap, canaries, perturbation tests) and why fresh held-out evals matter.

A reward model converts pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

By swapping costly human labels for an LLM's preferences, RLAIF scales cheaply but carries over the labeler model's biases. What matters is how AI feedback gets gathered and where it silently breaks down.

Evict the oldest tokens to cap the KV cache and the model falls apart. Retain only the first few tokens and it keeps generating forever. What matters is grasping why those opening tokens behave as an attention sink.

At long context it is the KV cache, not the weights, that saturates the GPU. What matters is naming the levers (quantization, token eviction, head sharing, low-rank) and the quality cost of each.

In naive LLM serving most GPU memory bleeds away to KV-cache fragmentation. PagedAttention adapts virtual memory paging to win it back. What matters is explaining the fragmentation problem and how blocks resolve it.

Both compress an LLM to 4-bit weights, yet they choose what to protect in very different ways. What matters is GPTQ's error-correcting solve versus AWQ's activation-aware scaling, and the calibration each demands.

FP8 underpins modern H100-class training and serving. What matters is knowing the two FP8 variants, why a floating format beats INT8 on dynamic range, and where scaling still counts.

Candidates who cite one leaderboard number fail the follow-up. The interviewer wants you to name what SWE-bench measures, what it cannot (long-horizon multi-file refactors, real tool use, business logic), and which complementary benchmarks close those gaps.

VLMs such as GPT-4V and LLaVA attach a vision encoder to a language model through a projector. The signal is the image-tokens-as-prefix design, the alignment training, and the resolution and hallucination failure modes.

Speech LLMs either transcribe to text or model audio straight away as discrete tokens. The signal is the semantic-vs-acoustic token split and why end-to-end audio models outperform ASR-plus-LLM pipelines on latency and prosody.

Temperature 0 is not the same as deterministic, and the reason lives in the GPU kernels, not the sampler. What gets scored is naming the batch-invariance problem and knowing which fixes are real versus placebo.

Autoregressive models commit one token at a time, left to right. Diffusion language models generate the whole sequence at once and refine it over several denoising steps, which buys parallel decoding and global editing but has not yet caught the frontier. The signal is knowing the tradeoff, not just the buzzword.

The instinct is to blame the vision encoder. It is a token-budget problem, and the fix is to retrieve pages before you look at them. What separates a strong answer is knowing which question types page retrieval quietly cannot answer.

An emerging alternative to stuffing the window or building an index: keep the context outside the prompt and let the model write code to inspect it, recursing into the pieces. Knowing when it does not pay is most of the score.

Merging one adapter is a single matrix add. Merging four is where careers get interesting: the low-rank updates were each trained as if the others did not exist, and they collide. Here is what actually works and when to route instead.

The staged recipe is the easy half. The half that sinks real projects is data licensing, an evaluation bar that needs actual domain experts, and knowing which parts of the problem fine-tuning cannot solve at all.

Most staleness complaints are not model problems, and the engineer who says so first wins the round. Here is which changes belong in the index, which belong in the weights, and the lifecycle machinery that keeps months of refreshes from eroding the model.

Video breaks a VLM on arithmetic before it breaks it on vision: one frame per second of a ten-minute clip is hundreds of thousands of image tokens. Frame selection is not preprocessing, it is the architecture, and the strongest systems index the audio first.

A VLM is three parts and the interview is entirely about which ones you train. The staged recipe, why the vision encoder almost always stays frozen, and the silent failure where your model learns to answer from the text prior and never looks at the image.

Most candidates answer 'use the reasoning model for hard problems' and stop. The interviewer wants it framed as an eval and a budget problem: how you prove the extra thinking tokens paid off, and what you do when the model talks itself out of a correct answer.

One of the most common AI, ML, and GenAI design rounds. Sketching embed-retrieve-generate is easy; the signal lives in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents get reached for too often. A strong answer pushes past the hype: most tasks are better served by a single structured call, and agents pay off only under particular conditions. Here is when one is worth it and how to stop it from compounding errors.

Most RAG systems ship without genuine evaluation, which is why most of them quietly degrade. The signal is breaking evaluation into retrieval and generation, and scoring faithfulness apart from relevance. Here is the framework that makes RAG improvements measurable.

Enterprise RAG breaks on isolation, not on retrieval quality. The signal is enforcing tenant and document-level permissions server-side, at retrieval time, so the model can never surface data a user is not allowed to see. Here is the design that survives a security review.

A 2025-2026 question that trips up people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for deciding.

Multi-agent is the pattern most often reached for without cause in AI today. The signal is holding back unless the task genuinely calls for specialization or parallelism, knowing the supervisor and handoff patterns, and grasping why coordination magnifies failure.

Conversations and agent tasks grow past the context window, and the naive 'stuff the whole history' approach breaks on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieval quality is bounded by the query, and raw user queries are frequently poor for search. The signal is knowing the techniques that rewrite the query ahead of retrieval and which failure each one fixes.

Vector RAG quietly breaks on two query shapes: multi-hop chains and global 'what are the themes' questions. The signal is knowing precisely what the graph gives you and the construction cost it demands.

Retrieving every time is wasteful and occasionally harmful; retrieving never leads to hallucination. Self-RAG turns retrieval into a decision the model controls, then critiques what came back. The signal is the retrieve-on-demand plus self-critique loop.

Real corpora contradict themselves (old versus new policy, different teams), and naive RAG quietly picks one or fuses them into a wrong answer. The signal is spotting conflict and resolving it through recency, authority, and transparency.

Code-execution agents are powerful and risky: arbitrary model-generated code runs on your infrastructure. The signal is genuine isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Agents fail in the middle, not only the end, so final-answer-only scoring conceals the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and pinpointing where it broke. Here is the answer.

Real documents carry images, charts, and tables, not only text, and text-only RAG quietly drops them. The signal is knowing the two retrieval approaches and why the generator has to change too. Here is the answer.

A computer-use agent operates a real browser or desktop via screenshots and clicks. The signal is the perceive-decide-act loop together with the containment for an agent that can click, buy, or delete anything. Here is the answer.

An agent that performs actions is far riskier than one that only talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

Single-shot retrieval breaks on questions that chain facts ('who directed the highest-grossing film of 2019?'). The signal is decomposing or iterating retrieval rather than retrieving once, and guarding against errors compounding across hops. Here is the answer.

Late interaction reaches cross-encoder-like quality with near-bi-encoder scalability. The signal is the per-token embeddings plus MaxSim matching that earns you the middle ground, and understanding what it costs. Here is the answer.

Standard RAG retrieves once and generates; agentic RAG makes retrieval an iterative, reasoning-driven loop. The signal is the agent deciding whether, what, and when to retrieve, then re-retrieving until the evidence holds.

Late chunking reverses the usual order of operations to solve the lost-context problem with zero additional LLM calls. Everything hinges on which step runs first. Here is the answer.

At a billion vectors the index choice is a memory budget question before it's a recall question. Flat search is off the table, HNSW may not fit, and PQ swaps recall for RAM. Here is the call a staff engineer makes on the whiteboard.

A model that leads MTEB can still stumble on your jargon-heavy corpus, and most teams reach straight for the costly fix. There is a ladder, and the cheap rungs are the ones people skip. Here is the order to climb it.

An agent that retries the same failed action 40 times means a runaway bill and a stuck user. Termination is something you build into the harness, not something the model reliably decides. Here is the control layer.

Real tools time out, rate-limit, and hand back malformed JSON. An agent that assumes every call succeeds is a demo, not a product. Here is the error-handling layer that keeps it running in production.

When the right answer isn't even in the top-50, the generator can't rescue you. Low recall has a short list of usual suspects. Here is the order to check them so you fix the cause, not a symptom.

When a chain of LLM calls and tools yields a wrong final answer, 'the model was bad' is not a diagnosis. You need visibility into every step. Here is the tracing layer that turns a black box into something debuggable.

A 90% eval score alongside angry users means your eval set doesn't resemble reality. The fix is to make evaluation follow production, not the reverse. Here is how.

A live-coding round that tells apart engineers who have shipped agents from those who have only read about them. Interviewers want a genuine plan-act-observe loop, tools that query a real SQL database, guardrails that block the obvious failures, and a concrete evaluation plan, rather than one prompt masquerading as an agent.

Late interaction keeps one vector per token, so a corpus that fit in a few GB as single vectors can swell 100x. Interviewers want to see you know the compression and indexing tricks (centroids, residuals, PLAID) that make multi-vector retrieval shippable.

HyDE produces a fake answer to embed, which helps on vague queries but actively hurts on factual or out-of-domain ones. Interviewers want to hear you name precisely when the hypothetical document misleads retrieval and which multi-draft and hybrid variants recover.

Vector search over rows is the wrong tool when the user asks for a count or an aggregate. Interviewers want to see you retrieve the right schema, generate validated SQL, and know when to query the database instead of embedding it.

Splitting source files every 500 characters slices functions in half and wrecks retrieval. Interviewers want to see you chunk on syntax, retrieve by symbol and dependency, and blend lexical exact-match with semantic search the way code search actually needs.

Re-embedding 10M documents nightly is wasteful when only 0.5% changed. Interviewers want to see you upsert by stable id, handle deletes and tombstones in an ANN index, and compact before fragmentation tanks recall and latency.

An LLM can cite a source that does not say what the answer claims. The signal is verifying claim-by-claim entailment against the cited text, not merely that a citation marker exists, and knowing what to do when grounding fails.

Hybrid retrieval only beats either method when the fusion is tuned. The signal is knowing why you cannot simply add BM25 and cosine scores, how RRF's k constant behaves, and when a learned weight beats rank fusion.

An off-the-shelf cross-encoder is general; your domain carries jargon and relevance rules it never encountered. The signal is knowing when fine-tuning pays off, how to mine hard negatives, and how to avoid training a reranker that merely memorizes your retriever's mistakes.

A better embedding model is worthless if old and new vectors share an index, because their spaces are incompatible. The signal is the dual-index re-embed-then-cutover plan, the cost math, and how to prove the new model is actually better before you flip.

Linear agents lock into one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from decaying.

Multi-agent systems coordinate through handoffs and shared state. The signal is knowing the orchestration topologies, what a clean handoff protocol carries, and why naive multi-agent often loses to a single good agent.

Most people know MCP exposes tools. The deeper signal is the complete primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy conceals how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is grasping compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Any content an agent reads can smuggle in instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Screen-driving and browser agents break in ways chat agents never hit: stale DOM, pages that keep shifting, misplaced clicks. What interviewers watch for is the grounding and reliability methods that move a flaky demo toward something dependable.

Every RAG design says 'just filter by tenant_id.' Almost nobody can explain why that one line can quietly cut recall in half. The answer is in what the filter does to the ANN graph.

The CRM says the balance is $0 and the ledger says $412. An agent left to its own devices will average them into something confident and wrong. Designing the precedence before the model has to guess is the whole answer.

Freshness and reproducibility pull in opposite directions, and almost every team builds only for freshness. A pointer into a mutable index is not a record. Here is what an auditable knowledge base actually requires.

float32 is far more precision than ranking needs. int8 is close to free, binary is 32x smaller, and the difference between a working binary index and a recall disaster is one word: rescoring.

An hour-long agent run is a distributed workflow wearing an LLM hat. The signal is designing serializable state with step-level checkpoints, and knowing why resume is easy while replaying a side effect is not.

Two frontier coding agents made opposite bets on the same problem, and both published evidence. Candidates who pick a side on vibes miss what the interviewer wants: the axes that actually decide it, and what each choice costs in staleness, latency, tokens, and infrastructure.

The from-scratch implementation that frontier labs and NVIDIA really do ask. Writing it shows you grasp shapes, the scale factor, the causal mask, and how heads split and recombine. Here is a correct, readable implementation plus the follow-ups.

Beam search is the decoding algorithm everyone cites and few can code correctly. The signal is keeping k hypotheses by cumulative log-prob, adding logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.

DP feels like pattern-matching magic until you have a method. The signal is a repeatable drill: spot overlapping subproblems and optimal substructure, define the state and recurrence, then memoize or tabulate. Here is that method applied to a worked example.

BPE powers GPT-style tokenizers, and coding it shows you grasp how subword vocabularies get constructed rather than just knowing they exist. What interviewers watch for is the repeated merge of the most common adjacent pair.

Coding conv2d shows you understand what a CNN layer really computes rather than just being able to name it. Interviewers watch for correct output-shape math, tidy stride and padding handling, and awareness of the im2col trick frameworks actually rely on.

Coding the KV cache shows you understand why decode is efficient: you append the new token's K/V and reuse the rest rather than recomputing. Interviewers look for the append-and-attend-over-cache logic. The implementation follows.

A classic that pays off the two-pointer insight over brute force. Interviewers look for realizing that water at each position is capped by the min of the max walls on each side, then computing it in a single pass. The answer follows.

Edit distance is the archetypal 2D string DP, powering spell-check, diff, and fuzzy matching. What interviewers watch for: naming the subproblem and the insert/delete/replace recurrence. Here is the answer.

Turning a tree into a string and reconstructing it faithfully rests on one choice most candidates overlook. Skip it and the structure turns ambiguous. Here is the answer and the interview-grade reasoning.

LIS has an obvious O(n²) DP and a clever O(n log n) solution that catches people off guard. Interviewers look for the patience-sorting/binary-search approach that maintains 'tails'. Both appear below.

Minimum Window Substring is the hard sliding-window problem that probes expand-and-contract with a character-count map. Interviewers look for the grow-to-valid then shrink-to-minimal pattern, verified in O(1) per step.

This notoriously hard problem demands better than the O(m+n) merge: an O(log) binary search on the partition. Interviewers look for binary-searching the split point so the left halves stay below the right halves. The answer follows.

Single-head attention shows up often; multi-head layers on the split-into-heads, attend-per-head, concatenate pattern. What gets scored is the reshape into heads plus a sharp explanation of why several heads help. Here is the implementation.

Computing the max of each size-k window the obvious way costs O(n*k). The move that cuts it to O(n) is a monotonic deque of indices, and why it stays linear catches most candidates off guard.

Comparing all pairs across a million documents is a trillion comparisons. MinHash estimates Jaccard similarity from a tiny signature, and LSH converts dedup into a near-linear scan. Here is the implementation.

RoPE is why modern LLMs extrapolate to longer contexts, and it works by rotation, not addition. Coding it shows you truly understand how position enters attention. Here is the implementation.

Negative sampling in word2vec and recommendation pulls millions of weighted samples; running each at O(log n) becomes a bottleneck. The alias method turns each draw into O(1) after an O(n) setup. Here it is.

When you shard a vector index or route requests across model replicas, naive 'hash mod N' reshuffles everything the moment N changes. Consistent hashing moves only a small fraction. Here is the implementation.

Negative subtrees, a path that bends through any node, and a return value distinct from the answer you track: this problem stuffs three traps into one DFS. Here is the clean O(n) solution and the reasoning that holds up under follow-ups.

The words encode a partial order among characters. Convert each adjacent pair into a directed edge, then topologically sort. The traps are the prefix edge case and spotting contradictions. Here is the full solution with cycle handling.

Treat each word as a node and one-letter edits as edges: the shortest ladder becomes a shortest path in an unweighted graph, which means BFS. The detail that decides pass or fail is producing neighbors in O(26 * L) without scanning the entire dictionary. Here is the pattern.

Working out arithmetic with precedence and nested parentheses is a parsing task, and a single stack manages both neatly. The traps are operator precedence, multi-digit numbers, and the sign of truncated division. Here is a one-pass solution.

The classic multi-round build screen: a plain key-value store that gains new requirements at each stage (TTL, transactions, scans). The signal is not stage 1, it is whether your code takes on stage 4 without a rewrite. Here is how to design for it.

OpenAI's signature build screen: a credit ledger that issues GPU-hours, spends them, and lets unused grants lapse in issue order. The trap is charging against the right grant first. Here is the FIFO-by-expiry design that holds up through every follow-up.

A classic build screen: a key-value store whose writes inside a transaction can be committed or discarded, with transactions that nest. The trap is mutating the base store directly. Here is the overlay-stack design that keeps rollback O(1).

Most candidates start rewriting. The interviewer is grading whether you isolate pure logic from side effects before you touch a line, because that is the difference between a safe refactor and a silent behavior change.

Sierra's signature build screen: cells hold values or formulas that reference other cells, and editing one has to recompute its dependents without infinite loops. The signal is the dependency graph plus cycle detection. Here is the topological-eval design.

A segment tree serves any associative range query (sum, min, max, gcd) with point or range updates in O(log n). The signal is the recursive split into covered, disjoint, and partial nodes, plus lazy propagation for range updates. Here is the answer.

KMP matches a pattern in linear time by precomputing a failure function that skips redundant comparisons rather than backtracking the text. The signal is what the prefix function actually holds and why the text pointer never moves backward. Here is the answer.

Matrix exponentiation evaluates linear recurrences like Fibonacci at index n in O(log n) by raising a transition matrix to the nth power through binary exponentiation. The signal is building the transition matrix and squaring it. Here is the answer.

A build-it-yourself check on the EM algorithm and soft clustering. What matters is the pair of alternating steps (responsibilities, then weighted re-estimation), log-sum-exp for numerical stability, and seeing how GMM extends k-means. The code follows.

A build-it-yourself check on recurrent forward passes and backprop through time. What matters is the shared-weight recurrence, summing gradients over timesteps, and articulating the vanishing-gradient problem. The code follows.

A build-it-yourself check on gated recurrence. What matters is wiring the forget/input/output gates and candidate correctly, keeping cell state distinct from hidden state, and explaining why the additive cell path cures vanishing gradients. The code follows.

A build-it-yourself check on how PyTorch actually works under the hood. What matters is assembling a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Below is a minimal engine.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams stays truncated mid-token for the whole generation. What matters is a clean strict parser plus a small repair layer, not a second parser. The code follows.

The lookup is three lines of linear algebra. What separates a cache from an incident is where the similarity threshold came from, and the one-token queries ('2023' vs '2024', 'not') that no threshold can catch because the embedding barely moves.

Asking a model 'is this answer hallucinated?' barely beats a coin flip. This is the build: claim extraction, a judge forbidden from using world knowledge, a denominator that excludes hedges, and one model call per claim, which decides where the check can run at all.

A 22-point AUC jump is both an opportunity and a red flag. Weaker candidates cheer; sharp ones grow wary and know precisely which leakage checks belong before anything ships.

Shipping a model is itself an experiment, and this question tells apart people who run A/B tests from those who p-hack them. The signal is pre-registering the metric, sizing the test, and holding back from peeking.

EM is the classic latent-variable algorithm, and a GMM is how it appears in practice. What interviewers reward is the E-step/M-step alternation, why it is soft clustering where k-means is hard, and the honest caveat that it only reaches a local optimum. Here is the answer.

Patches as tokens, global attention from layer one, and a weaker inductive bias than a CNN. What interviewers reward is naming the data regime where each architecture wins and why. Here is the answer interviewers score highest.

Forward noising is fixed, reverse denoising is learned, and the training loss is a plain noise-prediction regression. What interviewers reward is why that beats a GAN's minimax and what the VAE and U-Net each do in latent space. Here is the answer.

Senior DS loops test causal reasoning beyond the randomized A/B test. What interviewers reward is naming the quasi-experimental method that fits and stating the single assumption it lives or dies on. Here is the answer for when randomization is off the table.

Self-supervision is how modern models pretrain on unlabeled data, the engine behind LLMs and modern vision. The signal is the pretext-task idea and the genuine difference between contrastive and masked-prediction objectives.

Multimodal is now table stakes, and this tests whether you understand how images and text reach a shared model. The signal is CLIP's contrastive alignment and how modern VLMs feed image features into an LLM's token space.

GANs shaped generative modeling for years, and interviewers look for the minimax game, the failure modes (mode collapse, instability), and the exact reason diffusion overtook them for images while GANs still lead on speed.

Autoencoders and VAEs sit beneath representation learning and generative modeling, including the VAE inside latent diffusion. What matters is the difference between a plain autoencoder and a variational one, and why sampling blocks backprop without the reparameterization trick.

A bandit learns and optimizes at the same time, moving traffic toward the winning arm during the experiment rather than holding the split fixed like an A/B test. Interviewers look for the explore-exploit tradeoff, the three core algorithms, and a clear sense of when a bandit wins over a clean A/B test.

HMMs are the classic probabilistic sequence model behind speech and tagging, and Viterbi is how you decode them. What matters is the hidden-states plus transitions/emissions structure, and that Viterbi is dynamic programming for the single best state path, not a probability.

XGBoost and LightGBM win most tabular problems, and interviewers want more than 'it's boosting.' What matters is the fit-to-residuals mechanism plus the engineering (regularization, histograms, second-order) that makes it both fast and accurate.

Contrastive learning is how modern embeddings (CLIP, sentence encoders, SimCLR) are actually trained. What matters is the pull-positives-push-negatives objective, the InfoNCE loss, and why the number and hardness of negatives makes or breaks quality.

Grid and random search ignore past results. Bayesian optimization learns from them, and what matters is whether you can explain the surrogate plus acquisition loop and name the exact condition where the sample-efficiency is worth it. Here is the answer.

Gaussian Processes give predictions with principled uncertainty, which is why they power Bayesian optimization. What matters is the distribution-over-functions intuition, the kernel's role, and naming the O(n cubed) wall. Here is the answer.

Running an A/B test is easy; running a valid one is hard. What matters is naming the traps that produce confidently wrong conclusions and the fix for each. Here is the checklist a senior experimenter carries.

GNNs power recommendations, fraud, and molecule modeling by learning over graph structure. What matters is the message-passing mechanism, why k-hop matters, and why you keep them shallow. Here is the answer.

Two inexpensive regularizers that cure overconfident classifiers. What proves you understand them is knowing that one softens the target while the other softens the input, and precisely why gentler signals produce calibrated models. Here is the answer.

Ranking differs from regression: a wrong absolute score is acceptable, a wrong ordering is not. What shows depth is the pointwise/pairwise/listwise breakdown and why pairwise (LambdaMART) remains the practical default. Here is the answer.

Classic bias-variance predicts that larger models eventually overfit, but deep nets keep improving even after they memorize the data. What shows depth is explaining the second descent and why over-parameterized models generalize. Here is the answer.

Wrap any model and obtain prediction sets with a provable coverage rate, with no distributional assumptions needed. What shows depth is the calibration-set plus nonconformity-score mechanism and exactly what the guarantee does and does not promise. Here is the answer.

Time-to-event problems (churn, failure, conversion timing) conceal a twist that quietly biases ordinary regression. What shows depth is naming that twist and choosing the right framing. Here is the answer.

Plenty of real problems hand you confirmed positives but never confirmed negatives, only unlabeled data. The shortcut everyone grabs quietly biases the model. What shows depth is naming the regime and its fix. Here is the answer.

A pre-existing gap between your groups means randomization or instrumentation is broken, and the entire experiment is suspect. Strong candidates spot sample-ratio mismatch immediately. Here is the full diagnosis and the disciplined response.

A 92% aggregate accuracy can mask 60% on the segment that matters most. Averages are exactly where these failures stay buried. Here is how to surface them and the menu of fixes that actually map to the cause.

When labels exist only for the cases you already acted on, the model learns a distorted world: strong offline, blind to everyone you never saw. Worse, its own decisions choose the next labels. Here is how to spot it and counter it.

'The model got worse' can mean three very different things with three different fixes. Retrain a model that a pipeline bug actually broke and you just bake in garbage. Here is the order to triage it in.

A small labeled set is the normal place to start, not an excuse. The strong answer is a ladder of techniques that pull signal from unlabeled data, pretrained models, and the labeling budget. Here it is.

Most real datasets carry wrong labels, and they quietly cap your accuracy. Hand-cleaning all of it does not scale. Here is how to surface the bad labels and train robustly around them.

Generative training breaks in distinctive ways: GANs collapse to a handful of outputs, diffusion samples come out noisy or blurry. The symptom points to which knob to turn. Here is the diagnosis.

Metric learning succeeds or fails on the pairs you feed it. Random negatives teach almost nothing, and the margin plus the mining strategy determine whether the embeddings are any good. Here is how the choices interact.

In search and recommendation, A/B tests can be slow and noisy because they pit different users against each other. Interleaving compares two rankers inside the same user's results and spots winners with far less traffic. Here is how.

Boxes and labels, not a single tag per image. The signal is picking two-stage versus one-stage from your latency and accuracy budget, knowing what NMS and anchors really do, and reporting mAP correctly. Here is the answer interviewers score highest.

Semantic labels every pixel by class; instance pulls each object apart. The signal is knowing that the first cannot count overlapping objects, why Mask R-CNN adds a mask branch on RoIAlign, and what RoIAlign fixed. Here is the answer.

Most candidates can recite 'maximize the margin'. The dual is where you prove you really understand why only support vectors matter and where the kernel trick originates. Here is the derivation an interviewer wants.

A Gaussian mixture trained by EM has a well-known trap: one Gaussian shrinks onto a single point and the likelihood races to infinity. Understanding the cause, plus the three standard remedies, tells apart people who merely ran sklearn from those who grasp it.

Both label sequences, but one is generative and one is discriminative, and that gap decides whether you can add overlapping features. Here is the comparison that proves you understand the modeling tradeoff, not merely the acronyms.

XGBoost made boosting practical; LightGBM made it fast. The answer is histogram binning, gradient-based sampling, and leaf-wise growth, each carrying a real tradeoff. Here is what they do and where they hurt.

Bayesian inference requires an intractable normalizing constant, and MCMC works around it. The signal is explaining the acceptance ratio, detailed balance, and why you can drop the constant entirely. Here is the answer.

Resampling, reweighting, and focal loss tackle class imbalance from different angles, and each carries a real downside. The signal is matching the method to the model and metric, not blindly oversampling. Here is the breakdown.

IV is the causal tool people name but cannot defend. The signal is spelling out the two conditions an instrument must meet, why one is testable and one is not, and what 2SLS actually estimates. Here is the answer.

Propensity scores promise to imitate an experiment from observational data. The signal is knowing what they can and cannot fix, the overlap trap, and why IPW blows up. Here is the answer that sets careful candidates apart.

A response model predicts who will convert; an uplift model predicts who converts because of the treatment. The signal is the four-quadrant intuition and how you evaluate uplift when an individual's lift is never observed. Here is the answer.

CUPED can halve the sample size an A/B test requires without touching validity. The signal is explaining why subtracting a pre-experiment covariate reduces variance yet never biases the estimate. Here is the answer.

Checking an experiment daily and stopping the moment it hits significance can triple your false-positive rate. The signal is knowing why, and the family of methods that make continuous monitoring valid. Here is the answer.

Standard A/B math assumes one user's treatment does not affect another's outcome. In marketplaces and social networks that assumption breaks and your estimate is biased. The signal is naming SUTVA and the right randomization unit. Here is the answer.

The Kalman filter is optimal Bayesian tracking under linear-Gaussian assumptions, and it amounts to two steps repeated forever. The signal is explaining what the gain trades off and when the assumptions break. Here is the answer.

Threshold alarms page you all weekend yet sleep through Monday's actual outage. What interviewers reward is stripping out seasonality first, matching the detector to the anomaly, and tuning against alert fatigue. Here is the answer that holds up in production.

Fairness has no single number, and a well-known result proves the main three cannot hold together. What interviewers reward is defining each metric precisely and explaining the impossibility rather than picking one blindly. Here is the answer.

Raw gradient saliency maps are noisy and saturate. Integrated gradients cures both with two axioms and a path integral, yet the baseline choice quietly decides the answer. Here is what a careful candidate explains.

You observe repayment only for applicants you approved, so your training data is censored by your own past policy and your offline test set is censored the same way. The fix starts with something you must have logged years ago.

Pipelines fail and retry, so the real question is whether a retry corrupts your data. What interviewers look for is idempotent writes (not 'exactly-once delivery') alongside watermarks for late data. Here is how to keep re-runs safe without double-counting.

The Databricks-flavored performance question. What interviewers want is a candidate who heads straight for the usual suspects (skew, shuffle, spill) through the Spark UI rather than guessing. Here is the diagnostic order and the fixes that genuinely move the needle.

A hard streaming-systems question: dedup at high throughput while keeping state bounded. What interviewers reward is a tiered state design (a probabilistic filter ahead of durable state) plus watermark-driven eviction. Here is the architecture that does not OOM.

The classic product-analytics SQL question. The signal is grouping users by signup cohort, deriving each activity's period offset, and counting distinct returners per offset to build the retention triangle.

You cannot wait for all the data, and buffering an unbounded stream will OOM. The signal is windowed joins with watermarks for stream-stream, lookup joins for stream-table, and a clean account of late and out-of-order events.

The number-one cause of mysteriously slow Spark jobs: one partition handles most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.

A LATERAL join lets a FROM-clause subquery reference columns from the table ahead of it, which a normal join cannot. The signal is spotting the per-row use cases: top-N per group, table functions, unnesting arrays.

The most frequent way SQL leaks the future into a training set is a sloppy join to a feature table. The fix is point-in-time correctness, implemented as an as-of join. Here is how to write it.

Funnels seem like plain counts until you demand the steps occur in order and within a window. Tallying each step on its own overstates conversion. Here is the ordered-funnel query.

Consecutive-run problems (active streaks, uptime windows, price-stable periods) all boil down to one trick. Once the row-number difference clicks, they collapse to a window function and a group-by.

Defining the label is half the modeling problem, and the SQL conceals two leakage traps: reaching into the future for features, and a label window not yet fully observed. Here is the correct query.

Production drift monitoring frequently lives in the warehouse, not a fancy tool. Comparing two distributions in SQL with a metric like PSI is a few CTEs. Here is how to compute it.

An org-chart recursion walks a tree, but a general graph has multiple paths and back-edges. The signal is building up the visited path to prune cycles and ranking paths by cost to land the shortest one.

Two queries that look identical apart from ROWS vs RANGE return different numbers, and most candidates cannot explain why. The signal is knowing RANGE groups by value (peers) while ROWS counts physical rows.

The reflex is 'add an index,' but a good engineer reads the plan before touching anything. The signal is spotting the operator that dominates runtime, comparing estimated against actual rows, and knowing the seq-scan, bad-join, and spill patterns.

Pick the wrong partition key and you end up with millions of tiny files or lopsided giants. The signal is partitioning on a low-cardinality filter, clustering inside partitions on the next predicate, and keeping an eye on file size.

Each of the three brings ACID to object storage, yet they were built to solve different problems. The signal is fitting the format to the workload (engine neutrality, Spark-native ACID, or streaming upserts) instead of naming a favorite.

Overlapping subscription or session windows have to collapse into clean periods, and a naive self-join runs quadratic. The signal is the gaps-and-islands trick: a running max end-time that marks where each new island begins.

Events arrive minutes or days after they happened, and a window that has already closed reports wrong counts. The signal is event-time vs processing-time, watermarks with allowed lateness, and how you fix already-emitted aggregates.

A backfill that double-counts or knocks over the live pipeline is worse than the bug it fixes. The signal is idempotent partition-scoped rewrites, keeping backfill compute isolated from live runs, and validating before the swap.

Plenty of candidates know only 'use Debezium.' The signal is comparing three CDC mechanisms on how each captures deletes, loads the source, and preserves ordering, plus the snapshot-to-stream stitching that trips up real deployments.

Anyone can say 'filter and dedup.' The signal is the funnel arranged by cost, the MinHash/LSH banding math, and recognizing that the shuffle across billions of documents is what actually runs up the bill, plus the benchmark decontamination people forget until their eval numbers get challenged.

Two problems wear one costume here: how you would have caught this before full rollout, and how you debug it after the fact. Cover both and you show senior judgment. This walks through the staged-rollout and root-cause playbook.

The most common ML system design round. Interviewers reward the funnel structure: candidate generation, then ranking, then re-ranking, with the right model at each stage plus an honest plan for cold start, freshness, and feedback loops. This lays out that structure.

Severe class imbalance, a tight latency budget, and an adversary who keeps adapting. What interviewers watch for: treating imbalance honestly, setting the operating point from costs, and building for the feedback loop. This covers the end-to-end design.

Models fail silently, so the real question is whether you would notice. What interviewers look for: watching the right layers (operational, data, prediction, outcome) and alerting on drift without drowning in false pages. This covers the system and the metrics that count.

Multimodal search checks whether you grasp shared embedding spaces and the retrieve-then-rank pattern under a real catalog's scale and freshness. What interviewers reward: CLIP-style joint embeddings, hybrid retrieval, and honest relevance evaluation. This lays out the design.

Text-to-SQL is deceptively hard because correctness is all-or-nothing and the failure mode is a confident wrong number. What interviewers reward: schema grounding, query validation, and a safety layer, not merely 'prompt an LLM with the schema.' This lays out the production design.

Moderation is a multi-stage classification problem with harsh tradeoffs: false negatives cause real harm, false positives silence legitimate users, and the adversary keeps adapting. What interviewers reward: the tiered pipeline, per-severity precision/recall calibration, and human-in-the-loop. This lays out the design.

The moment a company runs LLMs in more than one place, it needs a gateway. The signal is the cross-cutting concerns (cost, reliability, observability, governance) that a gateway consolidates, not merely 'proxy the API.' Here is the design.

Ads ranking is where calibrated probabilities collide with tight latency and money. The signal is knowing that CTR has to be calibrated (not just ranked), the feature and serving design, and the auction context. Here is the design that goes past 'train a classifier.'

Anomaly detection seems simple until seasonality, cold start, and alert fatigue arrive. The signal is modeling the expected baseline (including weekly and daily cycles), picking unsupervised methods when labels are scarce, and tuning so you do not bury users in false alarms. Here is the design.

The canonical AI system-design question. The signal is covering both the model lifecycle and the serving stack at real scale, without rambling. Most candidates design only half and lose the points.

The modern agentic system-design question. The signal is the plan, search, read, synthesize, verify loop with per-claim citations, plus keeping cost and latency bounded. The hard parts are not the writing.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem rather than a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

Support is the flagship applied-AI use case, exercising agents, RAG, tools, and the critical human handoff. The signal is knowing when to answer, when to take action, and when to escalate, safely.

Limited GPUs, dozens of models, and every team demanding low latency for little money. The signal is whether you can shape that into a single governed serving fleet: continuous batching, KV cache, per-tenant quotas, and cost you can genuinely attribute.

AI code review succeeds or fails on precision: a handful of noisy comments and the team silences the bot for good. The signal is grounding in the diff plus repo context, unforgiving false-positive control, and winning developer trust one accepted comment at a time.

Summarize, draft, and triage, all across the most sensitive PII a person holds. The signal is grounding in the real thread, matching the user's voice, and a firm human-in-the-loop rule on anything that gets sent.

A GPU-heavy generative system where sampling steps drive the bill and a single bad image becomes a headline. The signal is the diffusion serving pipeline, the cost levers on per-image GPU time, and a required two-sided safety layer.

Screening resumes is both high-volume and high-stakes: a legally sensitive, bias-prone judgment about people. The signal is trading off throughput against fairness, human oversight, and explainability, not an LLM ranking resumes. Here is the design.

Transcribing in real time at scale probes streaming ASR, latency budgets, and GPU fleet management under heavy concurrency. The signal is chunked streaming inference with partial results, not batch transcription. Here is the design.

Document extraction (IDP) is a massive enterprise use case with a steep correctness bar: a wrong extracted number beats having none. The signal is the parse, extract, validate, human-review pipeline plus confidence-based routing. Here is the design.

A voice assistant links STT, an LLM, and TTS under a punishing latency budget where each stage piles on delay. The signal is streaming and pipelining the stages plus turn-taking, not three blocking calls. Here is the design.

Healthcare is the highest-stakes AI domain: a wrong answer can injure a patient, and regulators are paying attention. The signal is designing decision support with grounding, human oversight, and guardrails, never an autonomous diagnoser. Here is the design.

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation carries real consequences, and lawyers have been sanctioned over fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Dynamic pricing combines demand forecasting, optimization, and real-time serving, under fairness and trust constraints. The signal is the predict-then-optimize structure plus guardrails against perverse outcomes. Here is the design.

E-commerce search is where retrieval, ranking, and business goals meet, and it carries a strict exact-match requirement (brand, size, SKU). The signal is query understanding plus hybrid retrieval plus business-aware ranking. Here is the design.

A GPU-heavy generative-audio system with text/genre/lyric conditioning, minutes-long coherence, and copyright constraints. The signal is the generation pipeline plus the async, cost, and safety layer that gets it shippable.

Video generation is image generation plus the harsh constraint of temporal consistency, and it is the most GPU-intensive generative task around. The signal is diffusion over spacetime plus a serving design that holds up under multi-minute jobs.

Code migration is a high-value agentic task with a strict correctness bar: the migrated code has to still work. The signal is grounding in the real repo, transforming in verifiable chunks, and gating on tests, never one huge LLM rewrite.

Notification systems fail by spamming users until they mute, and a mute is a permanently lost channel. The signal is treating it as a per-user send/hold/batch/suppress decision optimized for long-term trust, not for clicks or volume.

Two models in a chain (ASR then summarization), a long-context problem buried inside, and a faithfulness bar where a fabricated action item carries real consequences. The signal is the pipeline plus how you keep a 3-hour transcript honest at thousands a day.

On-device AI gives up raw capability for privacy, offline use, and latency, under tight memory and battery limits. The signal is the small-quantized-model stack plus a hybrid router that sends the hard queries to the cloud without leaking private context.

The trap is swapping the classifier for an LLM. The real-time, imbalance, and adversarial constraints stay put. The signal is a hybrid: a fast calibrated model scores inline, LLMs investigate the gray zone off the hot path.

How large recommenders and search draw candidates from millions of items in milliseconds. The signal is why the user and item towers stay separate, how that enables precomputed embeddings plus an ANN index, and where ranking takes over.

Every keystroke has to return ranked suggestions in tens of milliseconds, at search-engine QPS. The interesting part is the data structure and what you precompute, not the query path. Here is the design that holds up under the latency budget.

ETA is spatiotemporal regression served in real time over conditions that shift by the minute. The signal is feature freshness, segment or stage decomposition, and the asymmetric cost of being wrong. Here is the design.

PYMK is graph recommendation at billion-node scale. The interviewer is testing for one instinct: do you generate candidates from the social graph, or naively try to score every pair? Here is the design that holds up through the follow-ups.

Feed ranking is multi-objective ML at massive scale with a harsh serving constraint. The interviewer wants to see whether you optimize a value model toward long-term satisfaction, or fall into the CTR trap that breeds clickbait. Here is the design.

The service under every RAG stack and semantic search box. The signal is your ANN index choice and how you deal with the three things that quietly break it: metadata filtering, live updates, and scale.

Predict, value, and bid on one impression in roughly 10ms, billions of times a day. The signal is the pCTR-to-bid pipeline under a tight latency budget, plus why calibration and budget pacing decide whether you make or lose money.

Image search is embedding retrieval at billion-image scale. A strong answer shows when a vision-only encoder is enough and when you need a CLIP-style shared space, plus how the ANN index actually serves the query. Here is the design.

Spam detection is adversarial, imbalanced classification under a low-latency bar. The interviewer is watching whether you set the threshold by cost asymmetry and rely on signals spammers cannot fake. Here is the design.

Query understanding is the front end of search that converts a raw query into structured intent. Get it wrong and even a perfect index returns garbage. The signal is the pipeline and how you measure it. Here is the design.

Online models need features that are fresh and computed the same way training computed them. The classic failure is training-serving skew that quietly degrades the model. The signal is the online/offline split built on one shared definition. Here is the design.

New users and new items carry zero interaction history, so collaborative filtering has nothing to work with. The candidates who pass treat it as a lifecycle, not a single trick. Here is how.

Rank items by relevance alone and you ship ten near-identical results and a filter bubble. What interviewers want is a re-ranking stage that scores the list as a set, plus the metric that proves it works.

Some questions need precise multi-hop facts that vector retrieval cannot chain. The signal is entity linking plus query translation over a graph (or GraphRAG), and knowing when the graph is worth its maintenance cost. Here is the design.

AV perception is safety-critical, real-time, multimodal ML: detect and track everything around the vehicle across multiple sensors. The signal is sensor fusion plus detection/tracking under a hard latency budget and a fail-safe bar. Here is the design.

Machine translation is seq2seq generation delivered across hundreds of language pairs under strict latency and quality budgets. What matters is sidestepping the quadratic pair blowup, saving low-resource languages, and serving it cheaply. Here is the design.

Detecting walking, driving, or a fall from a sensor stream is windowed time-series classification, typically on a battery-bound device. What matters is windowing, temporal smoothing, and the on-device constraints most candidates overlook. Here is the design.

Recommending events (or jobs) breaks ordinary recsys in two spots: items expire and items are local. What matters is how you handle perishability, geography, and the fact that every item is a cold-start item. Here is the design.

A recommender trains on data its own past recommendations produced, so it learns to confirm its own beliefs. What matters is spotting the loop, naming the biases it breeds, and knowing the exploration and debiasing fixes that break it.

Labeled data is the fuel for ML, and a labeling platform succeeds or fails on quality control. The signal is the workflow plus the quality math: consensus, gold honeypots, inter-annotator agreement, and active learning to spend the budget where it counts.

Detecting and boxing objects at scale reduces to one driving tradeoff plus a handful of CV specifics most candidates fumble: the detector family, NMS, focal loss, and why mAP rather than accuracy is the metric. Here is the design.

Forecasting a single series is textbook work; forecasting a million SKU-stores that have to reconcile, where stockouts cost more than overstock, is what the interview actually tests. The strong answer is global, hierarchical, and cost-aware.

A Devin-style agent that carries an issue all the way to a merged PR is the hardest kind to make reliable, since every step can fail and errors pile up. Here is the architecture and the loop that keeps it honest.

Exact-match caching does little when no two prompts match. Semantic caching reuses answers across similar queries, and its entire danger is handing back a near-match that is subtly wrong. Here is how to build it safely.

GPUs are scarce and costly, so multi-region HA for LLMs is not simply web-app HA with bigger machines. Capacity, routing, and failover all bend around the GPU constraint. Here is the design.

Batch scoring a billion items is a throughput-and-cost problem, the reverse of low-latency serving. Every choice that harms latency works in your favor here. Here is how to design for dollars-per-million-documents.

When every team evaluates prompts in one-off notebooks, an org ships regressions and argues about vibes. A shared eval platform turns quality into something measurable and comparable. Here is what it must provide.

An agent that clicks through a real website is slow, brittle, and a single wrong click away from a costly mistake. The architecture comes down to perception, action, and guardrails around irreversible steps. Here is how to build it.

RLHF and evals can only match the preference data behind them, and that data comes from humans whose quality swings wildly. The platform that yields trustworthy labels is a serious system in its own right. Here is its design.

'What was revenue by region last quarter?' looks like text-to-SQL, yet production analytics agents break on ambiguity, wrong joins, and confidently wrong numbers. Here is the architecture that makes the answers trustworthy.

An AI support agent working through millions of chats will occasionally be wrong, rude, or unhelpful, and no one can read them all. Watching quality at that scale is a design problem in itself, with its own failure modes.

Classic distributed-systems tradeoffs appear in ML infra with an ML twist: stale features and eventually-consistent reads carry model-accuracy consequences, not just correctness ones. Here is how to reason about it.

Anthropic's signature opener. The naive answer (one server pushes to N clients) is capped by the source uplink and scales linearly. The answer interviewers reward turns receivers into senders so the swarm's capacity grows with its size. Here is that reasoning.

The Dynamo question. Interviewers want three decisions made cleanly: how you partition keys so adding nodes does not reshuffle everything, how you replicate for durability, and where you sit on the CAP spectrum. Here is how to reason through all three with concrete quorum math.

Every API platform needs one. Interviewers reward the candidate who picks the right algorithm (token bucket vs sliding window) for the burst behavior, then cracks the genuinely hard part: holding one shared limit across many servers without a per-request round trip to a central store.

Looks simple until the follow-ups: which layer do you balance at, how do you keep traffic off a dead server, and how does the load balancer itself avoid being a single point of failure? Interviewers want the algorithm, the health-check loop, and the high-availability story.

The observability infrastructure question. Interviewers want you to split the three telemetry types, choose storage that withstands massive write volume, and design alerting that fires fast without burying on-call in noise. Here is the pipeline with retention and cardinality math.

The distributed ordering problem that sits at the heart of Kafka, replicated logs, and consensus systems. Interviewers look for you to face the uncomfortable fact that physical clocks lie, then pick the right instruments: a single sequencer, consensus, or logical clocks. This walks that progression.

The real-time messaging staple. Interviewers look for the delivery path (persistent connections, not polling), a fan-out strategy that survives large groups, and a clear story on ordering and delivery guarantees. This lays out that design, including the fan-out tradeoff that marks senior candidates.

This asks you to design the scoreboard, not the player. A strong answer keeps offline gates (AUC, calibration, NDCG) apart from the online verdict (A/B with revenue, user, and advertiser guardrails), and layers in counterfactual replay so you can trust a model before it ever serves a live auction.

A news feed differs from a generic recommender. Time is a first-class signal: a story that mattered this morning is noise by tonight. The interviewer expects recency decay, an engagement-versus-quality value model, and a plan for filter bubbles, not merely retrieval plus ranking.

Truth is not a label you can gather cheaply, and adversaries adapt as soon as you ship. A strong answer combines content, graph, and behavioral signals, places humans in the loop where precision matters, and treats adversarial drift as a permanent operating condition rather than a one-time training problem.

Video search is image search plus a time axis, and the time axis is the whole problem. A strong answer embeds frames, pools them into scene vectors, indexes with ANN at billion scale, and explains how a multimodal query (clip, text, or both) locates the right moment, not just the right video.

Inline completion lives or dies on tail latency: a suggestion that lands after the developer keeps typing is useless. Learn how to assemble the context window, hide model latency behind speculation and caching, and keep a tight feedback loop on acceptance rate.

Enterprise search lives or dies on permissions, freshness, and wiring into thirty messy SaaS sources; retrieval quality is table stakes. Learn how to fan out across connectors, enforce per-user access at query time, and blend lexical with vector search for results people trust.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context wins out, and how to evaluate summaries at scale.

Moderating billions of items a day with an LLM on each one is too slow and too expensive. Learn the tiered funnel (cheap classifier then LLM then human), how to tune thresholds for precision versus recall, and how to stay ahead of adversaries.

Personalizing an LLM is a retrieval and memory problem, not a per-user fine-tune. Learn how to assemble user context at request time, keep long-term memory from bloating the prompt, and respect privacy and the right to be forgotten.

Auto-resolving support tickets means grounding answers in your knowledge base, safely calling real APIs (refunds, account changes), and knowing when to hand off to a human. Learn the agent loop, the guardrails, and how to measure resolution without eroding trust.

Running experiments on LLM features is tough because outputs are open-ended and quality is fuzzy. See how to assign traffic, choose metrics beyond engagement, tame variance from non-determinism, and dodge the traps that let a winning variant lose once it ships.

Selling RAG as a product means wrangling messy ingestion, multi-tenant isolation, and per-customer index freshness, all behind a simple API. See the ingestion and query planes, how to keep tenants isolated, and how to hand customers eval and observability.

A celebrity posts and 50M followers need a notification. The tricky parts are not sending one message; they are fan-out strategy, deduplication, channel routing, and not flooding users. Here is the architecture that handles both the long tail and the viral spike.

A rider taps request and within seconds a nearby driver gets assigned. The interesting problems are geospatial indexing of moving drivers, the matching objective (nearest is not always best), and handling the race where two riders want the same car. Here is how to build it.

In payments, a duplicate charge or a lost credit is not a bug, it is a financial incident. The design turns on idempotency keys, double-entry bookkeeping, and an append-only ledger you can audit and reconcile. Here is how money systems stay correct.

A cache is easy until you spread it across many nodes. Then come consistent hashing, eviction policy, the thundering herd on a cache miss, hot keys, and how much staleness you can tolerate. Here is the design that holds up under real traffic.

Metrics, traces, and IoT data are append-heavy, time-ordered, and seldom updated. A general database handles this poorly. The wins come from columnar layout, compression tuned for timestamps, downsampling, and retention. Here is the time-series design interviewers want.

Raw clickstream lands in object storage and somehow turns into clean tables, dashboards, and ML features. The design questions are table format, the bronze-silver-gold layering, batch versus streaming, and how you handle schema drift and late data. Here is the lakehouse blueprint.

Crawling a few pages is trivial. Crawling the web means a URL frontier with priority, per-host politeness, dedup across billions of URLs, and defenses against spider traps and infinite content. Here is the crawler architecture that scales without getting your IPs banned.

Background jobs fail, retry, and get redelivered. If a job sends an email or charges a card, running it twice is a real problem. The design combines at-least-once delivery with idempotent handlers, visibility timeouts, and dead-letter queues. Here is how to make jobs safe.

When a page loads, an ad slot goes to auction across dozens of bidders and a winner is picked before the page finishes rendering, all in under 100ms. The design covers the auction mechanism, the brutal latency budget, budget pacing, and fraud. Here is how an exchange runs millions of auctions per second.

Moving a cloud-API POC into a bank or defense network kills your model provider, your judge API, and your vendor dashboards in one stroke. Learn the signed-bundle supply chain, local evaluation loop, and zero-egress observability that make an air-gapped LLM stack actually operable.

The LLM is the easy part. The system is a learner model: a skill graph, a per-student mastery estimate, and an item selector that keeps the student at the edge of their ability. Here is the design, including the pedagogical constraint most candidates miss.

Five thousand businesses, five thousand assistants, one platform. The hard parts are tenant isolation you can defend, noisy neighbors, onboarding a tenant with zero data, and evaluating assistants you can never manually QA. Here is the control plane and the data plane.

You cannot review a live stream after publishing, because it already reached the audience. The broadcast delay buffer is the design, and everything else (frame sampling, the audio channel, the cascade) is built to fit inside it. Here is how it fits together.

The root cause is not a weak system prompt. A model trained to be agreeable is dangerous to a user in crisis, because the objective and the safety requirement point in different directions. The fix has to live outside the model. Here is what that architecture looks like.

A classifier trained on one culture's annotations does not generalize to another culture's speech, and the aggregate metric hides it. The signal is recognizing that the ground truth itself is the bug, then designing the policy and eval stack around that.

At thousand-GPU scale, hardware failure is routine rather than rare, and a run that cannot resume burns weeks. The signal is checkpointing strategy, deterministic resume, and holding lost work to a minimum. Here is the fault-tolerant design.

A dependable experiment platform goes well beyond splitting traffic in half. What matters is stable assignment, exposure logging, statistical discipline, and guardrails that hold up against peeking and sample-ratio mismatch. Here is the design.

An ML platform is the internal system that lets many teams build and ship models reliably. What matters is naming the components (data, features, training, registry, serving, monitoring) and arguing why standardizing beats per-team reinvention, plus when not to build.

Same model, two answers: clean offline, ugly online. The cause is almost always a feature computed differently across the two paths. Here is the diff-based hunt that localizes it to a single column.

A fraud model fed a feature an hour behind is half-blind, but recomputing everything in real time wastes money you don't need to spend. Freshness is a per-feature decision on a real cost curve. Here is how to manage it.

Regulators and customers will ask 'Why was I denied?', and 'we retrained since then' will not satisfy them. Reproducing a prediction exactly requires versioning everything. This spells out what 'everything' really covers.

Waiting on labels to measure accuracy means learning a model broke a month after it did. Production monitoring must function before the truth shows up. These are the signals you track instead.

Automated retraining seems like a convenience right up until an auto-retrained model quietly ships worse. The gates are the entire point. Here is how to automate retraining without automating a regression straight into production.

Editing a prompt is a code change that ships without a compiler or a unit test by default, so quality regressions slip out invisibly. Treating prompts and models as testable artifacts is what divides a toy from a product. Here is the harness.

Every team rebuilds a spreadsheet of training runs and then drowns in it. The interviewer wants the platform that ingests runs, params, metrics, and artifacts at high write volume, ties them together by lineage, and makes thousands of experiments comparable, which is a different system from a model registry.

A green build says nothing about model quality. The signal is an eval harness that runs deterministically in CI, checks against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

Promotion is the easy part. Governance covers who may ship what, on whose sign-off, with what documented, and how a model gets retired. Here is the control plane an auditor actually asks for.

'Just set a seed' misses the point. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism together. Here is the full checklist and where it springs leaks.

Manual rollback means minutes of damage while someone wakes up. What matters is defining the trigger signals, thresholds, and guardrails that revert automatically, without flapping on noise. Here is how to make the loop safe.

Saying 'route 5% to the canary' is the easy part. What matters is how you split traffic deterministically, gather enough signal to decide, and ramp on evidence rather than vibes. Here is the mechanics layer.

A platform that no one uses is wasted headcount; a platform built too early is premature. What matters is justifying the investment by leverage and designing a paved path teams adopt willingly. Here is the framing.

Four drift tests, four different assumptions. The weak answer lists them; the strong one knows which handles high-dimensional embeddings, which requires binning, and which returns a calibrated p-value. Here is how to choose.

Retrain too often and you burn money and risk regressions; too rarely and the model rots. The strong answer is a layered trigger policy with guardrails, not one cron job. Here is how to set the cadence.

Classic SRE SLOs assume a request is right or wrong. ML predictions are probabilistic and labels lag, so naive uptime SLOs miss the failures that count. Here is how to set SLOs that actually cover model quality.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline pairs a versioned dataset, layered scorers, and a CI gate. Here is the architecture.

A provider swap does not fail with a 500. It fails by staying up and quietly getting worse. The signal is knowing what actually changes under you on a model swap, and the shadow-plus-staged-ramp protocol that catches it before your users do.

Migrating models is a project you plan. Being migrated is a Tuesday. When the weights move under a stable API and your prompts were tuned against the old ones, every dashboard stays green while quality quietly rots.

Interviewers here want concrete figures, not 'grab a bigger GPU.' Weight memory stays constant, the KV cache scales with traffic, and the order you pull levers in settles the outcome. This walks through the napkin math and the serving stack.

The interviewer is checking that you understand how a model too large for any one GPU still gets trained, and can run the optimizer-state memory math that justifies sharding. This covers the parallelism taxonomy and the 16-bytes-per-parameter figure.

Quantization is the opening move for shrinking and accelerating models, and the interviewer expects more than 'use fewer bits.' What they grade is whether you know what each precision level gains you, why outliers wreck naive quantization, and how you demonstrate quality survived. This lays that out.

A staple at hardware-aware teams. What they look for is grasping that attention's cost is memory traffic rather than FLOPs, and that FlashAttention is an exact, IO-aware reordering rather than an approximation. This is the answer that shows you reason about the memory hierarchy.

The KV cache is what makes LLM serving hard, and the interviewer wants the mechanics: what it holds, why it caps concurrency, and the tricks that shrink it. This is the answer that shows you grasp decode-time economics.

Decode runs sequentially and is memory-bound, so latency tricks count. What they grade is whether you can explain speculative decoding's draft-and-verify mechanism (and why it stays exact) along with the other levers and when each fits. This is the latency toolkit.

Mixed precision is routine at scale, and the interviewer wants the numerics: why FP16 needs loss scaling, why BF16 mostly does not, and what remains in FP32. What they grade is whether you understand dynamic range vs precision. This is that answer.

A hardware-literacy question, particularly at NVIDIA and the labs. What they grade is whether you understand throughput vs latency hardware, the memory hierarchy, and why matrix multiplication maps onto GPUs and TPUs. This is the architecture-aware answer.

Kernel-level questions separate people who have written CUDA from people who have read about it. The signal is commanding the SIMT execution model and the three classic throughput killers, with a concrete fix for each. Here is the low-level answer.

FSDP and ZeRO let you train models too large for one GPU without splitting the compute. The signal is the ZeRO stages (what gets sharded at each level) and the communication you pay for it. Here is the answer.

Distributed training is bottlenecked by GPU-to-GPU communication, and these collectives are how the data travels. The signal is what each one does and which parallelism strategy relies on it. Here is the answer.

When a model is too large for one GPU you split the model itself, not just the data. The signal is telling tensor parallelism (split within a layer) apart from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.

LLM inference has two phases with opposite resource profiles, and co-locating them lets a long prompt stall everyone else's tokens. The signal is knowing why prefill and decode fight, and what separating them costs. Here is the answer.

OOM is the wall you hit most often in deep learning, and 'buy a bigger GPU' is the weakest reply. What counts is naming the memory consumers, knowing which one dominates, and pairing the right lever with it.

nvidia-smi reading 100% can mask the fact that you are tapping only a fraction of the hardware's real compute. What matters is MFU (useful FLOPs vs peak) and the gap between 'the GPU is busy' and 'the GPU is efficient'.

Past FP16/BF16, FP8 sits on the next rung of the precision ladder, and the newest tensor cores compute on it natively. What counts is knowing the two formats, what FP8 really buys, and why dynamic range demands careful scaling.

At long context the KV cache, not the weights, dominates GPU memory and decode bandwidth. What counts is spotting it as the binding constraint and naming the levers that shrink it without wrecking quality.

Running one full fine-tuned model per customer grows GPU count linearly and drains the budget quickly. A method exists to pack hundreds of variants onto a single GPU without sacrificing batching. The interviewer wants to hear how.

CPU-based autoscaling that suits a web tier quietly breaks on GPU inference: the signal is wrong, and replicas need minutes to warm. The interviewer wants the signals you genuinely scale on and how you mask the cold start.

Paying for accelerators that sit idle half the time is the most common waste in ML training, and the reflex to add more GPUs only makes it worse. The interviewer wants the profiling discipline that pinpoints what is starving them.

The counterintuitive reality of LLM serving: token generation is capped by how quickly you can read weights out of memory, not by arithmetic. Once that clicks, the entire optimization menu follows from one number.

Linear scaling is the marketing figure; the actual curve bends early for reasons rooted in physics, not bugs. Here is where the speedup leaks and how to recover it.

A 256-GPU job that freezes with no error at 3am is a special kind of pain. The causes come from a short, recurring list. Here is the systematic way to identify which one hit you.

Spot GPUs frequently run 60-90% cheaper, and they disappear with two minutes' notice. The savings hold only if a preemption costs you minutes rather than the whole run. Here is how to make that the case.

Naive 4-bit quantization can wreck quality, and the reflex to give up and serve fp16 leaves a large speedup unclaimed. The accuracy is usually recoverable. Here is the ladder of fixes.

Users experience the p99, not the median, and the tail is where serving systems quietly break. The causes are queuing and batching effects, not a slow model. Here is how to flatten it.

Pinning one GPU per model leaves most of a fleet stranded on idle silicon. Sharing safely is a genuine systems problem with four distinct mechanisms, each suited to a different traffic shape. Here is how to choose.

A serving box steady on short prompts falls over the instant a 30k-token request arrives, because KV-cache memory scales with sequence length times batch. Here is how to bound it without crashing.

At billion-parameter scale, training can be running smoothly and then the loss jumps and never returns, burning a fortune in compute. The causes and the playbook are familiar to the few who've done it. Here it is.

One long prompt can freeze every other user's token stream for hundreds of milliseconds. Chunked prefill carves that prompt up so decode keeps flowing. Here is the mechanism and the knob that controls it.

A shared system prompt is re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks align exactly. Here is the hashing and eviction machinery underneath.

GPU memory runs out well before you run out of conversations to cache. Offloading KV to CPU or NVMe buys capacity, but the interconnect can turn into the new bottleneck. Here is the bandwidth math.

FP8 serving is fast and usually fine, until it quietly degrades on the one workload your benchmark did not cover. What counts is knowing what FP8 breaks and how to validate it. Here is the recipe.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to strip it out.

Most models do not fill a GPU, so one model per GPU wastes money. The three sharing mechanisms (MIG, MPS, swapping) differ widely in isolation and overhead. Here is how to pick.

Autoscaling reacts in minutes, but a burst lands in seconds, and the gap is where your tail latency dies. Absorbing bursts is about buffers and shedding, not just adding replicas. Here is the playbook.

ZeRO and FSDP both shard training state across data-parallel ranks, but the trick is in when each shard is gathered and freed. Get the memory accounting and the all-gather timing and you can size any run.

Full activation checkpointing saves memory but costs a flat 30% extra compute. Selective recomputation wins most of that back by recomputing only the cheap, memory-heavy operations. Here is how to pick what to recompute.

The collective communication in distributed training is pure overhead unless it runs while the GPU computes. Hiding it is the difference between 30% and 55% MFU. Here is how the overlap actually works and how you check it on a trace.

On a large training run a node will drop, and the interesting question is not whether but how many GPU-hours vanish when it does. Checkpoint cadence, sharded writes, and quick restart settle that.

When all-reduce is your bottleneck, the culprit is nearly always a misconfigured path: traffic on the wrong link, a dead NIC, or a topology NCCL failed to discover. Here is the methodical way to track down the missing bandwidth.

Data, tensor, and pipeline parallelism each leave a single sequence's activations on one device, so training past 200k tokens hits a wall none of them can clear. The fourth axis shards the sequence itself, and the interview turns on the communication math.

When each request thinks for 30,000 tokens, serving swings from compute-bound prefill to memory-bound decode, and the KV cache turns into the resource you genuinely schedule. The levers that governed chat traffic stop being the ones that count.

Raw tokens per second is gameable: crank the batch size and the dashboard looks great while every request misses its latency target. Goodput is the throughput that actually meets your SLOs, and it is the number you size and autoscale on.

There is no one-shot patch for prompt injection, and answering 'sanitize the input' loses the round. What lands is defense in depth: privilege boundaries, handling retrieved content as data rather than instructions, and a human gate on irreversible actions. Here is the layered answer.

A CISO-facing question that trips up engineers who focus only on model quality. What matters is treating governance as architecture rather than a checkbox: minimization, isolation, auditability, and never training on customer data by default. Here is the framework.

Getting an LLM feature to production safely is an evaluation problem first and a model problem second. What lands is a layered eval-plus-guardrail design with honest, segmented metrics rather than a single 'safety classifier.' Here is how to measure and defend each failure mode.

Fairness questions catch engineers who treat it as a vibe. What lands is knowing the formal fairness metrics conflict mathematically, that you have to choose one deliberately for the context, and where bias enters the pipeline. Here is the rigorous, honest answer.

Most ML security centers on inference-time attacks; this one targets the training pipeline, where a poisoned dataset or a tampered dependency can hide a backdoor that clean-data evaluation never catches. What lands is naming the attack classes and recognizing that defense means provenance, not a single model fix.

Privacy starts as a governance requirement and turns into a training-time technique. What lands is knowing what differential privacy actually promises (and what it costs), plus how federated learning and DP fit together. Here is the rigorous, honest answer.

Two attacks aimed at a deployed model's confidentiality: copying its functionality through the API, and deducing who was in its training data. What lands is naming the precise mechanism each exploits and recognizing that every defense trades against utility.

Federated learning trains across decentralized devices without pooling data, yet a malicious participant can poison the shared model. What lands is the privacy mechanism plus the outlier-resistant aggregation defenses. Here is the answer.

GDPR erasure and copyright takedowns require pulling a user's influence out of trained weights, not just deleting a dataset row. The signal is knowing why deletion falls short and where retrain, SISA, and approximate scrubbing each sit on the cost-versus-proof curve.

Encryption protects data at rest and in transit, but the instant you compute on it the data sits as cleartext in memory. Confidential computing closes that gap. The signal is the TEE concept, attestation, and the AI workloads that genuinely need it.

Privacy in ML is a toolbox, not one switch, and each tool defends a different threat. The signal is mapping DP, federated learning, confidential computing, encryption, and minimization to what each one actually protects, and knowing they compose.

Federated learning transmits gradients, not raw data. The catch: a gradient is derived from the data, so it carries the data. The signal is knowing why 'we only share gradients' is not a privacy guarantee. Here is the answer.

With direct injection the user attacks the prompt. With indirect injection the payload sits inside a page, doc, or email the model reads. The signal is recognizing that retrieved and tool data is untrusted and can hijack the model. Here is the answer.

An enterprise RAG that retrieves across everyone's documents is a breach in waiting. The fix belongs in the retrieval layer, not the prompt. Here is the design that survives a security review.

Large models memorize and can be coaxed into emitting training examples verbatim, a genuine privacy and copyright liability. The defenses reach across data, training, and output. Here is the layered answer.

For consequential decisions, 'the model said so' will not satisfy a regulator. Adverse-action notices and the right to explanation limit which model you can even ship. Here is the governance view.

AI features create cross-tenant leak paths that classic SaaS isolation never had to reckon with: shared embeddings, shared caches, and fine-tunes that memorize. One leak ends an enterprise contract. Here is the isolation model.

Deleting a row is trivial. Removing a person's influence from embeddings, caches, derived datasets, and a trained model is not. GDPR and CCPA demand it regardless. Here is the plan that survives an audit.

When an agent holds private data, reads untrusted text, and can talk to the outside, injected instructions can steal data. This 'lethal trifecta' is the defining agent vulnerability. The fix is architectural, not a better prompt.

Before naming defenses, a security engineer maps the attack surface systematically. LLM apps present a wider, stranger surface than classic software. Here is how to threat-model one end to end.

A request that gets refused in one message often lands when broken across ten. The signal is understanding that conversational state is part of the attack surface, and that turn-by-turn classifiers see only fragments.

base64, leetspeak, invisible Unicode, and foreign scripts each let a payload look like gibberish to your filter yet read as a clear instruction to the model. The signal is knowing why the model decodes what the classifier cannot.

Roleplay, obfuscation, optimization-based suffixes, and multi-turn escalation are distinct attack classes that call for distinct defenses. The signal is organizing the space and pairing each class with a control instead of hoping one filter covers all.

A backdoored model acts perfectly until it meets a secret trigger, then flips. The signal is explaining why clean test accuracy never exposes it, and what detection actually buys you when the trigger is unknown.

An agent that trusts its tools and its own memory can be redirected by one poisoned record that resurfaces turns or sessions later. The signal is treating persistent state as an attack surface, not just the live prompt.

A hijacked agent does not need a 'send email' tool to leak secrets. A rendered markdown image, a URL parameter, or a DNS lookup will do. The signal is enumerating the covert channels and locking down egress, not just tools.

Asking the model to keep secrets or enforce permissions puts the job on the wrong component. The signal is enforcing access control before retrieval, in code you trust, and treating the LLM as untrusted compute over data that is already authorized.

Epsilon is the privacy knob, and shipping DP means picking a number, justifying it, and tracking what each query spends. The signal is treating epsilon as a finite budget that composes, not a magic constant. Here is the answer.

Federated learning keeps data on-device, but you pay for it in accuracy, debuggability, and engineering complexity. The signal is naming when those costs are justified and when a simpler centralized pipeline with DP wins. Here is the answer.

A fairness audit is a structured process: pick protected attributes, choose metrics that fit the harm, measure disaggregated performance, and document findings. The signal is knowing the metrics conflict and which one the use case requires. Here is the answer.

The EU AI Act hands different duties to high-risk deployers, GPAI model providers, and limited-risk systems. The signal is naming the specific obligations per tier, not merely reciting that risk tiers exist. Here is the answer.

An agent running on a single over-privileged service account is a confused deputy waiting to happen. The signal is per-user delegated credentials, token exchange, per-tool scopes, and an audit trail that names the human. Here is the answer.

Human-in-the-loop is the mitigation everyone writes into the risk register and almost nobody measures. When the reviewer anchors on a confident model output, your oversight control fails exactly on the cases it existed to catch.

Dropping the protected attribute is the answer most candidates give, and it accomplishes nothing: the attribute is reconstructible from the features you kept. Here is how to actually locate the proxies and what removing them costs you.

Single-axis fairness audits are just another form of averaging, and they average away the exact group that is being harmed. The hard part is not noticing that: it is handling the combinatorial blowup, the tiny cells, and the multiple-testing problem without chasing ghosts.

Picking epsilon is the easy half. The hard half is the accuracy you just gave up, and the levers that get it back are not the ones most candidates reach for.

Your guardrail reads text. The attack is not in the text. That single sentence is the whole vulnerability, and the red team you build from it looks nothing like the one you already have.

A failed prod deployment stress-tests judgment under pressure. Interviewers want to see stabilize-first instincts, clean communication, and a root-cause fix that heads off a repeat. Here is the arc.

Ethical pushback probes integrity and judgment under pressure. Interviewers look for principled, specific action, not a generic statement of values. Here is the arc that signals real backbone.

Rescuing a churning account is diagnosis under pressure: find the real reason, fix what you can, and win back trust with action. Interviewers look for a method, not a discount. Here is the move.

With billions of candidate items and only tens of milliseconds to choose the next 10, a feed is a latency problem first. The interview probes the two-stage architecture, how you set the objective when engagement fights integrity, and the biases that silently corrupt your training labels.

A track runs three minutes, a session runs an hour, and a fresh release has zero plays on launch day. The interview centers on mixing collaborative filtering with audio content embeddings, handling cold start on both sides, and reading a skip as the loud negative it really is.

Training a pCTR model is the easy part. The challenge is proving a change helps before it reaches revenue, when the model runs inside an auction, the logs surface only ads that won, and a 1% calibration error costs real money. This is an eval question, not a model question.

Watch time is the label everyone optimizes and hardly anyone measures cleanly. You see minutes only for videos people chose to play, the distribution is savagely skewed, and the slot they saw it in shifted the number. The interview asks whether you can predict a biased label honestly.

The positive class runs about 1 in 1,000, your labels show up late and noisy, and the instant you ship a model the adversary starts probing it. This is the uncommon ML problem where the data actively fights back, so the design hinges on labels, latency, and enforcement cost as much as on the classifier.

This model's output is a price input, not a ranking. A miscalibrated CTR skews the auction bid, so you overpay or underdeliver. Throw in conversions that arrive days after the click and advertisers with no history, and calibration plus delayed feedback become the entire interview.

An ETA is a promise. The interview tests whether you model it as a point estimate (and apologize when it misses) or as a distribution where p90 lets you under-promise, and how you feed live traffic and completed trips back into the model.

Millions of landmarks exist, most with only a handful of photos, and the next photo could show something not in your catalog at all. A flat classifier collapses on the long tail and never says 'I don't know.' The interview is embeddings plus retrieval plus a confident refusal.

Fraud is well under 1% of transactions, the labels land weeks late as chargebacks, and the fraudsters deliberately adapt to your model. Optimizing accuracy hands you a model that approves everything. The interview is about dollars-at-risk thresholding, label delay, and a review queue with a fixed headcount.

New courses launch every week and most browsers are signed-out or brand new, so the interesting half of this problem is what you recommend when you have almost no behavioral signal. The other half is avoiding the trap of optimizing engagement into a clickbait catalog.

This product succeeds or fails on a feedback loop measured in seconds: every swipe is a label, and the following video must respond to it. The tricky part is exploring widely enough to learn what someone enjoys while avoiding a filter bubble or the amplification of junk.

A search box is a recommender carrying a strong prior: the query. The difficult pieces are reading ambiguous intent, retrieving in two stages within a tight latency budget, and learning from clicks without teaching the model that whatever appeared first is best.