OpenAI AI Engineer interview questions

Nearly every candidate can recite softmax(QKᵀ/√d_k)V. What the interviewer is after is the variance derivation and the precise training failure the scale heads off. This is the response that tells rote recall apart from real grasp.

The most frequently asked applied GenAI question, and the one most candidates turn into a definition dump. What the interviewer wants is a decision framework with a default and the conditions that override it. Here is the call that earns points.

Alignment has become a baseline expectation even beyond 'safety' roles. The interviewer is looking for the three-stage RLHF pipeline and a sharp account of why DPO threw out the reward model completely. Here is the answer that leads with mechanism, not the buzzwords.

A tidy fundamentals question that trips up anyone who only knows GPT. The signal is pairing each architecture to its task through its attention pattern, and explaining why the field settled on decoder-only even for tasks that are not generation. Here is that answer.

Every customer wants to know how to stop the model inventing things. The weak reply is 'better prompts.' The signal is understanding why models hallucinate and stacking grounding, abstention, and measurement into a system you can defend.

Tokenization reads like plumbing until it accounts for your bill, your latency, and why the model stumbles on numbers and rare languages. The signal is understanding how subword tokenizers operate and the consequences that follow from them.

Attention is permutation-invariant, so with no position information a transformer cannot detect word order. The signal is knowing why, and why the field shifted from absolute sinusoidal encodings to relative ones like RoPE and ALiBi that extrapolate to longer contexts.

Every LLM product tunes these knobs, and the interviewer wants to hear what each one does to the probability distribution and which setting suits which task. That includes why factual and creative tasks call for opposite settings.

Evaluation is the toughest and most underrated part of shipping LLMs. The signal is understanding why public benchmarks mislead, why LLM-as-judge carries bias, and how to build a task-specific eval you can genuinely trust.

Perplexity is the classic LM metric. What matters is knowing it is just exponentiated cross-entropy, what it intuitively captures, and why low perplexity does not mean a good product.

In-context learning is the property that turned prompting into a paradigm; chain-of-thought is its most useful trick. What matters is knowing what 'learning' means here (no weight updates) and why making the model reason step by step actually helps. Here is the answer.

Two prompting strategies that purchase accuracy with extra compute. What matters is knowing how each explores the reasoning space, that they trade tokens and latency for reliability, and how to judge when the problem actually warrants it. Here is that answer.

ReAct is the pattern behind most tool-using agents. What matters is why interleaving reasoning with actions (and observing results) beats reasoning alone or acting alone. Here is the answer that ties the prompt pattern to real agent behavior.

RL alignment shifted from PPO to leaner methods, and DeepSeek-R1 put GRPO on the map. What matters is knowing what the value/critic model does in PPO and how GRPO replaces it. Here is the mechanism, not the acronyms.

The 2025-2026 landscape isn't 'bigger is better' anymore. What matters is knowing why small models and reasoning models exist, the test-time-compute tradeoff, and routing the right model to the right task. Here is the practitioner's view.

What separates a demo prompt from a production prompt is structure, versioning, and defense. What matters is treating the system prompt as a contract and templates as parameterized, testable artifacts. Here is the answer.

Complex tasks rarely fit one prompt, and free-text LLM output rarely drops into code cleanly. What matters is decomposing into chained steps and parsing each output. Here is the answer that ties prompting to real pipelines.

Jailbreaking is the adversarial sport of steering a model around its own safety training. What matters is naming the technique families and treating defense as a layered, never-finished arms race, not a one-time patch.

Instruction tuning converts a bare next-token predictor into a model that obeys instructions. What you are being tested on is slotting it correctly into the pretrain to SFT to alignment pipeline and stating precisely what it does and does not repair.

The classic automatic metrics for generated text. Interviewers want you to pair each with its task and identify the blind spot common to all three. Candidates who pass explain why none of them is the last word on quality.

Plenty of people cite benchmark numbers; almost nobody reads them carefully. What shows depth is explaining what each one measures and why leaderboard figures outrun real-world skill. The question the interviewer is saving is how you would spot contamination.

Fine-tuning lives or dies on data, not on hyperparameters. What shows depth is spelling out what makes a set good (quality, diversity, format, dedup) and defending why a few thousand clean examples outperform a million noisy ones.

Synthetic data is how teams manufacture training examples when real data runs short, and it comes with sharp failure modes. What shows depth is pinning down when it helps, how to keep quality up, and why training on model output over successive generations narrows the distribution.

LLM-as-judge is how you grade open-ended generation at scale, and G-Eval put a formal frame on it. What shows depth is naming the biases that make raw judge scores untrustworthy and demonstrating exactly how you harden them.

'Eval is the new system design' for LLM apps, and starting with zero data is the real test. What shows depth is bootstrapping a golden set, choosing task-specific metrics, and running evals continuously as a regression gate.

Cross-attention is how a model attends from one sequence into another, the mechanism behind encoder-decoder models and multimodal conditioning. What shows depth is knowing exactly where the queries versus keys/values come from.

Attention gets all the attention, but the feed-forward layers hold most of a transformer's weights and handle much of the per-token 'knowledge' work. What shows depth is knowing what the FFN computes and why it dominates the parameter count.

Context windows are not arbitrary figures; they follow from attention's quadratic cost, KV-cache memory, and how the model was trained. What shows depth is naming all three causes and matching each to the technique that extends it.

RLAIF and Constitutional AI trade human preference labels for AI-generated ones to scale alignment. What shows depth is naming exactly what they substitute for human feedback, and the consistency-versus-bias tradeoff that swap introduces.

Reasoning models are the 2025-2026 frontier. The signal is understanding they are trained (largely via RL on verifiable rewards) to generate long internal reasoning, and that they scale along a second axis: spending more compute at inference.

Tool calling powers every agent. The signal is grasping that the model executes nothing: it emits a structured request your code runs, then you hand the result back. Here is the loop and the details that determine reliability.

DPO simplified RLHF, and a family of variants now trade off what data and reference model they require. The signal is knowing what each variant drops or alters (reference model, paired data, separate SFT stage). Here is the answer.

Scaling laws explain why bigger models trained on more data predictably improve, and Chinchilla changed how we allocate compute. The signal is the model-size-vs-data tradeoff and the fact that many models were under-trained. Here is the answer.

How you decode from the model's probabilities shapes the output as much as the model itself. The signal is knowing the deterministic vs sampling methods and where each fits (factual vs creative). Here is the answer.

BERT and GPT diverge on one decision made before a single weight is trained: what does each token get to see? That choice ripples into attention, use case, and why one family now dominates. Here is the answer.

Prompting for JSON works until it doesn't, and at scale the tail breaks your parser. Constrained decoding makes malformed output literally impossible to sample. The signal is knowing exactly where in the loop the constraint takes hold. Here is the answer.

Chain-of-thought looks like the model showing its work, but it may not mirror the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is a separate one. The signal is the detection toolkit, why each signal is imperfect, and how you merge them into an action.

A model that claims a million-token window may not actually exploit it. The signal is knowing needle-in-a-haystack, where it fails, and how to probe genuine multi-fact reasoning across the entire context. Here is the answer.

Distillation gives you most of a frontier model's quality for a fraction of the serving cost. The signal is naming the three LLM-specific variants and knowing which one holds up against a closed API. Here is the answer.

The senior signal is fitting the method to what is genuinely missing rather than reaching for your favorite. Knowledge, behavior, or actions each steer you to a different tool. Here is the decision framework that scores highest.

A single trained embedding you can cut to any length and still use. The signal is the nested-prefix training objective and the coarse-to-fine retrieval win it enables at scale. Here is the answer.

Models are trained to be helpful, which quietly teaches them to never refuse. Restoring honest abstention is a system problem, not a prompt tweak. Here is the stack that actually shifts the refuse-when-unsure rate.

Token count is latency and dollars, not merely style. 'Be concise' hardly works, and max_tokens just cuts off mid-sentence. Here is how to genuinely shape length without amputating answers.

Alignment that piles on refusals and politeness often quietly erodes reasoning and coding. The alignment tax is real and measurable. Here is how to hold onto the safety gains without paying for them in capability.

Push any proxy hard enough and the model discovers the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

Few-shot accuracy can swing on example order by itself. If your prompt is fragile to things that shouldn't matter, the fix is structural, not lucky example-hunting. Here is what genuinely reduces variance.

The model's training cutoff is frozen, but your dependencies shift weekly. Retraining is the wrong reflex. Here is how teams keep code suggestions current without touching the weights.

Neural text degeneration is a known failure of likelihood-maximizing decoding, not a broken model. The fixes are specific decode-time knobs. Here is which lever to reach for and what each one costs.

Generation has to end somewhere, and the wrong stop rule either truncates answers or wastes tokens on trailing garbage. Here is how the EOS token, stop strings, and max-token caps actually interact.

CoT is not a blanket upgrade. On some tasks it does nothing or backfires, and the reason points to what to try instead. Here is the escalation ladder past 'let's think step by step'.

A model that scores 92% on your eval can still ship a product users hate, because the model is one component in a system. Teams that run only model evals get blindsided. Here is the distinction that matters.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field relies on a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

Push PPO hard enough and true quality peaks then declines while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

GRPO drops the value model and estimates advantage from a group of samples per prompt. The signal is the exact mechanics, why the group baseline works, and the normalization traps that silently bias training.

Swap a learned reward model for a checker that returns right-or-wrong, and reward hacking largely vanishes. The signal is why verifiable rewards beat learned ones for math and code, and where they break.

A reward model can only match the quality of its labels, and human preference labels arrive noisy and inconsistent. The signal is measuring inter-annotator agreement and the concrete steps that raise label quality.

Pairwise asks which one wins, pointwise asks how good a single output is. The signal is knowing which is more reliable, how Elo/Bradley-Terry aggregates comparisons, and where each method quietly misleads.

When a benchmark leaks into pretraining, the score measures memorization rather than ability. The signal is the detection methods (n-gram overlap, canaries, perturbation tests) and why fresh held-out evals matter.

The same model can land ten points apart on MMLU depending on prompt format and scoring method. The signal is knowing the knobs (log-prob vs generation, few-shot, normalization) that make evals reproducible.

A reward model converts pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

By swapping costly human labels for an LLM's preferences, RLAIF scales cheaply but carries over the labeler model's biases. What matters is how AI feedback gets gathered and where it silently breaks down.

Candidates who cite one leaderboard number fail the follow-up. The interviewer wants you to name what SWE-bench measures, what it cannot (long-horizon multi-file refactors, real tool use, business logic), and which complementary benchmarks close those gaps.

Grammar-constrained decoding seems like a solved problem until tokenization stops lining up with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing address it.

VLMs such as GPT-4V and LLaVA attach a vision encoder to a language model through a projector. The signal is the image-tokens-as-prefix design, the alignment training, and the resolution and hallucination failure modes.

Speech LLMs either transcribe to text or model audio straight away as discrete tokens. The signal is the semantic-vs-acoustic token split and why end-to-end audio models outperform ASR-plus-LLM pipelines on latency and prosody.

Temperature 0 is not the same as deterministic, and the reason lives in the GPU kernels, not the sampler. What gets scored is naming the batch-invariance problem and knowing which fixes are real versus placebo.

Autoregressive models commit one token at a time, left to right. Diffusion language models generate the whole sequence at once and refine it over several denoising steps, which buys parallel decoding and global editing but has not yet caught the frontier. The signal is knowing the tradeoff, not just the buzzword.

Your assistant scores 4.6 out of 5 on every individual turn and still loses the user by turn eight. Per-turn averages hide the failures that actually cancel subscriptions, and here is the session-level eval that catches them.

Most teams reach for a token-dropping compressor and pay an accuracy tax for tokens they could have had for almost nothing. The ranking of levers matters more than any single technique, and the top of the list is not compression at all.

Appending every turn is the default in tutorial code and it fails twice: on the context window and on cost. The signal is the layered assembly, the constraints you refuse to let the summarizer touch, and what you do the moment the user changes subject.

Four fusion families, and the choice between them is decided by one question most candidates never ask. Naming CLIP and LLaVA is table stakes; the score comes from knowing which one cannot reason and which one eats your context window.

The instinct is to blame the vision encoder. It is a token-budget problem, and the fix is to retrieve pages before you look at them. What separates a strong answer is knowing which question types page retrieval quietly cannot answer.

An emerging alternative to stuffing the window or building an index: keep the context outside the prompt and let the model write code to inspect it, recursing into the pieces. Knowing when it does not pay is most of the score.

Every LLM is one function: tokens in, a distribution over the next token out. The signal is what you say next, because the distance from that function to a useful assistant is three training stages, and almost every production complaint lands in exactly one of them.

The shift from one model per task to one model adapted many ways rewrote the job description. The interviewer is checking whether you know which half of the work got nearly free, which half did not move at all, and which problems classical ML still owns outright.

The whiteboard question that spans the entire stack, from BPE to the sampler. What separates a strong answer is knowing that prefill and decode stress opposite hardware resources, and that nearly every serving decision you will ever make falls out of that one fact.

One triangle of negative infinity, added before the softmax, is what makes transformer training parallel and generation stubbornly serial. Interviewers want the mechanism and the asymmetry it creates, not a definition of attention read back to them.

Most candidates answer with the 2015 ResNet answer and stop. The senior answer treats the residual stream as a shared communication bus, and that reframing is what makes logit attribution, the logit lens, and activation steering possible at all.

Definitions take thirty seconds. The interview is about the four things log-probs buy you in production: a confidence signal, banning tokens, guaranteeing valid output, and classification you can calibrate. Plus the tokenization gotcha that quietly breaks all of them.

The follow-up every interviewer holds in reserve after you say you fine-tuned something. A task-metric win proves almost nothing on its own. Here is the five-part protocol that separates a shipped model from a demo.

Most staleness complaints are not model problems, and the engineer who says so first wins the round. Here is which changes belong in the index, which belong in the weights, and the lifecycle machinery that keeps months of refreshes from eroding the model.

Summarization is the one task where the model was handed everything it needed and still made something up. The signal is separating intrinsic from extrinsic hallucination, knowing why ROUGE cannot see either, and gating on entailment before you ship.

Truncate, map-reduce, refine, retrieve, or stuff a long-context model. Each buys you something different and breaks differently. The candidates who score are the ones who pick based on the question being asked, not the document being read.

A one-word rewrite moving your accuracy several points is not a prompting problem, it is an uncalibrated decision function. The fix is measuring the model's prior over labels and dividing it out, not another round of tinkering.

The most repeated trick in prompt engineering, and the evidence for it is much thinner than the folklore. A screening question for whether you measure your prompts or inherit them from a thread of screenshots.

Every request pays for the prompt and almost nobody has looked at where the tokens actually go. It is rarely the part you can read: thirty tool schemas and seven stale examples usually outweigh the system prompt, and the output side costs several times more per token.

Naming RLHF is not an answer. The frame that scores is outer versus inner alignment, why Goodhart breaks any proxy objective you optimize hard enough, and why a classifier sitting at the boundary can constrain a model's behavior without changing what it is trying to do.

Video breaks a VLM on arithmetic before it breaks it on vision: one frame per second of a ten-minute clip is hundreds of thousands of image tokens. Frame selection is not preprocessing, it is the architecture, and the strongest systems index the audio first.

A VLM is three parts and the interview is entirely about which ones you train. The staged recipe, why the vision encoder almost always stays frozen, and the silent failure where your model learns to answer from the text prior and never looks at the image.

Most candidates answer 'use the reasoning model for hard problems' and stop. The interviewer wants it framed as an eval and a budget problem: how you prove the extra thinking tokens paid off, and what you do when the model talks itself out of a correct answer.

One of the most common AI, ML, and GenAI design rounds. Sketching embed-retrieve-generate is easy; the signal lives in chunking, hybrid retrieval, the rerank/latency tradeoff, and how you prove it works. Here is the structure that scores.

Agents get reached for too often. A strong answer pushes past the hype: most tasks are better served by a single structured call, and agents pay off only under particular conditions. Here is when one is worth it and how to stop it from compounding errors.

Most RAG systems ship without genuine evaluation, which is why most of them quietly degrade. The signal is breaking evaluation into retrieval and generation, and scoring faithfulness apart from relevance. Here is the framework that makes RAG improvements measurable.

A 2025-2026 question that trips up people clinging to dogma in either direction. The signal is a cost, latency, accuracy, and scale tradeoff, plus knowing the 'lost in the middle' failure of long context. Here is the framework for deciding.

Agents and integrations depend on the model returning valid, schema-conforming output. The signal is stacking constrained decoding, schema validation, and retries, rather than trusting a prompt to do the job. Here is the production-reliability answer.

Multi-agent is the pattern most often reached for without cause in AI today. The signal is holding back unless the task genuinely calls for specialization or parallelism, knowing the supervisor and handoff patterns, and grasping why coordination magnifies failure.

Conversations and agent tasks grow past the context window, and the naive 'stuff the whole history' approach breaks on cost, latency, and lost-in-the-middle. The signal is a tiered memory design: recent buffer, summarized mid-term, retrieved long-term.

Retrieving every time is wasteful and occasionally harmful; retrieving never leads to hallucination. Self-RAG turns retrieval into a decision the model controls, then critiques what came back. The signal is the retrieve-on-demand plus self-critique loop.

MCP standardized how agents connect to tools and data, but tool design is what determines reliability. The signal is knowing what MCP actually standardizes and the principles behind a tool the model can call correctly from its description alone.

Reflection is the pattern in which an agent critiques and revises its own work. The signal is knowing when it truly helps (external feedback grounds the critique) versus when self-critique on its own is theater.

Citations turn a RAG answer from 'trust me' into something checkable, and they are a genuine engineering problem. The signal is grounding each claim to a specific passage and verifying the attribution, not tacking a bag of links onto the end.

Plan-and-Execute and ReAct are the two leading agent control patterns. The signal is knowing the upfront-plan versus step-by-step tradeoff, why one costs less, and why most production agents wind up blending them.

Agents need memory beyond the context window, and 'just store the history' is the wrong answer. The signal is telling the memory types apart and mapping each to a storage and retrieval strategy. Here is the answer.

Context engineering is the discipline that took over from 'prompt engineering' for serious agent work. The signal is grasping that what you place in the context window (and what you omit) shapes behavior more than clever wording. Here is the answer.

Code-execution agents are powerful and risky: arbitrary model-generated code runs on your infrastructure. The signal is genuine isolation (containers/VMs), resource limits, and network/filesystem restrictions, not 'trust the model.' Here is the answer.

Tool selection and parameter extraction are where agents quietly break. The strongest answer treats the tool spec as the only thing the model sees, then repairs the design, not the model. Here is how.

Agents fail in the middle, not only the end, so final-answer-only scoring conceals the real problems and rewards lucky paths. The signal is evaluating the whole trajectory and pinpointing where it broke. Here is the answer.

Sub-agents are how you scale a complex agent task without one huge, noisy context. The signal is context isolation and clean delegation: a focused subtask in, a compact result out. Here is the answer.

Multi-agent systems succeed or fail on how agents share information and hand off work. The signal is naming the three communication patterns and the coordination failures that sink the naive version. Here is the answer.

One model can behave completely differently depending on the harness wrapped around it. The signal is recognizing that the scaffolding frequently sets agent quality more than the underlying model does. Here is the answer.

A computer-use agent operates a real browser or desktop via screenshots and clicks. The signal is the perceive-decide-act loop together with the containment for an agent that can click, buy, or delete anything. Here is the answer.

An agent that performs actions is far riskier than one that only talks. The signal is action-level guardrails: least privilege, argument validation, human approval for irreversible actions, and a blast-radius mindset. Here is the answer.

An agent that retries the same failed action 40 times means a runaway bill and a stuck user. Termination is something you build into the harness, not something the model reliably decides. Here is the control layer.

A live-coding round that tells apart engineers who have shipped agents from those who have only read about them. Interviewers want a genuine plan-act-observe loop, tools that query a real SQL database, guardrails that block the obvious failures, and a concrete evaluation plan, rather than one prompt masquerading as an agent.

Linear agents lock into one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

A context window is not memory. Real agents need a storage architecture: working buffer, episodic recall, and consolidated facts. The signal is the read/write paths and how you keep memory from decaying.

Tools fail: timeouts, rate limits, bad arguments, garbage output. A naive agent retries blindly or quits. The signal is classifying errors and matching each to the right recovery, with hard caps.

Most people know MCP exposes tools. The deeper signal is the complete primitive set (resources, prompts, sampling, roots) and the patterns: server composition, sampling for nested LLM calls, and scoping access safely.

Final-answer accuracy conceals how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is grasping compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Agents can critique and retry their own work. The signal is knowing how Reflexion's verbal feedback loop operates, when self-critique genuinely helps versus when it is theater, and what makes it real.

Any content an agent reads can smuggle in instructions that hijack it. The signal is knowing why filtering can't fully solve injection and which containment controls actually bound the damage when it succeeds.

Agents re-send growing context every step, so cost scales worse than linearly. The signal is knowing where the tokens go and the levers (caching, model routing, context trimming, step caps) that shrink the bill.

Screen-driving and browser agents break in ways chat agents never hit: stale DOM, pages that keep shifting, misplaced clicks. What interviewers watch for is the grounding and reliability methods that move a flaky demo toward something dependable.

Retrieval is exonerated, so the bug lives in generation. There are four usual causes and they have different fixes, and the one nobody names is that the model was never given permission to say it does not know.

An hour-long agent run is a distributed workflow wearing an LLM hat. The signal is designing serializable state with step-level checkpoints, and knowing why resume is easy while replaying a side effect is not.

Tool calling costs one LLM round trip per action. Code costs one round trip for a loop over 500 records. The signal is knowing exactly what you trade away when you let the model write the program instead of the call.

Agent latency is a step-count problem disguised as a speed problem. The winning move is removing round trips, not shaving milliseconds off each one, and knowing which step you can delete is the whole interview.

A proactive agent runs on a trigger, with no human watching at the moment of action. That inverts the risk model, and the interviewer is listening for whether you separate notifying from acting.

The agent loop assumes an observation is a short string, and images break that assumption in both directions. The signal is a context policy that keeps pixels out of the transcript and a verification step for the files the agent produces.

Everyone says 'cap the budget'. The signal is the enforcement mechanism: a budget object that travels with the run, checked before each call rather than after, with reserved headroom for a final answer and correct accounting across sub-agents.

The runaway loop gets all the attention. The failure that actually ships is the agent that declares victory on step 4 with half the work missing. The signal is a finish tool the harness can reject, and a completion check that reads the artifact instead of taking the model's word for it.

Labs reach for this practical screen often, since it probes concurrency, time handling, and judgment inside 20 lines. The pitfall is a background thread that burns CPU. Below is the lazy-refill approach interviewers expect, along with the follow-ups.

The single most-asked design-coding question, and a common warm-up at the labs ahead of the ML follow-ups. The signal is the hashmap-plus-doubly-linked-list for genuine O(1), then handling the TTL and concurrency follow-ups cleanly. Here is that build.

The from-scratch implementation that frontier labs and NVIDIA really do ask. Writing it shows you grasp shapes, the scale factor, the causal mask, and how heads split and recombine. Here is a correct, readable implementation plus the follow-ups.

BPE powers GPT-style tokenizers, and coding it shows you grasp how subword vocabularies get constructed rather than just knowing they exist. What interviewers watch for is the repeated merge of the most common adjacent pair.

LayerNorm appears in every transformer, and coding it verifies you know what it normalizes over and why. Interviewers look for normalization across the feature dimension per token, the learnable scale and shift, and the RMSNorm simplification modern LLMs took up.

Coding the KV cache shows you understand why decode is efficient: you append the new token's K/V and reuse the rest rather than recomputing. Interviewers look for the append-and-attend-over-cache logic. The implementation follows.

Coding Adam shows you know what .step() actually does: momentum combined with per-parameter adaptive rates and bias correction. What interviewers watch for is the two moments plus the bias-correction step.

Single-head attention shows up often; multi-head layers on the split-into-heads, attend-per-head, concatenate pattern. What gets scored is the reshape into heads plus a sharp explanation of why several heads help. Here is the implementation.

Every chat model's 'temperature' and 'top_p' knobs come down to a few lines of logit surgery. Implementing them proves you understand decoding rather than just calling it. Here is the from-scratch version.

Many state-of-the-art training runs hold a shadow copy of the weights that averages the trajectory, then serve that rather than the final step. It's a few lines and a genuine accuracy gain. Here it is.

Nearly every modern training run relies on warmup-then-cosine, and getting it wrong destabilizes early training or squanders the tail. It's a closed-form function of the step. Here is the implementation and the why.

The classic multi-round build screen: a plain key-value store that gains new requirements at each stage (TTL, transactions, scans). The signal is not stage 1, it is whether your code takes on stage 4 without a rewrite. Here is how to design for it.

OpenAI's signature build screen: a credit ledger that issues GPU-hours, spends them, and lets unused grants lapse in issue order. The trap is charging against the right grant first. Here is the FIFO-by-expiry design that holds up through every follow-up.

A classic build screen: a key-value store whose writes inside a transaction can be committed or discarded, with transactions that nest. The trap is mutating the base store directly. Here is the overlay-stack design that keeps rollback O(1).

A practical build screen: transform a stream of sampled call stacks into a flame tree, then report self-time against total-time per function. The trap is mixing up the two times. Here is how to aggregate the tree and rank the hot functions.

Most candidates start rewriting. The interviewer is grading whether you isolate pure logic from side effects before you touch a line, because that is the difference between a safe refactor and a silent behavior change.

Plain multi-head attention is rarely the whole exercise: the usual extension is to chain it into KV-cached decoding and GQA in one sitting. The locked answer walks the NumPy implementation, the equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.

Google's 'vibe coding' screen: the spec is intentionally vague and shifts mid-interview. The signal isn't the algorithm, it's whether you clarify before coding, structure for change, and keep tests green as the requirements move. Here is how to run that loop.

A build-it-yourself check on recurrent forward passes and backprop through time. What matters is the shared-weight recurrence, summing gradients over timesteps, and articulating the vanishing-gradient problem. The code follows.

A build-it-yourself check on gated recurrence. What matters is wiring the forget/input/output gates and candidate correctly, keeping cell state distinct from hidden state, and explaining why the additive cell path cures vanishing gradients. The code follows.

A build-it-yourself check on modern optimizers. What matters is the moment estimates, bias correction, and the single change that counts: AdamW decouples weight decay from the gradient rather than folding it into the loss. The code follows, plus why it wins.

A build-it-yourself check on how learning-rate schedules are actually computed. What matters is correct closed-form formulas, knowing when each schedule shines, and why decaying the rate aids convergence. Below is the implementation of the common ones.

A build-it-yourself check on how PyTorch actually works under the hood. What matters is assembling a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Below is a minimal engine.

A build-it-yourself check on subword tokenization. What matters is greedy longest-match encoding against a fixed vocabulary, the continuation-prefix convention, and how WordPiece diverges from BPE. The code follows.

A build-it-yourself check on distributed input pipelines. What matters is partitioning data across workers with no overlap and no gaps, epoch-consistent shuffling with a shared seed, and handling the uneven-last-batch problem. The code follows.

The classic recursive-descent exercise with an applied-AI twist: the JSON your model streams stays truncated mid-token for the whole generation. What matters is a clean strict parser plus a small repair layer, not a second parser. The code follows.

The most job-shaped coding screen in AI, ML, and GenAI engineering: fan out N LLM calls without melting the rate limit or losing the batch to one bad request. What matters is the retry policy, not the async syntax. Below is the version that passes.

The most-asked hands-on LLM exercise: a chat history that outgrows the context window. What gets scored is not the sliding window, it is the region eviction is never allowed to touch. Below is the version that passes.

Every RAG system packs a prompt, and the packing bug is always the same one: the input fits the window exactly, so the model has nowhere left to answer. The arithmetic here is what separates a candidate who has shipped from one who has read about it.

The lookup is three lines of linear algebra. What separates a cache from an incident is where the similarity threshold came from, and the one-token queries ('2023' vs '2024', 'not') that no threshold can catch because the embedding barely moves.

Asking a model which answer is better is one line of code. Getting a number you would let block a deploy takes position-bias control, a confidence interval, and a human-labeled set the judge is measured against. Here is the harness.

Every agent is a loop around this function. The candidates who fail it raise an exception on a bad tool call; the ones who pass hand the error back to the model as a tool result and let it fix itself on the next turn.

Writing the regexes takes ten minutes. The half of the question that separates candidates is the evaluation: a detector that blocks 'ignore the noise in the data' has shipped a bug to every analyst using your product. Here is the layered detector and the harness that proves it works.

The naive version splits on newlines and works right up until a TCP chunk lands mid-line. Then there is the error that arrives after a 200 OK, the user who closes the tab while you keep paying for tokens, and JSON you cannot parse until it closes.

Asking a model 'is this answer hallucinated?' barely beats a coin flip. This is the build: claim extraction, a judge forbidden from using world knowledge, a denominator that excludes hedges, and one model call per claim, which decides where the check can run at all.

Everyone can write the happy path. The screen is decided by the two branches most candidates skip: what the system does when retrieval finds nothing good, and what it does when the model cites a chunk you never sent.

A prompt is code, but most teams edit it in a dashboard with no history and then cannot explain last Tuesday's quality drop. This is the small piece of infrastructure that makes prompts reviewable, pinnable, and instantly reversible.

Self-supervision is how modern models pretrain on unlabeled data, the engine behind LLMs and modern vision. The signal is the pretext-task idea and the genuine difference between contrastive and masked-prediction objectives.

Multimodal is now table stakes, and this tests whether you understand how images and text reach a shared model. The signal is CLIP's contrastive alignment and how modern VLMs feed image features into an LLM's token space.

Voice is a major modality and a common applied-AI surface. The signal is the audio-to-text pipeline, why Whisper is resilient, and the cumulative latency budget that can make or break a real-time voice agent.

The learning rate is the single hyperparameter that can NaN your run at step 50 or stall it indefinitely. Modern training schedules it rather than pinning it to one value. Interviewers want you to know precisely what warmup and decay each give you. Here is the answer.

These four quantities sit under cross-entropy loss, decision-tree splits, distillation, and the KL penalty in RLHF. What matters is deriving them from one another and pointing to precisely where each turns up in a real training loop.

Contrastive learning is how modern embeddings (CLIP, sentence encoders, SimCLR) are actually trained. What matters is the pull-positives-push-negatives objective, the InfoNCE loss, and why the number and hardness of negatives makes or breaks quality.

RL sits under RLHF, robotics, and recommendation, and interviewers want the core framing. What matters is the agent-environment-reward loop and the three things that make it harder than supervised learning. Here is the answer.

Classic bias-variance predicts that larger models eventually overfit, but deep nets keep improving even after they memorize the data. What shows depth is explaining the second descent and why over-parameterized models generalize. Here is the answer.

'The model won't train' comes down to a short list of usual suspects, each with a distinctive loss-curve signature. The shape of the curve names the bug before you touch a single hyperparameter. Here is how to read it.

The notion that models, like students, learn better starting from easy examples is intuitive, and it sometimes works and sometimes does nothing. Knowing when it pays off is the real signal. Here is the honest answer.

Most teams run human eval as a vibe check with ten examples and a 1-5 slider. The signal is treating it as a designed experiment: pairwise, blinded, powered, and agreement-measured. Here is the protocol that survives scrutiny.

Accuracy against a held-out label is the wrong instrument for a system with no single right answer, an open failure space, and a metric that is itself a model. Here is what breaks, what survives, and what replaces it.

An eval suite that never varies the image will happily pass a model that has stopped looking at it. Start with the modality ablation, then pick metrics by what the system actually produces. Here is the full stack.

Anyone can say 'filter and dedup.' The signal is the funnel arranged by cost, the MinHash/LSH banding math, and recognizing that the shuffle across billions of documents is what actually runs up the bill, plus the benchmark decontamination people forget until their eval numbers get challenged.

Moderation is a multi-stage classification problem with harsh tradeoffs: false negatives cause real harm, false positives silence legitimate users, and the adversary keeps adapting. What interviewers reward: the tiered pipeline, per-severity precision/recall calibration, and human-in-the-loop. This lays out the design.

The moment a company runs LLMs in more than one place, it needs a gateway. The signal is the cross-cutting concerns (cost, reliability, observability, governance) that a gateway consolidates, not merely 'proxy the API.' Here is the design.

The canonical AI system-design question. The signal is covering both the model lifecycle and the serving stack at real scale, without rambling. Most candidates design only half and lose the points.

The modern agentic system-design question. The signal is the plan, search, read, synthesize, verify loop with per-claim citations, plus keeping cost and latency bounded. The hard parts are not the writing.

Cross-session memory is what makes an assistant feel personal, and it is mostly a retrieval and state-management problem rather than a bigger context window. The signal is the tiered architecture plus what to store, forget, and protect.

Limited GPUs, dozens of models, and every team demanding low latency for little money. The signal is whether you can shape that into a single governed serving fleet: continuous batching, KV cache, per-tenant quotas, and cost you can genuinely attribute.

Summarize, draft, and triage, all across the most sensitive PII a person holds. The signal is grounding in the real thread, matching the user's voice, and a firm human-in-the-loop rule on anything that gets sent.

A GPU-heavy generative system where sampling steps drive the bill and a single bad image becomes a headline. The signal is the diffusion serving pipeline, the cost levers on per-image GPU time, and a required two-sided safety layer.

Transcribing in real time at scale probes streaming ASR, latency budgets, and GPU fleet management under heavy concurrency. The signal is chunked streaming inference with partial results, not batch transcription. Here is the design.

A voice assistant links STT, an LLM, and TTS under a punishing latency budget where each stage piles on delay. The signal is streaming and pipelining the stages plus turn-taking, not three blocking calls. Here is the design.

A GPU-heavy generative-audio system with text/genre/lyric conditioning, minutes-long coherence, and copyright constraints. The signal is the generation pipeline plus the async, cost, and safety layer that gets it shippable.

Video generation is image generation plus the harsh constraint of temporal consistency, and it is the most GPU-intensive generative task around. The signal is diffusion over spacetime plus a serving design that holds up under multi-minute jobs.

Two models in a chain (ASR then summarization), a long-context problem buried inside, and a faithfulness bar where a fabricated action item carries real consequences. The signal is the pipeline plus how you keep a 3-hour transcript honest at thousands a day.

A Devin-style agent that carries an issue all the way to a merged PR is the hardest kind to make reliable, since every step can fail and errors pile up. Here is the architecture and the loop that keeps it honest.

Exact-match caching does little when no two prompts match. Semantic caching reuses answers across similar queries, and its entire danger is handing back a near-match that is subtly wrong. Here is how to build it safely.

GPUs are scarce and costly, so multi-region HA for LLMs is not simply web-app HA with bigger machines. Capacity, routing, and failover all bend around the GPU constraint. Here is the design.

When every team evaluates prompts in one-off notebooks, an org ships regressions and argues about vibes. A shared eval platform turns quality into something measurable and comparable. Here is what it must provide.

When prompts live inside source code, every wording tweak becomes a full deploy and nobody knows which prompt is running. The fix is to treat prompts as managed, versioned configuration. Here is the platform.

An agent that clicks through a real website is slow, brittle, and a single wrong click away from a costly mistake. The architecture comes down to perception, action, and guardrails around irreversible steps. Here is how to build it.

RLHF and evals can only match the preference data behind them, and that data comes from humans whose quality swings wildly. The platform that yields trustworthy labels is a serious system in its own right. Here is its design.

Inline completion lives or dies on tail latency: a suggestion that lands after the developer keeps typing is useless. Learn how to assemble the context window, hide model latency behind speculation and caching, and keep a tight feedback loop on acceptance rate.

Summarizing a 200-page contract is not one LLM call: it is chunking, hierarchical reduction, and a faithfulness check so you never invent facts. Learn the map-reduce pattern, when long-context wins out, and how to evaluate summaries at scale.

Moderating billions of items a day with an LLM on each one is too slow and too expensive. Learn the tiered funnel (cheap classifier then LLM then human), how to tune thresholds for precision versus recall, and how to stay ahead of adversaries.

Personalizing an LLM is a retrieval and memory problem, not a per-user fine-tune. Learn how to assemble user context at request time, keep long-term memory from bloating the prompt, and respect privacy and the right to be forgotten.

A writing assistant has to feel instant as the cursor moves through a live document. Learn how to separate fast deterministic checks from LLM rewrites, stream suggestions, and keep edits stable so the text does not flicker while the user types.

Auto-resolving support tickets means grounding answers in your knowledge base, safely calling real APIs (refunds, account changes), and knowing when to hand off to a human. Learn the agent loop, the guardrails, and how to measure resolution without eroding trust.

Agents that loop for minutes, invoke tools, and spawn sub-tasks demand durable execution rather than one request handler. See how to checkpoint state, recover after failures, cap runaway loops, and trace every move a non-deterministic agent makes.

Running experiments on LLM features is tough because outputs are open-ended and quality is fuzzy. See how to assign traffic, choose metrics beyond engagement, tame variance from non-determinism, and dodge the traps that let a winning variant lose once it ships.

SSE versus WebSocket is the easy half. The half that separates candidates: a buffer-everything output filter throws away the time-to-first-token you paid a GPU to deliver, and you cannot send an HTTP error after the 200 has already flushed.

Every AI system spends quality to buy speed, whether or not the team admits it. What scores is knowing the exchange rate for each knob, taking the free wins first, and having already decided what you will sell when the system saturates.

The root cause is not a weak system prompt. A model trained to be agreeable is dangerous to a user in crisis, because the objective and the safety requirement point in different directions. The fix has to live outside the model. Here is what that architecture looks like.

A classifier trained on one culture's annotations does not generalize to another culture's speech, and the aggregate metric hides it. The signal is recognizing that the ground truth itself is the bug, then designing the policy and eval stack around that.

Every AI product that runs generated code (notebooks, agents, code interpreters) ends up building this, and the interview turns on two numbers: how strong your isolation is and how fast a session starts. The answer is microVMs plus snapshot restore, and knowing why the VM boot is not the slow part.

At thousand-GPU scale, hardware failure is routine rather than rare, and a run that cannot resume burns weeks. The signal is checkpointing strategy, deterministic resume, and holding lost work to a minimum. Here is the fault-tolerant design.

You cannot improve an LLM app you cannot see into, and LLM observability is not service monitoring. What counts is tracing multi-step chains, capturing inputs/outputs/tokens/cost, and online quality signals, not just latency and errors.

Caching is one of the biggest LLM cost levers, but 'cache the response' is naive for a non-deterministic system. What counts is telling prompt (prefix) caching apart from semantic caching and knowing when each is safe.

Routing each request to the right model is one of the biggest LLM cost/latency levers in production. What counts is matching query difficulty to model capability and knowing when the cascade pattern beats a classifier.

AI systems get expensive fast, and 'the GPU bill is huge' means nothing you can act on without attribution. What matters is breaking spend down by model, feature, team, and request, then naming the levers. Here is the answer.

Editing a prompt is a code change that ships without a compiler or a unit test by default, so quality regressions slip out invisibly. Treating prompts and models as testable artifacts is what divides a toy from a product. Here is the harness.

A green build says nothing about model quality. The signal is an eval harness that runs deterministically in CI, checks against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline pairs a versioned dataset, layered scorers, and a CI gate. Here is the architecture.

Production usage can become your best source of training data, or a self-reinforcing trap. What matters is knowing how to capture clean feedback and how to break the loops that quietly corrupt the model.

A provider swap does not fail with a 500. It fails by staying up and quietly getting worse. The signal is knowing what actually changes under you on a model swap, and the shadow-plus-staged-ramp protocol that catches it before your users do.

Your eval set is clean, well-punctuated, and phrased the way you think about the problem. Your users are not. The move that turns this from a vibe check into an engineering artifact is measuring invariance rather than accuracy, and gating on the number it produces.

Migrating models is a project you plan. Being migrated is a Tuesday. When the weights move under a stable API and your prompts were tuned against the old ones, every dashboard stays green while quality quietly rots.

Interviewers here want concrete figures, not 'grab a bigger GPU.' Weight memory stays constant, the KV cache scales with traffic, and the order you pull levers in settles the outcome. This walks through the napkin math and the serving stack.

The interviewer is checking that you understand how a model too large for any one GPU still gets trained, and can run the optimizer-state memory math that justifies sharding. This covers the parallelism taxonomy and the 16-bytes-per-parameter figure.

Quantization is the opening move for shrinking and accelerating models, and the interviewer expects more than 'use fewer bits.' What they grade is whether you know what each precision level gains you, why outliers wreck naive quantization, and how you demonstrate quality survived. This lays that out.

A staple at hardware-aware teams. What they look for is grasping that attention's cost is memory traffic rather than FLOPs, and that FlashAttention is an exact, IO-aware reordering rather than an approximation. This is the answer that shows you reason about the memory hierarchy.

The KV cache is what makes LLM serving hard, and the interviewer wants the mechanics: what it holds, why it caps concurrency, and the tricks that shrink it. This is the answer that shows you grasp decode-time economics.

Decode runs sequentially and is memory-bound, so latency tricks count. What they grade is whether you can explain speculative decoding's draft-and-verify mechanism (and why it stays exact) along with the other levers and when each fits. This is the latency toolkit.

Mixed precision is routine at scale, and the interviewer wants the numerics: why FP16 needs loss scaling, why BF16 mostly does not, and what remains in FP32. What they grade is whether you understand dynamic range vs precision. This is that answer.

A hardware-literacy question, particularly at NVIDIA and the labs. What they grade is whether you understand throughput vs latency hardware, the memory hierarchy, and why matrix multiplication maps onto GPUs and TPUs. This is the architecture-aware answer.

Kernel-level questions separate people who have written CUDA from people who have read about it. The signal is commanding the SIMT execution model and the three classic throughput killers, with a concrete fix for each. Here is the low-level answer.

Continuous batching is the biggest single throughput lever in modern LLM serving. The signal is explaining why static batching leaves the GPU idle on variable-length generation, and why PagedAttention is what makes the fix practical.

Picking GPUs is a genuine applied decision, and 'get the biggest one' misses the point. The signal is knowing which spec binds your workload (VRAM, bandwidth, interconnect) and the train-versus-serve difference that flips the answer.

Under load, a shared inference service has to decide whose request runs now. The signal is queuing with priorities, backpressure, and fairness wired into batching, not first-come-first-served until the service falls over.

FSDP and ZeRO let you train models too large for one GPU without splitting the compute. The signal is the ZeRO stages (what gets sharded at each level) and the communication you pay for it. Here is the answer.

Distributed training is bottlenecked by GPU-to-GPU communication, and these collectives are how the data travels. The signal is what each one does and which parallelism strategy relies on it. Here is the answer.

Activations, not just weights, can dominate training memory, and gradient checkpointing is the standard remedy. The signal is the precise trade: recompute activations in the backward pass rather than storing them. Here is the answer.

When a model is too large for one GPU you split the model itself, not just the data. The signal is telling tensor parallelism (split within a layer) apart from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.

LLM serving has its own metrics, and a single latency number hides the real bottleneck. The signal is separating prefill from decode and treating GPU utilization as a clue, not a verdict. Here is the diagnostic toolkit.

LLM inference has two phases with opposite resource profiles, and co-locating them lets a long prompt stall everyone else's tokens. The signal is knowing why prefill and decode fight, and what separating them costs. Here is the answer.

Batch size is a training knob whose effects on speed, memory, and generalization are non-obvious. The signal is the large-batch tradeoffs and the learning-rate-scaling and gradient-accumulation tricks that make scaling actually work.

OOM is the wall you hit most often in deep learning, and 'buy a bigger GPU' is the weakest reply. What counts is naming the memory consumers, knowing which one dominates, and pairing the right lever with it.

nvidia-smi reading 100% can mask the fact that you are tapping only a fraction of the hardware's real compute. What matters is MFU (useful FLOPs vs peak) and the gap between 'the GPU is busy' and 'the GPU is efficient'.

Past FP16/BF16, FP8 sits on the next rung of the precision ladder, and the newest tensor cores compute on it natively. What counts is knowing the two formats, what FP8 really buys, and why dynamic range demands careful scaling.

At long context the KV cache, not the weights, dominates GPU memory and decode bandwidth. What counts is spotting it as the binding constraint and naming the levers that shrink it without wrecking quality.

CPU-based autoscaling that suits a web tier quietly breaks on GPU inference: the signal is wrong, and replicas need minutes to warm. The interviewer wants the signals you genuinely scale on and how you mask the cold start.

Paying for accelerators that sit idle half the time is the most common waste in ML training, and the reflex to add more GPUs only makes it worse. The interviewer wants the profiling discipline that pinpoints what is starving them.

The counterintuitive reality of LLM serving: token generation is capped by how quickly you can read weights out of memory, not by arithmetic. Once that clicks, the entire optimization menu follows from one number.

A 256-GPU job that freezes with no error at 3am is a special kind of pain. The causes come from a short, recurring list. Here is the systematic way to identify which one hit you.

Users experience the p99, not the median, and the tail is where serving systems quietly break. The causes are queuing and batching effects, not a slow model. Here is how to flatten it.

A serving box steady on short prompts falls over the instant a 30k-token request arrives, because KV-cache memory scales with sequence length times batch. Here is how to bound it without crashing.

At billion-parameter scale, training can be running smoothly and then the loss jumps and never returns, burning a fortune in compute. The causes and the playbook are familiar to the few who've done it. Here it is.

Launching an LLM service on a guessed GPU count is how you get paged on day one. Capacity planning for LLMs differs from web services because tokens, not requests, are the unit. Here is how to size it.

One long prompt can freeze every other user's token stream for hundreds of milliseconds. Chunked prefill carves that prompt up so decode keeps flowing. Here is the mechanism and the knob that controls it.

A shared system prompt is re-prefilled on every request unless the server remembers it. Prefix caching skips that work, but only when the blocks align exactly. Here is the hashing and eviction machinery underneath.

Round-robin routing throws away a feature that doubles throughput: cache locality. Designing the router means picking a replica by KV state, not just least-loaded. Here is the two-layer design.

Three batching strategies, three very different latency profiles. Choosing wrong leaves throughput or tail latency on the floor. Here is what each one costs and when to use it.

FP8 serving is fast and usually fine, until it quietly degrades on the one workload your benchmark did not cover. What counts is knowing what FP8 breaks and how to validate it. Here is the recipe.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to strip it out.

Autoscaling reacts in minutes, but a burst lands in seconds, and the gap is where your tail latency dies. Absorbing bursts is about buffers and shedding, not just adding replicas. Here is the playbook.

Full activation checkpointing saves memory but costs a flat 30% extra compute. Selective recomputation wins most of that back by recomputing only the cheap, memory-heavy operations. Here is how to pick what to recompute.

RLHF with PPO is not one model training, it is four models in the loop at once, three of them on the GPU during every step. The memory math and the generation bottleneck are what catch people out.

FP8 can nearly double training throughput over BF16, but with only a few mantissa bits the numerics leave no slack. Per-tensor scaling and a selective recipe are what get it to converge.

Data, tensor, and pipeline parallelism each leave a single sequence's activations on one device, so training past 200k tokens hits a wall none of them can clear. The fourth axis shards the sequence itself, and the interview turns on the communication math.

When each request thinks for 30,000 tokens, serving swings from compute-bound prefill to memory-bound decode, and the KV cache turns into the resource you genuinely schedule. The levers that governed chat traffic stop being the ones that count.

Raw tokens per second is gameable: crank the batch size and the dashboard looks great while every request misses its latency target. Goodput is the throughput that actually meets your SLOs, and it is the number you size and autoscale on.

An image is not one input, it is a large prompt. Once you internalize that image tokens dominate prefill, the cost levers stop being model choices and start being resolution, tiling, and cache placement.

There is no one-shot patch for prompt injection, and answering 'sanitize the input' loses the round. What lands is defense in depth: privilege boundaries, handling retrieved content as data rather than instructions, and a human gate on irreversible actions. Here is the layered answer.

Getting an LLM feature to production safely is an evaluation problem first and a model problem second. What lands is a layered eval-plus-guardrail design with honest, segmented metrics rather than a single 'safety classifier.' Here is how to measure and defend each failure mode.

LLM endpoints are unusually exposed: every call can spend real money and consume scarce GPU capacity, so abuse is a security and a financial problem at once. What lands is layered controls on tokens and cost, not just request count. Here is the defense.

Red teaming is how you uncover an AI system's failures before users or attackers do. What lands is a structured program covering the real attack surface (jailbreaks, harmful content, leakage, bias) and routing every finding back into defenses.

Copyright is among the largest unresolved risks in generative AI, on both the training and output side. What lands is naming both exposures and the engineering mitigations for each rather than offering a legal opinion. Here is the engineer's framing.

As generated output grows indistinguishable from real, knowing what is AI-made matters for trust, misinformation, and regulation. What lands is the split between in-content watermarks and attached provenance metadata, plus the fact that detection is an arms race. Here is the answer.

'Add guardrails' says nothing concrete. The strong answer names the specific input and output checks, the mechanisms that enforce them, the fail-safe behavior when one trips, and the honest admission that they are imperfect. Here is the implementation answer.

Attackers coax models into spilling the hidden system prompt, exposing IP, business logic, or worse, embedded secrets. The signal is treating the prompt as non-secret and defending in layers, rather than betting on 'never reveal your instructions.'

Toxicity detection looks like plain text classification but is nothing of the sort: context flips labels, adversaries evolve weekly, and naive models tag dialects as hate. The signal is naming those failure modes and building the human-in-the-loop system around them.

Safety is not a single number. The candidates who pass name the axes, run benchmarks as a gate, and then explain why benchmarks by themselves certify nothing. Here is the framing interviewers score highest.

With direct injection the user attacks the prompt. With indirect injection the payload sits inside a page, doc, or email the model reads. The signal is recognizing that retrieved and tool data is untrusted and can hijack the model. Here is the answer.

Large models memorize and can be coaxed into emitting training examples verbatim, a genuine privacy and copyright liability. The defenses reach across data, training, and output. Here is the layered answer.

An attacker doesn't have to breach your LLM to hurt you, only make it do expensive work. A few crafted prompts can pin your GPUs and run up the bill. The defense is not a bigger rate limit.

When an agent holds private data, reads untrusted text, and can talk to the outside, injected instructions can steal data. This 'lethal trifecta' is the defining agent vulnerability. The fix is architectural, not a better prompt.

A request that gets refused in one message often lands when broken across ten. The signal is understanding that conversational state is part of the attack surface, and that turn-by-turn classifiers see only fragments.

base64, leetspeak, invisible Unicode, and foreign scripts each let a payload look like gibberish to your filter yet read as a clear instruction to the model. The signal is knowing why the model decodes what the classifier cannot.

Roleplay, obfuscation, optimization-based suffixes, and multi-turn escalation are distinct attack classes that call for distinct defenses. The signal is organizing the space and pairing each class with a control instead of hoping one filter covers all.

Gibberish-looking token strings tacked onto a prompt can reliably break refusals, and they carry over between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.

An agent that trusts its tools and its own memory can be redirected by one poisoned record that resurfaces turns or sessions later. The signal is treating persistent state as an attack surface, not just the live prompt.

A hijacked agent does not need a 'send email' tool to leak secrets. A rendered markdown image, a URL parameter, or a DNS lookup will do. The signal is enumerating the covert channels and locking down egress, not just tools.

C2PA cryptographically signs where media originated, but a signature you can peel off is not the same as a watermark you cannot. The signal is knowing what provenance proves, what it does not, and why you pair it with watermarking. Here is the answer.

The EU AI Act hands different duties to high-risk deployers, GPAI model providers, and limited-risk systems. The signal is naming the specific obligations per tier, not merely reciting that risk tiers exist. Here is the answer.

Rate limits stop a user who costs you too much. They do nothing about a user doing something you forbid. The enforcement system is a different design, and interviewers can tell within a minute whether you have built one.

Your guardrail reads text. The attack is not in the text. That single sentence is the whole vulnerability, and the red team you build from it looks nothing like the one you already have.

The technique works and everybody knows it works. The interview is not testing whether you can generate the training set; it is testing whether you know what you are agreeing to when you do.

The point is not whether you failed; everyone has. Panels are probing for ownership, debugging discipline, and candor under stress. Here is the frame that turns a failure story into a hire signal, plus the pitfalls that turn it into a flag.

Every loop includes this question, and most answers slip one of two ways: too combative or too much of a doormat. The signal is evidence-based persuasion plus real disagree-and-commit. Here is the arc that connects and the traps that sink it.

This is an entire interview round (OpenAI's project deep dive, Amazon's Tech Talk), not small talk. It is where they check whether you genuinely understand the work you claim. The signal is depth, your specific contribution, and the tradeoffs you can defend. Here is how to prepare and deliver it.

AI moves fast enough that learning speed is itself a core competency, and several companies ask about it directly. The signal is a concrete method for ramping fast plus a real, recent example of putting something new to use. Here is how to show learning agility rather than merely claim it.

At the frontier labs this is a genuine gate, not a pleasantry. They turn away strong engineers who cannot articulate real motivation. The signal is specific, honest alignment between what you want to build and what this company actually does.

A depth probe dressed up as a story. They want to watch you reason through genuine difficulty and keep going when pushed three levels down. The signal is a real hard problem, a systematic attack, and a root cause you own cold.

AI features are probabilistic, yet stakeholders hear 'it works.' Setting honest expectations without draining enthusiasm is a core applied-AI skill. Here is how to frame reliability so trust holds through the first mistake.

AI features break in public, sometimes embarrassingly. How you respond to the customer, not only the bug, is what this question actually tests. Here is the recovery that rebuilds trust.

Every AI team feels the tug between velocity and doing it responsibly. How you work through that tension, with judgment rather than dogma in either direction, is what this question screens. Here is the answer.

Chasing every new model is as harmful as ignoring them all. Interviewers want a repeatable filter for signal versus hype, plus the discipline to test on your own problem. Here is that filter.

Ethical pushback probes integrity and judgment under pressure. Interviewers look for principled, specific action, not a generic statement of values. Here is the arc that signals real backbone.

The opener in half of all Applied AI screens, and most candidates answer it with a definition nobody scores. The distinction that actually earns points is about where you start and what your bottleneck turns out to be.

The number is bait. Interviewers use this to see whether you treat a benchmark delta as a decision or as a claim to be audited, and whether you can price latency, cost, and on-call burden against accuracy. Here is the answer that scores.

The classic closer, and most candidates answer it with a horoscope or a list of tool names. The signal is whether you hold an actual view, can justify it from what is already happening, and know what it means for the job.

Most candidates answer this with a speech about tradeoffs. The ones who get hired describe a mechanism: the teams that ship fastest in AI are not the ones with fewer guardrails, they are the ones whose guardrails make a mistake cheap.

The ordering is the answer, and one move separates the people who have shipped from the people who have prototyped: where evaluation sits in the sequence. Most candidates put it in the wrong place and lose the question in the first sentence.