Google DeepMind AI & ML Engineer interview questions

Push any proxy hard enough and the model discovers the exploit, not the goal. Sycophancy, padding, and fake citations are reward hacking. Here is why it happens and the controls that actually hold.

CoT is not a blanket upgrade. On some tasks it does nothing or backfires, and the reason points to what to try instead. Here is the escalation ladder past 'let's think step by step'.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field relies on a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

Push PPO hard enough and true quality peaks then declines while the reward keeps climbing. The signal is knowing the Gold-vs-proxy gap, the KL budget that bounds it, and how you actually measure when to stop.

GRPO drops the value model and estimates advantage from a group of samples per prompt. The signal is the exact mechanics, why the group baseline works, and the normalization traps that silently bias training.

Swap a learned reward model for a checker that returns right-or-wrong, and reward hacking largely vanishes. The signal is why verifiable rewards beat learned ones for math and code, and where they break.

The proportions of web, code, books, and math in pretraining quietly set downstream skills. The signal is knowing how mixtures get chosen, why upsampling helps, and how methods like DoReMi automate it.

When a benchmark leaks into pretraining, the score measures memorization rather than ability. The signal is the detection methods (n-gram overlap, canaries, perturbation tests) and why fresh held-out evals matter.

A reward model converts pairwise preferences into a scalar signal RLHF can optimize. The signal is the Bradley-Terry loss, the base-model and head choices, and how you validate it before trusting it.

By swapping costly human labels for an LLM's preferences, RLAIF scales cheaply but carries over the labeler model's biases. What matters is how AI feedback gets gathered and where it silently breaks down.

MQA and GQA cut the KV cache by sharing key/value heads. MLA goes another way: compress K and V into a low-rank latent and cache that instead. What matters is recognizing it as a cache trick rather than a head-sharing trick, and why it preserves quality.

A model trained at 4K tokens can reach 128K without a full retrain. What matters is understanding why naive extrapolation breaks and how PI, NTK-aware scaling, and YaRN each rescale RoPE frequencies in different ways.

Speculative decoding accelerates generation, and with the right verification rule it leaves the output distribution untouched. What matters is understanding how draft-model, self-drafting (Medusa/EAGLE), and lookahead approaches differ in where the guesses originate, and which of them are still exact.

FP8 underpins modern H100-class training and serving. What matters is knowing the two FP8 variants, why a floating format beats INT8 on dynamic range, and where scaling still counts.

VLMs such as GPT-4V and LLaVA attach a vision encoder to a language model through a projector. The signal is the image-tokens-as-prefix design, the alignment training, and the resolution and hallucination failure modes.

Speech LLMs either transcribe to text or model audio straight away as discrete tokens. The signal is the semantic-vs-acoustic token split and why end-to-end audio models outperform ASR-plus-LLM pipelines on latency and prosody.

Four fusion families, and the choice between them is decided by one question most candidates never ask. Naming CLIP and LLaVA is table stakes; the score comes from knowing which one cannot reason and which one eats your context window.

The instinct is to blame the vision encoder. It is a token-budget problem, and the fix is to retrieve pages before you look at them. What separates a strong answer is knowing which question types page retrieval quietly cannot answer.

An emerging alternative to stuffing the window or building an index: keep the context outside the prompt and let the model write code to inspect it, recursing into the pieces. Knowing when it does not pay is most of the score.

Two defaults that every Llama-class model shares and almost no candidate can justify. One is a compute win that turned out to cost nothing in quality; the other is what makes a 60-layer stack converge at all.

Most candidates answer with the 2015 ResNet answer and stop. The senior answer treats the residual stream as a shared communication bus, and that reframing is what makes logit attribution, the logit lens, and activation steering possible at all.

Naming RLHF is not an answer. The frame that scores is outer versus inner alignment, why Goodhart breaks any proxy objective you optimize hard enough, and why a classifier sitting at the boundary can constrain a model's behavior without changing what it is trying to do.

Video breaks a VLM on arithmetic before it breaks it on vision: one frame per second of a ten-minute clip is hundreds of thousands of image tokens. Frame selection is not preprocessing, it is the architecture, and the strongest systems index the audio first.

A VLM is three parts and the interview is entirely about which ones you train. The staged recipe, why the vision encoder almost always stays frozen, and the silent failure where your model learns to answer from the text prior and never looks at the image.

Linear agents lock into one path and can't backtrack. Tree-of-Thoughts and LATS add search over reasoning paths. The signal is knowing what they buy, what they cost, and when a cheaper loop wins.

Final-answer accuracy conceals how an agent got there. The signal is trajectory-level metrics: tool-selection and argument accuracy, step efficiency, and matching against reference paths, plus when exact-match is the wrong yardstick.

Per-step accuracy looks fine, yet a 50-step task fails. The signal is grasping compounding error and the techniques (decomposition, verification, checkpointing) that keep long-horizon agents from collapsing.

Agents can critique and retry their own work. The signal is knowing how Reflexion's verbal feedback loop operates, when self-critique genuinely helps versus when it is theater, and what makes it real.

Screen-driving and browser agents break in ways chat agents never hit: stale DOM, pages that keep shifting, misplaced clicks. What interviewers watch for is the grounding and reliability methods that move a flaky demo toward something dependable.

RoPE is why modern LLMs extrapolate to longer contexts, and it works by rotation, not addition. Coding it shows you truly understand how position enters attention. Here is the implementation.

A build-it-yourself check on modern optimizers. What matters is the moment estimates, bias correction, and the single change that counts: AdamW decouples weight decay from the gradient rather than folding it into the loss. The code follows, plus why it wins.

A build-it-yourself check on how PyTorch actually works under the hood. What matters is assembling a computation graph during the forward pass, local derivatives per op, and a topological-order backward pass that accumulates gradients. Below is a minimal engine.

The notion that models, like students, learn better starting from easy examples is intuitive, and it sometimes works and sometimes does nothing. Knowing when it pays off is the real signal. Here is the honest answer.

Generative training breaks in distinctive ways: GANs collapse to a handful of outputs, diffusion samples come out noisy or blurry. The symptom points to which knob to turn. Here is the diagnosis.

Raw gradient saliency maps are noisy and saturate. Integrated gradients cures both with two axioms and a path integral, yet the baseline choice quietly decides the answer. Here is what a careful candidate explains.

An eval suite that never varies the image will happily pass a model that has stopped looking at it. Start with the modality ablation, then pick metrics by what the system actually produces. Here is the full stack.

An agent that clicks through a real website is slow, brittle, and a single wrong click away from a costly mistake. The architecture comes down to perception, action, and guardrails around irreversible steps. Here is how to build it.

'Just set a seed' misses the point. Reproducibility means pinning the environment, the data, the code, and the hardware-level nondeterminism together. Here is the full checklist and where it springs leaks.

No single parallelism dimension trains a 175B model by itself. The art is stacking three of them onto the right hardware so the expensive communication stays on the fast links. Here is how to size each axis.

MoE gives you more parameters for the same FLOPs, but the gating network, the load imbalance, and the all-to-all shuffle bring failure modes dense models never see. Here is the training playbook.

Data, tensor, and pipeline parallelism each leave a single sequence's activations on one device, so training past 200k tokens hits a wall none of them can clear. The fourth axis shards the sequence itself, and the interview turns on the communication math.

Large models memorize and can be coaxed into emitting training examples verbatim, a genuine privacy and copyright liability. The defenses reach across data, training, and output. Here is the layered answer.

Gibberish-looking token strings tacked onto a prompt can reliably break refusals, and they carry over between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.

Every AI team feels the tug between velocity and doing it responsibly. How you work through that tension, with judgment rather than dogma in either direction, is what this question screens. Here is the answer.