Expert AI, ML & GenAI interview questions

GRPO drops the value model and estimates advantage from a group of samples per prompt. The signal is the exact mechanics, why the group baseline works, and the normalization traps that silently bias training.

MQA and GQA cut the KV cache by sharing key/value heads. MLA goes another way: compress K and V into a low-rank latent and cache that instead. What matters is recognizing it as a cache trick rather than a head-sharing trick, and why it preserves quality.

A model trained at 4K tokens can reach 128K without a full retrain. What matters is understanding why naive extrapolation breaks and how PI, NTK-aware scaling, and YaRN each rescale RoPE frequencies in different ways.

Speculative decoding accelerates generation, and with the right verification rule it leaves the output distribution untouched. What matters is understanding how draft-model, self-drafting (Medusa/EAGLE), and lookahead approaches differ in where the guesses originate, and which of them are still exact.

MoE trims compute but is awkward to serve: experts have to be sharded, tokens routed across devices, and batches balanced. What matters is the all-to-all communication and the load-imbalance problem, not the training story.

Grammar-constrained decoding seems like a solved problem until tokenization stops lining up with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing address it.

Agent PR volume scales with compute; human review capacity does not. Most candidates answer 'add an AI reviewer' and miss the real design problem: deciding which changes deserve human judgment at all. Interviewers want a risk-tiered pipeline you would trust in production.

Plain multi-head attention is rarely the whole exercise: the usual extension is to chain it into KV-cached decoding and GQA in one sitting. The locked answer walks the NumPy implementation, the equivalence check that proves your cache is correct, and the reshape bugs that silently corrupt attention.

At-least-once creates duplicates and at-most-once drops data; everyone asks for exactly-once yet few can explain how Kafka provides it. The signal is idempotent producers, transactional read-process-write, and the read-committed isolation that binds them together.

Agents that loop for minutes, invoke tools, and spawn sub-tasks demand durable execution rather than one request handler. See how to checkpoint state, recover after failures, cap runaway loops, and trace every move a non-deterministic agent makes.

Live speech-to-speech translation is a latency race across three models (recognize, translate, synthesize) as audio keeps streaming in. See the streaming pipeline, how to commit partial results without flip-flopping, and the tradeoff between latency and translation quality.

Every AI product that runs generated code (notebooks, agents, code interpreters) ends up building this, and the interview turns on two numbers: how strong your isolation is and how fast a session starts. The answer is microVMs plus snapshot restore, and knowing why the VM boot is not the slow part.

Round-robin routing throws away a feature that doubles throughput: cache locality. Designing the router means picking a replica by KV state, not just least-loaded. Here is the two-layer design.

No single parallelism dimension trains a 175B model by itself. The art is stacking three of them onto the right hardware so the expensive communication stays on the fast links. Here is how to size each axis.

MoE gives you more parameters for the same FLOPs, but the gating network, the load imbalance, and the all-to-all shuffle bring failure modes dense models never see. Here is the training playbook.

RLHF with PPO is not one model training, it is four models in the loop at once, three of them on the GPU during every step. The memory math and the generation bottleneck are what catch people out.

FP8 can nearly double training throughput over BF16, but with only a few mantissa bits the numerics leave no slack. Per-tensor scaling and a selective recipe are what get it to converge.

Gibberish-looking token strings tacked onto a prompt can reliably break refusals, and they carry over between models. The signal is explaining the gradient search that finds them and why output checks beat input pattern matching.