The ML System Design Interview: A Framework That Works for Any Prompt

There is one structure that works for every ML system design prompt, and it is the same whether you are asked to design feed ranking, fraud detection, search, or ads. Start from the business metric, frame it as an ML problem, then walk down a fixed spine: data and labels, features, model, training pipeline, serving and latency, offline and online evaluation, and monitoring for drift. The order is the point. Metric first, architecture last. Candidates who jump to a model in the first two minutes fail this round more than any other mistake, because they end up building a well-engineered system for an objective nobody agreed on. Reported guidance from interview prep sources puts raw modeling knowledge at roughly a fifth of the score. The rest is structure, judgment, and whether you close the loop from serving back to retraining.

The framework, one pass

Every prompt is the same skeleton with different constraints bolted on. Learn the spine once and you stop memorizing per-problem playbooks.

Clarify the business metric. Spend the first five minutes here. Who is the user, what action are you trying to move, and how does the business measure success. Not "predict clicks" but "increase time spent without tanking creator diversity" or "cut fraud loss without raising false declines past a threshold." Get the interviewer to agree on the metric before you draw anything.

Frame it as an ML problem. Turn the metric into a task: binary classification, ranking, regression, retrieval. Name the label. Name the offline proxy you will optimize and say where it diverges from the online metric, because that gap is what gets probed. This is the hinge of the whole interview.

Data and labels. Where does training data come from, how is it labeled, how fresh is it, and what biases does it carry. Labels are usually harder than the model. Delayed labels, scarce positives, and feedback loops from the system's own decisions are where senior candidates earn points.

Features. Move from raw data to model inputs. Which signals, computed how often, and served from where. Call out leakage: a feature that encodes the label or uses information not available at prediction time will look great offline and collapse in production.

Model choice. Pick something defensible for the task and latency budget, and justify it in one or two sentences. A gradient-boosted tree that scores in a millisecond can beat a deep model you cannot serve in budget. State the trade-off out loud.

Training pipeline. How the model gets built and rebuilt: how often you retrain, how you version data and models, how you validate before promotion. This is the MLOps half of the round, and skipping it reads as someone who has trained models in notebooks but never shipped one.

Serving and latency. How predictions reach the product inside a real budget. Batch precompute versus real-time scoring, feature stores, caching, and a fallback for when the model is slow or down. For real-time prompts this is a scored dimension, not a footnote.

Offline and online evaluation. Offline metric to gate promotion, online A/B test against the business metric to confirm it moved. Name both and the gap between them.

Monitoring and drift. Track feature and prediction distributions, watch the online metric, and trigger retraining when drift crosses a threshold. The loop back to data closes the system.

If you want the practice bank organized around exactly these stages, the ML system design questions map to them, and the must-know set is the fastest way to find your weak stage before a loop. For the general distributed-systems patterns this round assumes, the system design questions are the base layer.

Applying it: feed ranking

Metric first. "Increase engagement" is too vague to design against, so pin it: daily time spent, or meaningful interactions, with a guardrail on diversity and integrity so the feed does not collapse into clickbait. Frame as ranking, and immediately reach for the standard multi-stage pipeline, because you cannot score millions of candidates per request in budget.

Candidate generation retrieves a few thousand items cheaply, often with embeddings and approximate nearest-neighbor lookup. Ranking scores those with a heavier model on rich features. Re-ranking applies diversity and business rules at the top. Labels come from implicit feedback: clicks, dwell, likes, which are plentiful but biased toward whatever the current system already shows. Say that out loud, because position bias and the feedback loop are the held-back follow-up.

Features split into user, item, and context, many precomputed and read from a feature store at request time. Model is typically a deep ranker predicting multiple engagement events, with the serving path built around a tight latency budget. Offline you optimize a ranking metric like NDCG; online you A/B test against time spent and the integrity guardrails. Monitoring watches for distribution shift as content and user behavior move, and retraining runs frequently because feeds go stale fast. The data-and-labels depth here overlaps heavily with the ML and data science questions.

Applying it: fraud detection

Same spine, opposite constraints. The metric is dollars of fraud loss avoided against a hard cap on false declines, because blocking good customers has a real cost. Frame as binary classification with extreme class imbalance, often well under one percent positives, under a hard real-time latency budget measured in tens of milliseconds inline with the transaction.

Labels are the crux. Confirmed fraud arrives late through chargebacks and is scarce, and your training data is biased by what the current system already blocks, so you never see the outcome of transactions you declined. Address it directly: a small holdout of unblocked transactions to measure what you would have missed, and honesty that today's decisions cannot be scored today. Handle the imbalance with cost-sensitive learning or sampling rather than optimizing plain accuracy, which a "predict everything legitimate" model would ace.

Features lean on behavioral, device, merchant, and graph signals. A common production shape is a rules engine for hard blocks plus an ML score for the gray zone, which keeps the model's latency budget honest. Offline you report PR-AUC or precision at a fixed recall, never raw accuracy; online you watch fraud loss and false-decline rate. Monitoring has to assume an adversary who adapts, so drift here is not passive: attackers probe and shift, and your retraining cadence is part of the defense. This closing-the-loop instinct is the same one that carries the constraint-driven AI system design round.

Why the order beats the details

Two candidates can name the same models and only one passes. The difference is almost always sequence and completeness. The one who states the metric, defines the label, connects offline to online, and closes the loop back to retraining looks like someone who has operated a model in production. The one who opens with architecture and treats evaluation and monitoring as an afterthought looks like someone who has only trained them. In 2026 the rubric weights that operational maturity heavily, so a plain design that runs the full spine beats a slick diagram that stops at the model.

The one-line version

Walk every ML design prompt down the same spine: business metric, ML framing, data and labels, features, model, training, serving, evaluation, monitoring. Feed ranking and fraud detection are the same framework under different constraints. Say the metric and the label out loud before you draw a box, and finish the loop back to retraining. Do that and you look like someone who has shipped, which is the whole point of the round.