ml system design

With billions of candidate items and only tens of milliseconds to choose the next 10, a feed is a latency problem first. The interview probes the two-stage architecture, how you set the objective when engagement fights integrity, and the biases that silently corrupt your training labels.

A track runs three minutes, a session runs an hour, and a fresh release has zero plays on launch day. The interview centers on mixing collaborative filtering with audio content embeddings, handling cold start on both sides, and reading a skip as the loud negative it really is.

Training a pCTR model is the easy part. The challenge is proving a change helps before it reaches revenue, when the model runs inside an auction, the logs surface only ads that won, and a 1% calibration error costs real money. This is an eval question, not a model question.

Watch time is the label everyone optimizes and hardly anyone measures cleanly. You see minutes only for videos people chose to play, the distribution is savagely skewed, and the slot they saw it in shifted the number. The interview asks whether you can predict a biased label honestly.

The positive class runs about 1 in 1,000, your labels show up late and noisy, and the instant you ship a model the adversary starts probing it. This is the uncommon ML problem where the data actively fights back, so the design hinges on labels, latency, and enforcement cost as much as on the classifier.

This model's output is a price input, not a ranking. A miscalibrated CTR skews the auction bid, so you overpay or underdeliver. Throw in conversions that arrive days after the click and advertisers with no history, and calibration plus delayed feedback become the entire interview.

An ETA is a promise. The interview tests whether you model it as a point estimate (and apologize when it misses) or as a distribution where p90 lets you under-promise, and how you feed live traffic and completed trips back into the model.

Millions of landmarks exist, most with only a handful of photos, and the next photo could show something not in your catalog at all. A flat classifier collapses on the long tail and never says 'I don't know.' The interview is embeddings plus retrieval plus a confident refusal.

Fraud is well under 1% of transactions, the labels land weeks late as chargebacks, and the fraudsters deliberately adapt to your model. Optimizing accuracy hands you a model that approves everything. The interview is about dollars-at-risk thresholding, label delay, and a review queue with a fixed headcount.