drift

'The model got worse' can mean three very different things with three different fixes. Retrain a model that a pipeline bug actually broke and you just bake in garbage. Here is the order to triage it in.

Threshold alarms page you all weekend yet sleep through Monday's actual outage. What interviewers reward is stripping out seasonality first, matching the detector to the anomaly, and tuning against alert fatigue. Here is the answer that holds up in production.

Production drift monitoring frequently lives in the warehouse, not a fancy tool. Comparing two distributions in SQL with a metric like PSI is a few CTEs. Here is how to compute it.

Truth is not a label you can gather cheaply, and adversaries adapt as soon as you ship. A strong answer combines content, graph, and behavioral signals, places humans in the loop where precision matters, and treats adversarial drift as a permanent operating condition rather than a one-time training problem.

Monitoring an ML system goes past CPU and latency; the model can silently rot while the dashboard stays green. What matters is the four-layer taxonomy (operational, data, prediction, outcome) and using inputs as leading indicators because labels lag.

Waiting on labels to measure accuracy means learning a model broke a month after it did. Production monitoring must function before the truth shows up. These are the signals you track instead.

Fairness is a property of the model and the data it meets, so it drifts even when the weights never change. Treating it as a monitoring problem exposes three things a one-off audit never has to solve: noisy small slices, multiple comparisons, and labels that arrive months late.

Migrating models is a project you plan. Being migrated is a Tuesday. When the weights move under a stable API and your prompts were tuned against the old ones, every dashboard stays green while quality quietly rots.