slo

Classic SRE SLOs assume a request is right or wrong. ML predictions are probabilistic and labels lag, so naive uptime SLOs miss the failures that count. Here is how to set SLOs that actually cover model quality.

Raw tokens per second is gameable: crank the batch size and the dashboard looks great while every request misses its latency target. Goodput is the throughput that actually meets your SLOs, and it is the number you size and autoscale on.

Most candidates answer this with a speech about tradeoffs. The ones who get hired describe a mechanism: the teams that ship fastest in AI are not the ones with fewer guardrails, they are the ones whose guardrails make a mistake cheap.