ads ranking

Ads ranking is where calibrated probabilities collide with tight latency and money. The signal is knowing that CTR has to be calibrated (not just ranked), the feature and serving design, and the auction context. Here is the design that goes past 'train a classifier.'

This asks you to design the scoreboard, not the player. A strong answer keeps offline gates (AUC, calibration, NDCG) apart from the online verdict (A/B with revenue, user, and advertiser guardrails), and layers in counterfactual replay so you can trust a model before it ever serves a live auction.

Training a pCTR model is the easy part. The challenge is proving a change helps before it reaches revenue, when the model runs inside an auction, the logs surface only ads that won, and a 1% calibration error costs real money. This is an eval question, not a model question.