This asks you to design the scoreboard, not the player. A strong answer keeps offline gates (AUC, calibration, NDCG) apart from the online verdict (A/B with revenue, user, and advertiser guardrails), and layers in counterfactual replay so you can trust a model before it ever serves a live auction.
← System Design for AI in Production / 81
Design an evaluation framework for an ads ranking system.
This asks you to design the scoreboard, not the player. A strong answer keeps offline gates (AUC, calibration, NDCG) apart from the online verdict (A/B with revenue, user, and advertiser guardrails), and layers in counterfactual replay so you can trust a model before it ever serves a live auction.
Updated Sep 2026 · Grounded in real GenAI, LLM, and AI/ML engineering interview loops and written to a senior-engineer editorial bar.
Unlock the other 847 answers · ₹2,000 / $25Your progress and mastery stay saved · 6 months · one payment · no auto-renew
UP NEXT ON YOUR JOURNEY
DISCUSSION · 0
No comments yet — be the first to share your approach.
