← 📊 Evaluation & ML FoundationsNEXT IN EVALUATION & ML FOUNDATIONSEval-Driven Development and Golden Datasets→
Core
Calibration and Uncertainty
A model is calibrated when its confidence lines up with reality: among the predictions it makes at 0.8, roughly 80% turn out correct. Modern neural nets (and LLMs) tend to be overconfident, so raw scores are not trustworthy probabilities. You correct it post-hoc with temperature scaling, Platt scaling, or isotonic regression on a held-out set, and you measure it with reliability diagrams and Expected Calibration Error. AI, ML, and GenAI engineer interviews probe it because any decision made on a probability (thresholds, expected value, abstention) is only as good as the calibration.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
Machine Learning & Data ScienceHow do you calibrate a model's probabilities (Platt scaling, isotonic, temperature)?→Machine Learning & Data ScienceWhat is model calibration, why does it matter, and how do you measure and fix it?→Machine Learning & Data ScienceWhat is conformal prediction, and how does it give calibrated uncertainty?→Machine Learning & Data ScienceYour churn model's AUC jumps from 0.71 to 0.93 after adding a 7-day rolling feature. What now?→Machine Learning & Data ScienceDefine precision, recall, F1, and AUC, and give a case where each (and accuracy) is misleading.→ML Infrastructure & GPUsExplain quantization for inference: INT8/INT4, GPTQ/AWQ, what breaks, and how you validate it.→
COMPANIES THAT ASSUME THIS