metrics
AI, ML & GenAI interview questions tagged metrics, across every topic.
13 questions · 3 unlocked for you
Concepts behind "metrics"
The curriculum that explains the ideas these questions test.
Foundational
The Bias-Variance TradeoffA model's error breaks into bias (error from being too simple to capture the pattern, underfitting) and variance (error from being too sensitive to the training sample, overfitting). Cutting one often raises the other, so generalization comes down to finding the balance. It is the lens behind regularization, model-complexity choices, and ensembling. AI, ML, and GenAI engineer interviews probe it because diagnosing whether a model underfits or overfits, and acting on it, is the core debugging skill of ML.📊 Evaluation & ML Foundations
Foundational
Cross-Validation (Done Right)Cross-validation estimates how a model generalizes by training and testing on rotating folds, yielding a more reliable estimate than a single split. The traps are what make it an interview topic: use stratified folds for imbalanced classes, grouped folds when records share an entity, and time-ordered splits for temporal data (never random), and fit all preprocessing inside each fold to avoid leakage. AI, ML, and GenAI engineer interviews probe it because the wrong scheme produces optimistic estimates that fall apart in production.📊 Evaluation & ML Foundations
Foundational
Ensembling: Bagging, Boosting, StackingEnsembles combine multiple models to beat any single one, because if their errors are decorrelated, combining cancels mistakes. Bagging trains parallel models on bootstrap samples and averages (reducing variance, e.g. random forest); boosting trains models sequentially to fix prior errors (reducing bias, e.g. XGBoost); stacking trains a meta-model to combine base models. Model diversity is the requirement. AI, ML, and GenAI engineer interviews probe it because gradient boosting dominates tabular ML and the bias/variance framing connects to everything.📊 Evaluation & ML Foundations
Foundational
Precision, Recall, and F1Precision is what fraction of your positive predictions were correct; recall is what fraction of the actual positives you caught. They trade off as you slide the decision threshold, and which one matters depends on the cost of false positives vs false negatives. F1 is their harmonic mean. On imbalanced data, accuracy misleads and these metrics (with PR-AUC) tell the truth. AI, ML, and GenAI engineer interviews probe them because choosing and tuning the threshold by business cost is a core, constantly-tested skill.📊 Evaluation & ML Foundations
Foundational
Eval-Driven Development and Golden DatasetsYou cannot improve an LLM system you cannot measure, so the first thing to build is an evaluation: a golden dataset of representative inputs with expected behavior, plus metrics, that you run on every change. This converts 'it feels better' into a number, catches regressions before users do, and lets you iterate fast. AI, ML, and GenAI engineer interviews probe it because teams that ship reliable LLM features evaluate continuously, and 'we tried some prompts and it looked good' is the anti-pattern.📊 Evaluation & ML Foundations
Foundational
Offline vs Online EvaluationOffline evaluation scores a model on held-out data; online evaluation measures its impact on real users (through an A/B test). They often disagree: an offline win frequently fails to move the online metric, because offline data is a static proxy while the real world carries feedback loops, distribution shift, and second-order effects. The discipline is to gate with offline evals (fast, cheap) and confirm with online tests (the truth). AI, ML, and GenAI engineer interviews probe it because shipping on offline metrics alone is a classic, costly mistake.📊 Evaluation & ML Foundations