82When do you use pairwise versus pointwise evaluation for LLM outputs, and what does each get wrong?▼mediumOpenAIMicrosoftCohere1 replies◆ premiumPairwise asks which one wins, pointwise asks how good a single output is. The signal is knowing which is more reliable, how Elo/Bradley-Terry aggregates comparisons, and where each method quietly misleads.Open full answer →
113What is CUPED, why does it shrink experiment variance, and how does it compare to stratification and regression adjustment?▼hardMicrosoftNetflixMeta1 replies◆ premiumCUPED can halve the sample size an A/B test requires without touching validity. The signal is explaining why subtracting a pre-experiment covariate reduces variance yet never biases the estimate. Here is the answer.Open full answer →
114Why does peeking at an A/B test inflate false positives, and how do sequential and always-valid tests fix it?▼hardNetflixUberMicrosoft1 replies◆ premiumChecking an experiment daily and stopping the moment it hits significance can triple your false-positive rate. The signal is knowing why, and the family of methods that make continuous monitoring valid. Here is the answer.Open full answer →
