65What are the common pitfalls that invalidate an A/B test?▼hardNetflixMetaAmazon2 replies◆ premiumRunning an A/B test is easy; running a valid one is hard. What matters is naming the traps that produce confidently wrong conclusions and the fix for each. Here is the checklist a senior experimenter carries.Open full answer →
115Your A/B test has interference between users (marketplace, social network). Why does it bias results and how do you fix it?▼hardUberMetaAirbnb1 replies◆ premiumStandard A/B math assumes one user's treatment does not affect another's outcome. In marketplaces and social networks that assumption breaks and your estimate is biased. The signal is naming SUTVA and the right randomization unit. Here is the answer.Open full answer →
