73Implement a leakage-safe, stratified train/validation/test split. What can go wrong?▼mediumMetaAmazonDatabricks2 replies◆ premiumA bad split silently inflates every metric you'll ever report, and the bugs are subtle: leaked groups, shuffled time, preprocessing fit on everything. Here is the split done right.Open full answer →
38Pull a random sample, and a stratified sample, for an ML training set in SQL.▼mediumMetaAmazonDatabricks1 replies◆ premiumBuilding a training set from a billion-row table means sampling, and the naive 'ORDER BY random() LIMIT n' sorts the whole table. Stratified sampling takes even more care. Here are both, done efficiently.Open full answer →
