23What is data skew in a distributed job (Spark), and how do you fix it?▼hardDatabricksAmazonMeta1 replies◆ premiumThe number-one cause of mysteriously slow Spark jobs: one partition handles most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.Open full answer →
08How do you run hyperparameter optimization efficiently across a cluster of GPUs?▼mediumGoogleNVIDIADatabricks1 repliesunlockedGrid search is the wrong answer at scale. The signal is knowing why random beats grid, how Bayesian optimization and early-stopping schemes (Hyperband/ASHA) spend compute wisely, and how to parallelize without stragglers.Open full answer →
20What is model sharding, and how do tensor and pipeline parallelism split a model across GPUs?▼hardNVIDIAOpenAIGoogle2 replies○ sign inWhen a model is too large for one GPU you split the model itself, not just the data. The signal is telling tensor parallelism (split within a layer) apart from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.Open full answer →
