14How do you select GPUs for LLM training and inference, and what specs actually matter?▼mediumNVIDIAOpenAIxAI1 replies○ sign inPicking GPUs is a genuine applied decision, and 'get the biggest one' misses the point. The signal is knowing which spec binds your workload (VRAM, bandwidth, interconnect) and the train-versus-serve difference that flips the answer.Open full answer →
62Your training collectives are slow. How do you debug the NCCL/interconnect path and find where bandwidth is lost?▼hardNVIDIAMetaMicrosoft2 replies◆ premiumWhen all-reduce is your bottleneck, the culprit is nearly always a misconfigured path: traffic on the wrong link, a dead NIC, or a topology NCCL failed to discover. Here is the methodical way to track down the missing bandwidth.Open full answer →
