42Why does the learning rate matter so much, and how do warmup and decay schedules help?▼mediumNVIDIAGoogleOpenAI1 replies◆ premiumThe learning rate is the single hyperparameter that can NaN your run at step 50 or stall it indefinitely. Modern training schedules it rather than pinning it to one value. Interviewers want you to know precisely what warmup and decay each give you. Here is the answer.Open full answer →
31What is model cold-start / warmup in serving, and how do you handle it for autoscaling?▼mediumAmazonMicrosoftGoogle1 replies◆ premiumA newly booted replica is sluggish on its opening requests while weights load and kernels compile, which fights with reactive autoscaling. The signal is identifying where cold start comes from plus the warmup and provisioning remedies.Open full answer →
