75Implement a learning-rate scheduler with linear warmup and cosine decay.▼mediumGoogleMetaOpenAI1 replies◆ premiumNearly every modern training run relies on warmup-then-cosine, and getting it wrong destabilizes early training or squanders the tail. It's a closed-form function of the step. Here is the implementation and the why.Open full answer →
48Design the request router and scheduler for a multi-replica LLM serving cluster.▼expertOpenAINVIDIAAWS2 replies◆ premiumRound-robin routing throws away a feature that doubles throughput: cache locality. Designing the router means picking a replica by KV state, not just least-loaded. Here is the two-layer design.Open full answer →