training stability
AI, ML & GenAI interview questions tagged training stability, across every topic.
3 questions · 0 unlocked for you
Concepts behind "training stability"
The curriculum that explains the ideas these questions test.
Core
Normalization in Transformers: LayerNorm, RMSNorm, Pre-Norm and Post-NormNormalization keeps activations in a range where a deep stack can actually train. LayerNorm re-centers and re-scales each token vector; RMSNorm drops the mean subtraction entirely and only divides by the root mean square, which costs nothing in quality while removing about half the elementwise arithmetic and one learned parameter tensor. Where you put the norm matters more: pre-norm leaves the residual path clean and is why 60-plus-layer stacks train at all, while post-norm can end slightly better but fights you the whole way. AI, ML, and GenAI engineer interviews probe it because it is the difference between a model that converges and one that diverges at step 300.🧠 Foundations of LLMs & GenAISign in
Core
Vanishing and Exploding GradientsIn a deep or recurrent network the backward gradient is a product of many per-layer Jacobians, so its magnitude compounds: factors mostly below one drive it toward zero (early layers stop learning) and factors above one make it explode (training diverges into NaNs). The root cause is that repeated multiplication, and the standard fixes attack it head-on: residual connections to hand gradient a shortcut, normalization to keep activations in scale, gating to hold signal across time, gradient clipping to cap the blow-up, and careful initialization. AI, ML, and GenAI engineer interviews probe it because it is the mechanism behind most deep-net training failures you have to diagnose.📊 Evaluation & ML FoundationsSign in