Vanishing and Exploding Gradients

In a deep or recurrent network the backward gradient is a product of many per-layer Jacobians, so its magnitude compounds: factors mostly below one drive it toward zero (early layers stop learning) and factors above one make it explode (training diverges into NaNs). The root cause is that repeated multiplication, and the standard fixes attack it head-on: residual connections to hand gradient a shortcut, normalization to keep activations in scale, gating to hold signal across time, gradient clipping to cap the blow-up, and careful initialization. AI, ML, and GenAI engineer interviews probe it because it is the mechanism behind most deep-net training failures you have to diagnose.