vanishing gradients

This question connects why deep nets were hard to train with the cluster of tricks that solved it. What interviewers reward is the multiplicative-gradient cause and naming the real fixes: ReLU, He/Xavier init, residuals, normalization. Here is the answer.

Sequence models remain interview staples, particularly the gating that repaired RNNs and the reason attention took over. What matters is linking the vanishing-gradient narrative to the parallelism case that let transformers ride the scaling wave.

Bad initialization makes deep nets fail to train at all, with activations that vanish or explode through depth. What matters is knowing why zero init is fatal and how to match Xavier or He to your activation function.