momentum

Coding Adam shows you know what .step() actually does: momentum combined with per-parameter adaptive rates and bias correction. What interviewers watch for is the two moments plus the bias-correction step.

Everyone says 'gradient descent,' but the batch-size decision and momentum are the actual interview content. What earns credit is the noise-vs-cost tradeoff across batch/SGD/mini-batch and why momentum pays off.