gradient checkpointing

Activations, not just weights, can dominate training memory, and gradient checkpointing is the standard remedy. The signal is the precise trade: recompute activations in the backward pass rather than storing them. Here is the answer.

Full activation checkpointing saves memory but costs a flat 30% extra compute. Selective recomputation wins most of that back by recomputing only the cheap, memory-heavy operations. Here is how to pick what to recompute.