mfu

nvidia-smi reading 100% can mask the fact that you are tapping only a fraction of the hardware's real compute. What matters is MFU (useful FLOPs vs peak) and the gap between 'the GPU is busy' and 'the GPU is efficient'.

Full activation checkpointing saves memory but costs a flat 30% extra compute. Selective recomputation wins most of that back by recomputing only the cheap, memory-heavy operations. Here is how to pick what to recompute.

The collective communication in distributed training is pure overhead unless it runs while the GPU computes. Hiding it is the difference between 30% and 55% MFU. Here is how the overlap actually works and how you check it on a trace.

FP8 can nearly double training throughput over BF16, but with only a few mantissa bits the numerics leave no slack. Per-tensor scaling and a selective recipe are what get it to converge.

Most answers optimize the training run. For any model that gets used, inference dominates lifetime energy, and the biggest carbon lever is the serving stack. Measure first, then cut the thing that is actually large.