knowledge distillation

Distillation gives you most of a frontier model's quality for a fraction of the serving cost. The signal is naming the three LLM-specific variants and knowing which one holds up against a closed API. Here is the answer.

Distillation is not a single method. The signal is knowing when to match logits versus train on generated text, why on-policy distillation beats off-policy, and how reasoning models are distilled.