generation

Every LLM product tunes these knobs, and the interviewer wants to hear what each one does to the probability distribution and which setting suits which task. That includes why factual and creative tasks call for opposite settings.

How you decode from the model's probabilities shapes the output as much as the model itself. The signal is knowing the deterministic vs sampling methods and where each fits (factual vs creative). Here is the answer.

Neural text degeneration is a known failure of likelihood-maximizing decoding, not a broken model. The fixes are specific decode-time knobs. Here is which lever to reach for and what each one costs.

Generation has to end somewhere, and the wrong stop rule either truncates answers or wastes tokens on trailing garbage. Here is how the EOS token, stop strings, and max-token caps actually interact.

For open-ended generation there's no ground-truth string to match, so accuracy is meaningless. The field relies on a layered mix of automatic, model-based, and human metrics. Here is how to assemble a credible eval.

Every chat model's 'temperature' and 'top_p' knobs come down to a few lines of logit surgery. Implementing them proves you understand decoding rather than just calling it. Here is the from-scratch version.