44Compare LLM decoding strategies: greedy, beam search, temperature, top-k, top-p, and repetition penalties.▼mediumOpenAIGoogleCohere1 replies◆ premiumHow you decode from the model's probabilities shapes the output as much as the model itself. The signal is knowing the deterministic vs sampling methods and where each fits (factual vs creative). Here is the answer.Open full answer →
07Implement beam search for a sequence model given a next-token log-probability function.▼hardGoogleNVIDIAMeta1 repliesunlockedBeam search is the decoding algorithm everyone cites and few can code correctly. The signal is keeping k hypotheses by cumulative log-prob, adding logs (not multiplying probs), and handling completed sequences and length. Here is a correct implementation and the tradeoffs.Open full answer →
