98Why does constrained decoding break on token boundaries, and how do you handle the edge cases?▼expertOpenAIMicrosoftHugging Face2 replies◆ premiumGrammar-constrained decoding seems like a solved problem until tokenization stops lining up with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing address it.Open full answer →
119What are logits and log-probs, and how do you actually use them in production?▼easyOpenAIAnthropicCohere◆ premiumDefinitions take thirty seconds. The interview is about the four things log-probs buy you in production: a confidence signal, banning tokens, guaranteeing valid output, and classification you can calibrate. Plus the tokenization gotcha that quietly breaks all of them.Open full answer →
