structured decoding
AI, ML & GenAI interview questions tagged structured decoding, across every topic.
2 questions · 0 unlocked for you
Concepts behind "structured decoding"
The curriculum that explains the ideas these questions test.
Core
Constrained and Structured DecodingAsking a model nicely for JSON sometimes fails; constrained decoding guarantees valid output by masking, at each generation step, every token that would break a schema or grammar, so only valid continuations can be sampled. It is the dependable way to get JSON, enums, or function-call arguments, and it underpins tool calling. The caveat: it guarantees structural validity, not semantic correctness. AI, ML, and GenAI engineer interviews probe it because production systems depend on parseable output, and 'just prompt for JSON' breaks at scale.🧠 Foundations of LLMs & GenAISign in
Core
Logits, Log-Probs, and Logit BiasThe output layer of an LLM is an API surface, not just an implementation detail. Logits are raw per-token scores, softmax turns them into probabilities, and log-probs are what providers actually return because they are numerically stable and add up across a sequence. Four production techniques live here: confidence scoring for routing and abstention, logit bias to ban or force a token, structured decoding by masking invalid tokens, and classification by reading a single position's log-probs instead of parsing prose. AI, ML, and GenAI engineer interviews probe it because it is the difference between treating the model as a text box and treating it as a probabilistic component.🧠 Foundations of LLMs & GenAISign in