grammar

Prompting for JSON works until it doesn't, and at scale the tail breaks your parser. Constrained decoding makes malformed output literally impossible to sample. The signal is knowing exactly where in the loop the constraint takes hold. Here is the answer.

Grammar-constrained decoding seems like a solved problem until tokenization stops lining up with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing address it.