json schema

Tool calling powers every agent. The signal is grasping that the model executes nothing: it emits a structured request your code runs, then you hand the result back. Here is the loop and the details that determine reliability.

Grammar-constrained decoding seems like a solved problem until tokenization stops lining up with the grammar. The signal is the token-vs-character boundary mismatch and how prefix automata and token healing address it.

Every agent is a loop around this function. The candidates who fail it raise an exception on a bad tool call; the ones who pass hand the error back to the model as a tool result and let it fix itself on the next turn.

Forcing valid JSON sounds free, but the mask computation can stall every decode step. The fix is precompiling the grammar into a fast automaton. Here is where the cost hides and how to strip it out.