Structured Output

Layer 4 · Foundations

Structured Output

The formal layer: grammars as finite-state constraints on decoding, and the token-masking math that enforces them.

15 min read180 XP

  1. Grammar / schema

    compiled ahead of time

  2. Parser state

    tracks valid continuations

  3. Token mask

    per-step valid token set

  4. Masked logits

    invalid tokens set to -inf

  5. Softmax + sample

    guaranteed valid choice

Grammar-constrained decoding as a per-step masking pipeline.

Formally, a JSON Schema (or any context-free grammar) can be compiled into a finite-state or pushdown automaton over *tokens* rather than characters. At each decoding step, the automaton's current state defines exactly the set of tokens that keep the output on a path to a valid completion; every other token is masked out before sampling.