Free-text output
- "The user is named Alex, 29, from NYC"
- Requires fragile string parsing
- Occasionally malformed or incomplete
Structured output
- {"name": "Alex", "age": 29, "city": "NYC"}
- Directly parseable, type-checked
- Guaranteed schema-valid
LLMs naturally produce free-flowing prose, but software needs data in predictable shapes: a JSON object with specific fields, an enum from a fixed set of choices, a value matching a particular type. Structured output is the set of techniques that make a model's response reliably conform to a schema you define, so it can be safely parsed and used by downstream code without guesswork.
- JSON mode — a provider feature that guarantees the output is *syntactically valid JSON*, though not necessarily matching any particular schema.
- Function calling / tool calling — a related mechanism (its own star) where the structured output represents a request to call a specific function with typed arguments.
- JSON Schema — a standard way to describe the exact shape (fields, types, required-ness) a piece of JSON must have.
- Constrained decoding / grammar-constrained generation — a stronger, decoding-time guarantee that every token generated is forced to keep the output valid against a schema or grammar, not just 'likely to be JSON'.
Structured output matters because most real applications don't display raw model text to a human — they feed it into a database, a UI component, another function call, or another model. Any of those downstream consumers breaks the moment the format is even slightly off (an extra sentence before the JSON, a missing quote, a number written as a word).
- Structured output is what makes LLMs usable as a component inside deterministic software pipelines, not just chat interfaces.
- It underlies function/tool calling, form extraction, data labeling at scale, and any agent that needs to hand off structured decisions to code.
- A model can still be *substantively* wrong (wrong values) even when it's *structurally* perfect (valid schema) — structure and correctness are separate guarantees.
Think of the guarantee spectrum as a ladder: prompting for a format, provider JSON-mode, schema validation with retries, and full grammar-constrained decoding — each rung trading some flexibility for a stronger guarantee. Later layers walk up that ladder mechanically and mathematically.