A practical field guide to structured outputs in LLM pipelines, covering the critical differences between JSON mode, function calling, and schema-constrained generation. Explains how each major provider (OpenAI, Anthropic, Google Gemini) implements structured outputs, with guidance on schema design best practices, failure modes, streaming partial JSON, schema versioning, and building evals for structured extraction. Includes a decision framework for choosing the right approach based on use case, and warns against common mistakes like using JSON mode for data-integrity-critical pipelines.
Table of contents
The Three Things People Mean When They Say “Structured Outputs”Where Each Provider Lands in 2026Function Calling vs Structured Outputs vs Tool UseDesigning Schemas That Do Not Fight the ModelWhen the Schema Is Wrong: Failure Modes and RecoveryPrompting With SchemasStreaming Structured OutputsSchema VersioningEvals for Structured OutputsThe Decision FrameworkSort: