Structured output: System design
Requires model responses to conform to a declared schema with distinct handling for parse, schema, and semantic failures, plus a bounded repair loop that cannot retry indefinitely or silently drop required fields.
Package status: AI-assisted reference package. Static package validation has passed; implementation scenarios remain not run and human domain review is required before stable adoption.
Decision
Structured LLM output is validated in explicit stages before any downstream side effect occurs. Parsing confirms syntactic validity; schema validation confirms shape and required fields; semantic validation confirms domain rules that JSON Schema cannot express alone. Each failure class maps to a bounded repair policy that may ask the model to regenerate with error context, but repair attempts are capped, loop detection stops repeated identical failures, and required fields must never be silently dropped to force acceptance. Terminal rejection returns a typed error to the caller without storing a partial accepted record unless the profile explicitly allows human review queues.
Scope
- Output envelopes for extraction, classification, and tool-result parsing.
- Parse, schema, and semantic validation ordering and failure taxonomy.
- Bounded repair loops and terminal accept or reject states.
- Observability for failure classes and repair counts.
Outside this block
- Model selection and cost routing (see related model routing block).
- Prompt injection guardrails and tool authorization.
- Human labeling workflows for rejected envelopes.
Contract
- Every structured output request references a schema version; validation runs parse, then schema, then semantic checks in that order.
- Parse failures reject trailing non-json text and malformed delimiters; they never partially deserialize into downstream systems.
- Schema failures identify missing or mistyped required fields; repair must not omit required fields to pass validation.
- Semantic failures are recorded separately from schema failures and use the profile repair.semantic.max_attempts budget, which may be lower than parse or schema budgets when business risk is higher.
- Repair attempts are limited per failure class by profile keys repair.parse.max_attempts, repair.schema.max_attempts, and repair.semantic.max_attempts, and in aggregate by repair.total_max_attempts; identical validation error hashes on consecutive attempts terminate repair immediately when repair.stop_on_same_error_hash is enabled.
- Terminal rejected status returns failure class and repair count to the caller and does not write an accepted record unless an explicit human-review path is enabled.
Implementation guidance
- Keep validation code independent of vendor SDK parsing helpers so failure classes remain portable.
- Pass structured error paths into repair prompts rather than raw model output alone.
- Store schema version and validator version alongside accepted records for replay and audit.
Failure handling and safeguards
- Time out repair loops with a wall-clock budget separate from per-call model latency.
- Do not execute tools or database writes until the envelope reaches accepted state.
- Redact secrets from repair prompts when prior output accidentally included sensitive tokens.
Verification and operations
- Measure accept rate, failure class distribution, average repair count, and loop-detection stops.
- Fixture-test parse, schema, and semantic failures independently.
- Prove required-field dropping attempts never reach accepted state.
The executable-looking examples in this package are fixtures and acceptance contracts. Run static validation on the package structure and metadata, then implement and execute the scenarios in the target repository.
Adoption assumptions
- Names and numeric values in
example.yamlare an adoption profile, not universal defaults. - JSON Schema 2020-12 is referenced descriptively; teams may use equivalent validators.
- Vendor structured-output features may assist generation but do not replace server-side validation.