Prompt engineering → Structured outputs & tool use (schema, validate, retry)

LLM Structured Outputs: Schema, Validate, Retry

What it is

Free text is for humans. Every place a model output is consumed by code — a pipeline step, a database write, a tool invocation, a UI component — the output must be structured and validated:

  • Structured outputs / JSON mode: the model returns JSON matching a schema. Providers offer varying guarantees: from "valid JSON" to "guaranteed to match this JSON Schema" (constrained decoding). Anthropic: tool use with an input_schema, or a JSON-schema-constrained output mode where available.
  • Function / tool calling: the model chooses a tool and emits its arguments as JSON matching the tool's schema. The runtime executes it and returns the result. This is how agents act (single agent loop) — and the schema is the contract.
  • Validate + retry: parse → validate against a typed model (Pydantic/Zod) → on failure, re-prompt with the validation error text so the model can fix exactly what's wrong. Bounded retries; then fallback (default value, abstain, human).

The metric that matters: schema violation rate per call site and model version. A rising rate is a regression signal as important as accuracy.

How to implement

  1. Define the schema as code (Pydantic/Zod/JSON Schema) with tight types, enums, required fields, and descriptions — descriptions are read by the model.
  2. Use native enforcement where available (tool input_schema, structured-output mode); it eliminates most syntax errors. Still validate — enforcement guarantees shape, not semantics (a date in the past, an ID that doesn't exist).
  3. Semantic validators: cross-field rules (end > start), referential checks (cited IDs exist in retrieved set, citations), value ranges, enum membership, PII policies.
  4. Retry loop: on failure, append the error ("field amount: must be ≥ 0; got -12") and ask for a corrected object only; max 2–3 retries; temperature 0.
  5. Fallback policy per call site: default object / abstain / escalate tier / human queue. Never silently proceed with a partially valid object.
  6. Log every violation with the error type; alert on rate changes.
  7. For tools: fewer, well-described tools (tool design); validate arguments; make execution idempotent; return structured results with a status.

Small example

from pydantic import BaseModel, Field, field_validator, ValidationError
from typing import Literal
import anthropic, json

class Extraction(BaseModel):
    vendor: str = Field(min_length=1, description="Legal name of the vendor as written on the invoice")
    invoice_id: str = Field(pattern=r"^[A-Z0-9-]{4,20}$")
    total: float = Field(ge=0, description="Grand total in the invoice currency")
    currency: Literal["USD", "EUR", "GBP"]
    due_date: str = Field(pattern=r"^\d{4}-\d{2}-\d{2}$")
    citations: list[str] = Field(description="IDs of passages the values were taken from")

    @field_validator("citations")
    @classmethod
    def non_empty(cls, v):
        if not v: raise ValueError("at least one citation required")
        return v

client = anthropic.Anthropic()
TOOL = {"name": "record_invoice", "description": "Record the extracted invoice fields.",
        "input_schema": Extraction.model_json_schema()}

def extract(text: str, valid_ids: set[str], max_retries: int = 2) -> Extraction | None:
    messages = [{"role": "user", "content": f"<document>{text}</document>\nExtract the invoice fields and call record_invoice."}]
    for attempt in range(max_retries + 1):
        r = client.messages.create(model="claude-sonnet-5", max_tokens=500, temperature=0,
                                   tools=[TOOL], tool_choice={"type": "tool", "name": "record_invoice"}, messages=messages)
        args = next(b.input for b in r.content if b.type == "tool_use")
        try:
            obj = Extraction.model_validate(args)
            bad = [c for c in obj.citations if c not in valid_ids]          # semantic check the schema can't express
            if bad: raise ValueError(f"citations not in provided passages: {bad}")
            log_schema_result("invoice_extract", ok=True, attempt=attempt)
            return obj
        except (ValidationError, ValueError) as e:
            log_schema_result("invoice_extract", ok=False, attempt=attempt, error=str(e)[:300])
            messages += [{"role": "assistant", "content": r.content},
                         {"role": "user", "content": f"The output was invalid: {e}\nCall record_invoice again with corrected values only."}]
    return None   # fallback policy decided by caller: abstain / human queue

Pros

  • Reliability: code consumes validated objects, not regexes over prose.
  • Agents can act safely: tool arguments are typed and checked before execution.
  • Error feedback retries fix most violations in one round.
  • Schema violation rate is a cheap, early regression detector for prompt/model changes.
  • Descriptions in schemas double as compact instructions.

Cons

  • Shape ≠ correctness: a perfectly valid JSON with a wrong total. Semantic validation and evals remain necessary.
  • Over-constrained schemas force answers where refusal was right — include null/unknown/abstain options.
  • Retries cost tokens and latency; unbounded retry loops burn budget.
  • Provider differences in enforcement strength and tool-call semantics; migrating providers means re-testing every schema.
  • Very large or deeply nested schemas degrade model accuracy.

When to use

  • Every model output consumed by code: extraction, classification, routing decisions, tool calls, citations, pipeline handoffs.
  • Every agent action.

When NOT to use

  • Pure free-text outputs for humans (a summary) — but even then, wrap in a small envelope ({"answer": …, "citations": […]}) if you need citations or confidence.
  • Don't force a schema so rigid that "I don't know" becomes impossible (refusal / no-answer path).

Production at scale

Schema registry. Schemas are versioned artifacts alongside prompts (prompt versioning & evaluation); a schema change is a release with eval coverage. Consumers pin schema versions; add fields additively.

Violation-rate SLO per call site (e.g. < 1% after retries). Dashboards by error type (missing field, enum, pattern, semantic). A jump after a deploy = prompt/model regression; after a provider update = enforcement change.

Retry budget is part of cost: expected_calls = 1 + violation_rate × …. Keep retries ≤ 2, temperature 0, and prefer native enforcement to cut the first-attempt violation rate.

Semantic validators are where the bugs are. Referential integrity (cited IDs exist; tool argument IDs exist and belong to the tenant), range/format, cross-field consistency, and policy (an action's arguments are within the caller's permissions, excessive agency). Run them before any side effect.

Tool execution safety. Validate → authorise (policy, not the model) → execute idempotently with an idempotency key (state & durability) → return {status, result|error} structured. Errors go back to the model as data (single agent loop), never as exceptions that crash the loop.

Streaming structured output is awkward; parse incrementally only if the UI needs it; otherwise buffer.

Model upgrades: run the schema suite (every schema × sample inputs) on the candidate; compare violation rates (versioning & rollout).

Observability: log request/response pairs for violations (redacted, PII) — they're the fastest way to see what the model misunderstood in the schema description.

Prod checklist

  • Typed schema per call site with descriptions, enums, unknown/abstain options
  • Native enforcement (tool input_schema / structured mode) + code validation
  • Semantic validators: referential, range, cross-field, policy
  • Bounded retry with error feedback; temperature 0; fallback policy defined
  • Violation rate logged by type; SLO and alerts
  • Schemas versioned and registered; additive changes; consumers pin versions
  • Tool execution: validate → authorise → idempotent execute → structured result
  • Schema suite run on model/provider upgrades

Metrics to watch

MetricWhy
Schema violation rate (first attempt / after retries) per call siteRegression detector
Retry count per callCost
Semantic-validation failures by ruleWhere the model misunderstands
Tool-call argument rejection rateAgent safety