aillmjson-schemavalidation

Validating LLM JSON Output with JSON Schema

·9 min read·AI & JSON

Syntactic validity is not enough

A model can return perfectly valid JSON that is completely wrong for your application: missing required fields, a string where you need a number, an enum value you never defined. JSON.parse() will happily accept all of it. The only way to catch these is schema validation.

The validation pipeline

A production-grade LLM data pipeline has three gates:

  1. Parse — is it valid JSON syntax?
  2. Validate — does it match the expected schema (fields, types, enums, ranges)?
  3. Sanity-check — are the values plausible (dates not in the year 9999, totals not negative)?

Defining a schema once

Write the schema in one place and use it both to constrain the model and to validate the response. Example JSON Schema:

json
{
  "type": "object",
  "properties": {
    "ticket_id": {"type": "string", "pattern": "^TK-[0-9]{6}$"},
    "priority": {"type": "string", "enum": ["low", "medium", "high", "urgent"]},
    "tags": {"type": "array", "items": {"type": "string"}, "maxItems": 5},
    "due_date": {"type": "string", "format": "date"}
  },
  "required": ["ticket_id", "priority"],
  "additionalProperties": false
}

Validating with Ajv (JavaScript)

typescript
import Ajv from "ajv";
import addFormats from "ajv-formats";

const ajv = new Ajv({ allErrors: true });
addFormats(ajv);
const validate = ajv.compile(schema);

function validateModelOutput(raw: string) {
  const data = JSON.parse(raw);
  if (!validate(data)) {
    throw new Error("Schema validation failed: " + ajv.errorsText(validate.errors));
  }
  return data;
}

Validating with Pydantic (Python)

python
from pydantic import BaseModel, field_validator
from typing import Literal

class Ticket(BaseModel):
    ticket_id: str
    priority: Literal["low", "medium", "high", "urgent"]
    tags: list[str] = []

    @field_validator("ticket_id")
    @classmethod
    def check_id(cls, v: str) -> str:
        if not v.startswith("TK-"):
            raise ValueError("ticket_id must start with TK-")
        return v

ticket = Ticket.model_validate_json(raw)  # parses AND validates

Validating with Zod (TypeScript)

typescript
import { z } from "zod";

const Ticket = z.object({
  ticket_id: z.string().regex(/^TK-\d{6}$/),
  priority: z.enum(["low", "medium", "high", "urgent"]),
  tags: z.array(z.string()).max(5),
});

type Ticket = z.infer<typeof Ticket>;
const ticket = Ticket.parse(JSON.parse(raw)); // typed + validated

The retry-on-failure pattern

When validation fails, the best recovery is often to send the error back to the model and ask it to fix its own output:

text
Your previous JSON failed validation with this error:
  "priority" must be one of: low, medium, high, urgent

Return corrected JSON only.

This "validate then re-prompt with the error" loop pushes reliability toward 100% without human intervention. Cap it at two or three attempts to avoid runaway cost.

Best practices

  • Set additionalProperties: false so unexpected keys are caught, not silently ignored.
  • Validate ranges and formats, not just types (minimum, maxLength, format).
  • Log the raw response whenever validation fails — it is your best prompt-tuning signal.
  • Keep the same schema in version control; treat it as an API contract.

Frequently asked questions

Yes. Provider guarantees are strong but not absolute across retries, streaming, and model upgrades. Validation is cheap insurance.

Use what matches your stack: Ajv/Zod for TypeScript, Pydantic for Python. All three validate against the same conceptual schema.

Define nested schemas and compose them. Both Zod and Pydantic make nested validation straightforward, and JSON Schema supports $ref for reuse.

Try JSON Schema Validator

Validate LLM JSON against a schema and see every error explained.