JSON for AI & LLM Engineering: Every Response Shape You'll Parse

Almost every AI feature you build is, underneath, a JSON parsing problem. You send JSON to a model and you get JSON back — chat completions, tool calls, embeddings, retrieval results, agent traces. Knowing the exact shape of each payload is the difference between code that handles real responses and code that breaks the first time finish_reason is "length" or content is null.

This is a field guide to the JSON formats you'll meet across the LLM stack. Every section links to a copy-ready example with documented fields, real-world variants, and a one-click "open in tool" button so you can format, validate, or generate TypeScript types from it instantly.

> Browse the whole set on the JSON Examples Library — the AI & LLM category sits right at the top.

Chat completions: the core request/response

The workhorse of any LLM app is the chat completion. You send an array of role-tagged messages and get back a message plus token usage.

• OpenAI Chat Completion Response — choices, finish_reason, usage, plus the streaming-chunk delta shape.
• Anthropic Claude Messages Response — ordered content blocks, stop_reason, and input/output token usage.
• Chat Conversation History — the messages array you actually send, including multimodal turns and summarized memory.

The two gotchas that bite everyone: content can be null when the model returns a tool call instead of text, and a finish_reason (or stop_reason) of length/max_tokens means the answer was truncated — always handle it.

Tool calling, function calling & MCP

When a model needs to *do* something — look up an order, query a database — it emits a tool call: a function name plus JSON arguments. You run the function and feed the result back.

• LLM Tool / Function Call — the tool definition you register and the call the model returns. Note the arguments arrive as a JSON-encoded string, so parse and validate before use.
• AI Agent Tool Registry — the full manifest of tools you expose, each with a JSON Schema for its parameters.
• MCP Message (Model Context Protocol) — the JSON-RPC 2.0 messages (tools/list, tools/call) behind MCP servers. New to MCP? Read MCP Explained: The JSON-RPC Behind AI Tool Use.

Structured output: making the model return your shape

Free-text parsing is fragile. Structured output constrains the decoder to a JSON Schema so the response matches your shape exactly — no regex, no defensive parsing.

• LLM Structured Output (JSON Schema) — a response_format request with strict: true and the guaranteed-valid result.
• Prompt Template (Reusable) — versioned prompts with variables and few-shot examples, stored as JSON outside your code.

Embeddings, vector search & RAG

Retrieval-augmented generation is mostly moving vectors and chunks around. The shapes:

• Embedding Vector Response — the dense float vector plus model and usage.
• Vector Database Search Query — a topK query with a metadata filter and the ranked matches response.
• RAG Document Chunk — chunk text with the metadata that powers citations and filtering.
• LangChain / LlamaIndex Document — the page_content + metadata object loaders and splitters pass around.
• RAG Answer with Citations — a grounded answer with cited sources, plus the all-important "no-answer" refusal shape.

A reliable RAG system abstains when retrieval finds nothing relevant instead of hallucinating — that grounded-refusal payload is what separates trustworthy RAG from confident nonsense. For a stage-by-stage walkthrough of these shapes, see RAG JSON Formats Explained.

Agents: tracing what happened

Agents loop: think, call a tool, observe, repeat. Capturing each step as JSON makes runs debuggable and reproducible.

• AI Agent Run Trace — the ReAct-style steps array (thought / action / observation) plus aggregate token and latency usage, with a failed-run variant. For a full breakdown of agent JSON, see Agent JSON: Traces, Tool Registries & State.

Training, evaluation & safety

• Fine-Tuning Dataset Example (JSONL) — the chat messages format, one JSON object per line. Remember: a fine-tuning file is JSONL, not a JSON array.
• LLM Evaluation / Benchmark Result — per-case scores, aggregate pass rate, and an LLM-as-judge variant.
• Content Moderation Response — flagged plus per-category booleans and confidence scores.

Multimodal: audio & images

• Speech-to-Text Transcription (Whisper) — full text, detected language, and timestamped segments for captions.
• AI Image Generation Request & Response — prompt, size, and the returned image url or b64_json.

Tips for working with AI JSON

1. Never trust the shape blindly. Even with structured output, validate against a schema. Models can omit fields or, in failure modes, return prose. Run model output through the JSON Validator or JSON Schema Validator.
2. Parse stringified arguments. Tool-call arguments come as a JSON string — JSON.parse() them, then validate.
3. Handle truncation. A length/max_tokens stop reason means incomplete output; raise max_tokens or continue the response.
4. Generate types from real payloads. Paste any example into JSON to TypeScript to get interfaces you can actually rely on.
5. Explore deeply nested responses. LLM responses nest fast — drop one into the JSON Tree Explorer to navigate choices[0].message.tool_calls[0] without losing your place.