TOON vs JSON: Cut LLM Token Costs 30–60% Without Losing Structure

Every field name in your prompt is a tax

When you paste JSON into an LLM prompt, you pay for structure you never actually needed the model to read. In an array of 500 user records, the keys "id", "name", "email" and every {, }, ", and : are repeated 500 times — and each one costs tokens, which cost money and eat into your context window. For data-heavy prompts (RAG chunks, event logs, tabular records) that overhead can be the majority of your tokens.

TOON — Token-Oriented Object Notation — is a serialization format built to fix exactly that. Released in October 2025 by Johann Schopplich, it's a compact, human-readable encoding of the *same* JSON data model, designed specifically for passing structured data into LLM prompts. It borrows YAML's indentation for nested objects and a CSV-style tabular layout for uniform arrays, so repeated keys are declared once instead of on every row.

The headline result: across benchmarks TOON uses roughly 30–60% fewer tokens than JSON, and — counter-intuitively — often *improves* retrieval accuracy because the explicit row/column structure is easier for models to follow than deeply-bracketed JSON.

The same data, two formats

Here's an array of records as JSON:

{
  "hikes": [
    { "id": 1, "name": "Blue Lake Trail", "distance": 7.5, "elevation": 320 },
    { "id": 2, "name": "Ridge Overlook", "distance": 9.2, "elevation": 540 },
    { "id": 3, "name": "Cedar Loop", "distance": 4.1, "elevation": 150 }
  ]
}

And the exact same data in TOON:

hikes[3]{id,name,distance,elevation}:
  1,Blue Lake Trail,7.5,320
  2,Ridge Overlook,9.2,540
  3,Cedar Loop,4.1,150

Every key (id, name, distance, elevation) appears once in the header instead of three times. Scale that to hundreds of rows and the savings compound. The [3] is a length marker and {...} is the column header — together they give the model an explicit contract: *"three rows, four fields each"* — which is why accuracy tends to go up, not down.

The syntax in five minutes

TOON has a small surface area. There are really only three shapes to learn.

1. Uniform arrays of objects → a table. This is TOON's sweet spot. Declare the length and the columns once, then emit one row per line:

users[2]{id,name,role}:
  1,Ada Lovelace,admin
  2,Alan Turing,editor

2. Nested objects → YAML-style indentation. No braces, just two-space indents:

config:
  theme: dark
  cache:
    ttl: 3600
    enabled: true

3. Scalars and simple arrays. Key-value pairs read like YAML; primitive arrays inline with their length:

name: JSONKit
version: 2.4.0
tags[3]: json,tools,ai

The default delimiter is a comma, but you can switch to tabs for an extra token saving (tabs are a single token in most tokenizers and never collide with values that contain commas). Values that contain the delimiter, a newline, or leading/trailing spaces get quoted — the same escaping instinct you already have from CSV.

What the benchmarks actually say

Two numbers matter when you evaluate a prompt format: how many tokens it costs and how accurately the model reads it back. TOON is notable because it improves on both at once for the right data shapes.

• On mixed-structure datasets, TOON scored ~76% retrieval accuracy vs JSON's ~75%, while using ~40% fewer tokens.
• Measured as *accuracy per 1K tokens*, TOON lands around 27.7 vs JSON's 16.4 — roughly a 1.7× efficiency gain.
• On flat tabular data, TOON is only about 6% larger than raw CSV, but unlike CSV it carries explicit length and field metadata that makes the model far less likely to drop or misalign a column.

The takeaway isn't "TOON is always better." It's that for uniform, table-like data at scale, you get cheaper *and* more reliable prompts — a rare combination.

When NOT to use TOON

TOON is a specialized tool, not a JSON replacement. Reach for plain JSON when:

• Your data is deeply nested and non-uniform. If almost nothing is a clean array-of-same-shaped-objects, TOON's tabular advantage disappears and its indentation can cost *more* than JSON's braces.
• Arrays are only semi-uniform. When rows have optional or varying fields (say 40–60% share a shape), the token savings shrink sharply and readability suffers.
• You're feeding a machine, not a model. TOON is for the *prompt*. Your API responses, config files, and inter-service payloads should stay JSON — it has universal parser support and TOON does not. TOON lives at the boundary where data enters an LLM, and the model's *output* should still be validated JSON.
• Latency matters more than token count. Some local or quantized models parse compact JSON just as fast, so the token win may not translate to a speed win.

A good rule: TOON in, JSON out. Encode your context as TOON to save input tokens, but ask the model to *respond* in JSON so you can validate it with a schema.

A practical workflow

You don't rewrite your app to use TOON — you convert at the prompt boundary. The official @toon-format/toon TypeScript SDK gives you encode() and decode(), and ports exist for Python and other languages. The flow looks like this:

1. Build/fetch your context as normal JSON.
2. Clean it up — format and validate so the shape is correct BEFORE encoding.
3. encode(json) → TOON string, drop it into the prompt.
4. Ask the model to reply in JSON (structured output / JSON mode).
5. Validate the model's JSON reply against a schema.

Steps 2 and 5 are where JSONKit fits. Before you encode, make sure your JSON is actually well-formed and uniform:

1. Format and eyeball the shape. Paste your context into the JSON Formatter — uniform arrays are exactly the shape TOON compresses best, so confirm your records really do share fields.
2. Flatten irregularities. If some objects are missing keys, the array isn't uniform; normalize it first (a quick pass in the JSON Explorer makes gaps obvious).
3. Estimate the win. Minify your JSON to see its true byte size, then compare — TOON's savings track roughly with how much of your payload is repeated keys and punctuation.
4. Validate the round-trip. After the model replies, lock its output down with a JSON Schema so a malformed response never reaches your database.

Want realistic uniform data to experiment with? Generate a few hundred same-shaped records in the Random JSON Generator — that's the ideal input to feel TOON's compression, and it mirrors the telemetry/user-list workloads where TOON shines. For the broader picture of trimming prompt costs, see Reduce LLM Token Cost with JSON.