duckdbsqljsonanalyticsdata-engineeringadvanced

Query JSON with DuckDB: SQL Analytics on JSON Files

·9 min read·Advanced

SQL over a JSON file, with nothing to set up

For years, "analyze this JSON file" meant writing a script, looping, and accumulating totals by hand — or spinning up a database and importing first. DuckDB collapses that to a single query. It's an in-process analytical database (think "SQLite for analytics") that reads JSON, NDJSON, CSV, and Parquet directly from disk or a URL, no server and no import step. By 2026 it's a staple of the data-engineering toolkit precisely because the friction is near zero.

If you've been reaching for jq or ad-hoc scripts to slice JSON, DuckDB gives you the full power of SQL over the same files.

The one-liner: read_json

Point DuckDB at a file and select from it as if it were a table:

sql
SELECT * FROM read_json('events.json');

DuckDB inspects the file and infers the schema and types automatically — it detects columns, numbers, booleans, dates, and nested structures without you declaring anything. For newline-delimited files it's the same call; DuckDB detects the format, and you can be explicit with read_ndjson('events.ndjson'). You can even skip the function entirely for a local file:

sql
SELECT count(*), avg(amount) FROM 'events.json';

Reaching into nested JSON with arrow operators

Real JSON is nested, and DuckDB navigates it with arrow operators. Given records like:

json
{ "id": 1, "user": { "name": "Ada", "country": "US" }, "amount": 42.50 }

use -> to get a JSON value and ->> to extract it as text:

sql
SELECT
  user ->> 'name'    AS name,
  user ->> 'country' AS country,
  amount
FROM 'events.json'
WHERE user ->> 'country' = 'US';

-> keeps the result as JSON (good for further navigation), while ->> returns a plain string you can compare and group. This is the same access model as Postgres JSONB, so the mental model transfers.

Flattening arrays with UNNEST

The operation that's painful in scripts — exploding an array into rows — is one keyword in DuckDB. Given:

json
{ "order": 100, "items": ["book", "pen", "mug"] }

UNNEST turns each array element into its own row:

sql
SELECT order, UNNEST(items) AS item
FROM 'orders.json';
-- 100, book
-- 100, pen
-- 100, mug

From there you can GROUP BY item to count occurrences across every order — a one-line answer to "what are the most common items?" that would be a nested loop in application code. This is the SQL equivalent of a JSON flatten.

Querying many files at once

DuckDB accepts glob patterns, so a whole directory of daily exports becomes one table:

sql
SELECT count(*)
FROM read_json('logs/2026-07-*.ndjson');

If the files share a key, you can join across them just like ordinary tables. This turns "I have 300 JSON log files" from a scripting chore into a single aggregation.

From analysis to output

Because it's SQL, the result of any query is itself a table you can transform, export, or convert. Compute an aggregate and write it straight to CSV or Parquet:

sql
COPY (
  SELECT user ->> 'country' AS country, sum(amount) AS revenue
  FROM 'events.json'
  GROUP BY country
  ORDER BY revenue DESC
) TO 'by_country.csv' (HEADER);

For very large inputs, DuckDB streams and processes data in chunks rather than loading everything into memory — complementing the streaming-parser techniques you'd otherwise write by hand, and it uses Apache Arrow columns internally for speed.

When to use DuckDB vs a JSON tool

  • Reach for DuckDB when you want to *aggregate, filter, join, or group* JSON data — analytical questions over many records or files.
  • Reach for a JSON tool when you want to *inspect, reshape, or validate* a single document — format it, test a JSONPath, or convert JSON to CSV for a quick look.

They're complementary: use the JSON toolkit to understand a payload's shape, then DuckDB to run analytics across thousands of them. For the database angle see JSON in databases and Postgres JSONB.

Frequently asked questions

Use read_json in the FROM clause: SELECT * FROM read_json('file.json'). For a local file you can also select directly from the path string, e.g. SELECT * FROM 'file.json'. DuckDB infers the schema and types automatically.

Yes. DuckDB reads newline-delimited JSON with the same read_json call (it detects the format) or explicitly via read_ndjson. Line-delimited files stream naturally, which is efficient for large logs and exports.

Use the arrow operators: -> returns a nested value as JSON for further navigation, and ->> extracts it as text. For example user ->> 'name' pulls the name string out of a nested user object.

Use UNNEST on the array column: SELECT id, UNNEST(items) AS item FROM 'data.json'. Each array element becomes its own row, which you can then group or aggregate.

Yes. Pass a glob pattern like read_json('logs/2026-*.ndjson') and DuckDB treats all matching files as one table. If they share a key, you can join across them like ordinary tables.

Try JSONKit — JSON Developer Tools

Format, validate, convert and diff JSON. Instant, accurate and 100% private.