ML Experiment Tracking: The JSON Behind MLflow & Model Registries

Reproducibility is a JSON logging problem

"Which hyperparameters produced our best model, and can we reproduce it?" is a question every ML team eventually needs to answer under pressure — a production incident, a regulator asking how a model was validated, or just wanting to beat last quarter's accuracy. Experiment tracking tools (MLflow, Weights & Biases, Neptune) answer it by logging every training run as structured JSON: the exact hyperparameters, metrics over time, and artifacts produced — turning "I think we used learning rate 0.01" into a queryable fact.

The anatomy of a tracked run

A single training run's record captures parameters (inputs, fixed for the run), metrics (outputs, often logged repeatedly over time), and metadata:

{
  "run_id": "a1b2c3d4",
  "experiment_id": "42",
  "status": "FINISHED",
  "start_time": 1751800000000,
  "end_time": 1751803600000,
  "params": {
    "learning_rate": "0.001",
    "batch_size": "32",
    "model_type": "resnet50",
    "epochs": "20"
  },
  "metrics": {
    "accuracy": 0.94,
    "loss": 0.18,
    "val_accuracy": 0.91
  },
  "tags": {
    "mlflow.user": "ada",
    "mlflow.source.git.commit": "8f3a1c2"
  }
}

Notice params values are strings even for numbers ("0.001" not 0.001) in MLflow's actual API — a quirk worth knowing, since comparing runs by parameter value in your own tooling needs to parse them back to numbers first rather than relying on JSON's native number type.

Metrics are logged as a time series, not a single value

Unlike params (fixed per run), metrics are typically logged repeatedly during training — once per epoch or batch — so tracking systems store each metric as a series of timestamped points, not a single final number:

{
  "key": "val_accuracy",
  "history": [
    { "step": 1, "value": 0.62, "timestamp": 1751800100000 },
    { "step": 2, "value": 0.74, "timestamp": 1751800200000 },
    { "step": 3, "value": 0.91, "timestamp": 1751800300000 }
  ]
}

This is what powers the training curves you see in MLflow's or W&B's UI — each point in the chart is one entry in this JSON series, which is exactly why a stalled or diverging metric is visible as a shape in the curve rather than requiring you to eyeball a wall of log lines.

Comparing runs: the whole point of tracking

The real value of tracking JSON isn't any single run — it's querying across many runs to find what actually drove a result:

[
  { "run_id": "a1b2", "params": { "learning_rate": "0.001" }, "metrics": { "val_accuracy": 0.91 } },
  { "run_id": "c3d4", "params": { "learning_rate": "0.01" },  "metrics": { "val_accuracy": 0.87 } },
  { "run_id": "e5f6", "params": { "learning_rate": "0.0001" }, "metrics": { "val_accuracy": 0.89 } }
]

Because every run is the same JSON shape, this becomes a simple sort-and-filter problem — "show me every run with val_accuracy > 0.9, sorted by learning_rate" — rather than manually cross-referencing scattered notebook cells or Slack messages.

The model registry: versioning the artifact, not just the code

Once a run produces a model worth keeping, it's registered — the JSON shape shifts from "how was this trained" to "what is this model's lifecycle status":

{
  "name": "fraud-detector",
  "version": "7",
  "source_run_id": "a1b2c3d4",
  "current_stage": "Production",
  "creation_timestamp": 1751803600000,
  "description": "Retrained on Q2 2026 data with class rebalancing"
}

current_stage (commonly None, Staging, Production, Archived) is what lets a deployment pipeline query "give me whatever model is currently marked Production" as a stable API, decoupled from any specific run ID — the model can be promoted or rolled back by updating this one field, without touching deployment code.

Logging a run in practice

import mlflow

with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.001)
    mlflow.log_param("batch_size", 32)
    for epoch in range(20):
        # ... train one epoch ...
        mlflow.log_metric("val_accuracy", val_acc, step=epoch)
    mlflow.log_artifact("confusion_matrix.png")

Behind this Python API, MLflow is writing exactly the JSON structures shown above to its tracking store (a local file store, or a database-backed server) — the SDK exists to save you from hand-authoring that JSON on every run.

Querying tracked experiments programmatically

from mlflow.tracking import MlflowClient

client = MlflowClient()
runs = client.search_runs(
    experiment_ids=["42"],
    filter_string="metrics.val_accuracy > 0.9",
    order_by=["metrics.val_accuracy DESC"],
)
for run in runs:
    print(run.data.params, run.data.metrics)

Under the hood this is a query over exactly the JSON shape shown earlier — the client library exists to spare you from writing that filter logic by hand against raw tracking files.