NER & Sentiment Analysis: The JSON Behind Classic NLP

NLP existed before chatbots, and it has its own JSON shapes

Large language models get most of the attention now, but classic NLP tasks — named entity recognition, sentiment analysis, part-of-speech tagging — power a huge amount of production software that has nothing to do with chat: search relevance, content moderation, resume parsing, customer feedback analysis. These tasks predate LLMs by decades and have their own well-established JSON output conventions, used by spaCy, NLTK, Hugging Face pipelines, and cloud NLP APIs (AWS Comprehend, Google Natural Language). If your only NLP experience is prompting a chat model, these shapes will look refreshingly structured — no parsing prose, just typed fields.

Named Entity Recognition: spans, not sentences

NER identifies real-world entities — people, organizations, locations, dates — in text and reports exactly where in the string they occur, not just what they are:

{
  "text": "Ada Lovelace worked with Charles Babbage in London in 1843.",
  "entities": [
    { "text": "Ada Lovelace", "label": "PERSON", "start": 0, "end": 12 },
    { "text": "Charles Babbage", "label": "PERSON", "start": 26, "end": 41 },
    { "text": "London", "label": "GPE", "start": 45, "end": 51 },
    { "text": "1843", "label": "DATE", "start": 55, "end": 59 }
  ]
}

The start/end character offsets are the whole point — they let you highlight the exact substring in a UI, redact just that span for PII scrubbing, or link it to a knowledge base entry, all without re-searching the text for the entity string (which could appear multiple times with different meanings).

# spaCy — the de facto standard NLP library
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("Ada Lovelace worked with Charles Babbage in London in 1843.")

entities = [
    {"text": ent.text, "label": ent.label_, "start": ent.start_char, "end": ent.end_char}
    for ent in doc.ents
]

Common entity labels you'll see across libraries: PERSON, ORG (organization), GPE (geopolitical entity — countries, cities), DATE, MONEY, PERCENT — though exact label sets differ slightly between spaCy, Stanford NLP, and cloud providers.

Sentiment analysis: a score, not just a label

Modern sentiment JSON gives you more than "positive/negative" — a continuous score plus a magnitude (how strongly the text expresses sentiment, regardless of direction) and often per-sentence breakdowns:

{
  "document": {
    "sentiment": "positive",
    "score": 0.87,
    "magnitude": 1.2
  },
  "sentences": [
    { "text": "The checkout flow is fast.", "sentiment": "positive", "score": 0.9 },
    { "text": "But the search is completely broken.", "sentiment": "negative", "score": -0.8 }
  ]
}

score typically ranges from -1 (very negative) to 1 (very positive); magnitude is intensity independent of polarity — a long, emotionally neutral document and a short, emotionally flat one both score near-zero magnitude, while a short but intense complaint scores high magnitude despite being brief. The document-level score alone often lies: a review that's half glowing and half furious can average out to a deceptively neutral document score — the sentences breakdown is what reveals the real, mixed sentiment.

Part-of-speech tagging and dependency parsing

POS tagging labels every token's grammatical role — useful for grammar checking, keyword extraction, and as a preprocessing step for other NLP tasks:

{
  "tokens": [
    { "text": "Ada", "pos": "PROPN", "tag": "NNP", "dep": "nsubj" },
    { "text": "codes", "pos": "VERB", "tag": "VBZ", "dep": "ROOT" },
    { "text": "quickly", "pos": "ADV", "tag": "RB", "dep": "advmod" }
  ]
}

pos is the universal, cross-language part of speech (PROPN = proper noun, VERB, ADV); tag is the finer-grained, language/treebank-specific tag (Penn Treebank tags like NNP, VBZ for English); dep is the token's grammatical relationship to its head word in a dependency parse. Most applications only need pos; tag and dep matter for grammar-aware tooling.

Cloud NLP APIs: the same ideas, vendor-specific fields

AWS Comprehend, Google Natural Language, and Azure Text Analytics all return broadly the same shapes with different field names — a useful reminder that the *concepts* (entities as spans, sentiment as a score) are universal even when the exact JSON keys aren't:

// AWS Comprehend — DetectEntities response (abridged)
{
  "Entities": [
    { "Text": "Ada Lovelace", "Type": "PERSON", "Score": 0.99, "BeginOffset": 0, "EndOffset": 12 }
  ]
}

Using these shapes in a real pipeline

A common architecture: run NER over support tickets to auto-tag mentioned products, run sentiment analysis to prioritize angry customers, and store both alongside the original text:

{
  "ticketId": "tk_4471",
  "text": "My Pro plan renewal failed and support hasn't responded in 3 days.",
  "entities": [
    { "text": "Pro plan", "label": "PRODUCT", "start": 3, "end": 11 }
  ],
  "sentiment": { "label": "negative", "score": -0.82, "magnitude": 1.4 },
  "priority": "high"
}

priority: "high" here is derived downstream from the sentiment score plus the negative magnitude crossing a threshold — a pattern worth generalizing: NLP output JSON is rarely the end product, it's an intermediate signal that business logic acts on.

Validating and debugging NLP JSON

1. Inspect entity spans visually. Paste a large NER output into JSONKit's JSON Explorer to navigate dozens of entities without losing your place.
2. Diff pipeline versions. When you upgrade a model or library version, run the same input through both and JSON Diff the outputs — entity boundaries and labels can shift subtly between model versions.
3. Define a schema for your pipeline's output. If entities feed a database or downstream service, lock the shape down with a JSON Schema so a library upgrade that adds or renames a field doesn't silently break consumers.