The message is the unit of conversation
Every chat-based LLM API speaks the same core shape: a conversation is an array of message objects, each with a role and content. Understanding and designing this JSON well is the foundation of any chat or agent app.
[
{ "role": "system", "content": "You are a concise assistant." },
{ "role": "user", "content": "What's the capital of Japan?" },
{ "role": "assistant", "content": "Tokyo." }
]The three core roles
| Role | Purpose |
|---|---|
system | Sets behavior, persona, and rules. Usually first, often hidden from the user. |
user | What the human said. |
assistant | What the model replied — including tool-call requests. |
Some APIs add a tool (or function) role for returning tool results back to the model.
Content is not always a string
Simple messages use a plain string. Richer apps use an array of content parts so a single message can mix text, images, and other media:
{
"role": "user",
"content": [
{ "type": "text", "text": "What's in this chart?" },
{ "type": "image_url", "image_url": { "url": "https://example.com/chart.png" } }
]
}Designing your internal message type to allow either a string or an array of parts from day one saves a painful refactor when you add images, files, or audio.
Tool calls live in the messages too
When a model decides to call a tool, that request comes back as part of the assistant message, and your tool's result goes back as a tool message. The tool arguments and content are JSON strings — stringified JSON inside the message JSON — so always parse and validate them before acting.
Designing your app's message store
- Persist the full message array, including tool calls and results, so you can replay or audit a conversation.
- Store content as structured parts, not a flattened string, to keep images and citations attached.
- Add your own metadata (timestamps, message ids, token counts) at the top level — providers ignore unknown fields.
- Trim history to fit the context window, but keep the system message and recent turns. Summarize older turns rather than dropping them blindly.
A portable internal schema
{
"id": "msg_8821",
"role": "assistant",
"content": [{ "type": "text", "text": "Here's your summary." }],
"created_at": "2026-06-23T10:00:00Z",
"tokens": 42
}Map this to each provider's exact format at the API boundary, and keep your app logic provider-agnostic.