Why AI Agents Need JSON: From Tool Calling and Function Calling to MCP

Symptom: The agent chats well, then tool arguments fail validation, retries duplicate an order, or an MCP server is online yet no tools appear.
Fastest fix: Treat JSON as the only contract between the model and tools: constrain inputs with JSON Schema, return results in stable fields, and use MCP only to make that contract discoverable across processes.

This article is for developers, platform engineers, and architects connecting a language model to real systems. The useful question is not another buzzword. It is how the same JSON object moves through Function Calling, Tool Calling, and MCP, and which hop actually breaks the chain.

If you are still choosing a framework rather than a protocol, start with How to Choose an AI Agent Development Framework. When a tool actually executes, isolation matters as well; see AI Sandbox vs Docker vs a virtual file system.

Why an AI Agent cannot live without JSON

Models generate language. Production systems accept structured commands. The moment an agent must check inventory, update a ticket, run tests, or drive a browser, intent has to become data that can be validated, rejected, and audited. JSON sits in that seam: humans can read it, machines can parse it, and HTTP, logs, queues, and databases already speak it.

The real advantage is verifiability. Prose cannot tell a gateway that order_id must be a string, amount a positive number, and currency an enum. JSON Schema can. OpenAI’s Function Calling guide defines function parameters as JSON Schema so the model is steered toward legal structure while sampling, instead of hoping a later parser will guess correctly. Read the OpenAI Function Calling guide

JSON Schema also encodes types, required fields, enums, and additionalProperties. In production those rules are the first gate against invented fields, strings stuffed into numbers, or a missing idempotency key. Read the JSON Schema documentation

Think in three layers:

• Conversation: the user says “make that Shanghai warehouse order expedited.”
• Contract: the model emits {"name":"update_order","arguments":{"order_id":"A938","priority":"expedited"}}.
• Execution: your code validates the schema, checks permission, calls the business API, and feeds a JSON result back to the model.

Without the contract layer, an agent is a chat window. With it, the agent can become a governed executor.

⚠️ Rule of thumb: if tool arguments are still “free text plus regex,” you do not have an agent protocol. You have prompt luck. Freeze the schema before you orchestrate many tools.

Function Calling: the model learns to fill a form

Function Calling means the model may, before answering the user, choose a declared function and emit arguments that match a schema. The happy path is five steps, and each step can drop a field:

1. Register tools on the request: name, description, parameters as JSON Schema.
2. Decide whether to call a function or answer directly.
3. Fill the form via tool_calls, with arguments as a JSON string or object.
4. Execute in your app: parse, validate, authorize, call the real system.
5. Return a tool message so the model can write the user-facing reply.

Descriptions are routing signals, not marketing copy. “Look up weather” will compete with “look up shipping.” “Call only when the user supplied a city name; never use this for historical climate” reduces misfires. Field descriptions inside the schema are also prompts, so “order_id is the business order number, not a payment intent id” belongs in JSON Schema, not only in the system prompt.

Strict mode and Structured Outputs upgrade “looks like JSON” into “matches the schema.” JSON mode only promises braces that parse. It will not catch missing fields or wrong types. You still need a second business check: schema-valid is not domain-valid. Negative amounts, unknown warehouse codes, and unauthorized order IDs must fail in your code.

A minimal executable contract

The schema below freezes enums and an idempotency key. The model may refuse to call the tool, but it cannot invent extra warehouse fields or omit idempotency_key.

{
  "type": "function",
  "name": "update_shipment",
  "description": "Call only when the user explicitly asks to change delivery priority.",
  "strict": true,
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string", "description": "Business order id"},
      "priority": {"type": "string", "enum": ["standard", "expedited"]},
      "idempotency_key": {"type": "string"}
    },
    "required": ["order_id", "priority", "idempotency_key"],
    "additionalProperties": false
  }
}

On the execution side the order is: JSON parse → schema validate → permission check → idempotency lookup → business API → return {"ok":true,"shipment_id":"..."} or {"ok":false,"error":"not_found"}. Never dump stack traces or secrets into the tool result.

Tool Calling: from one function to a tool surface

Teams quickly needed more than home-grown functions: search, code interpreters, browsers, computer use, file retrieval. APIs moved from a single function_call field to a unified tools / Tool Calling surface. The name changed. The JSON contract did not: name, schema, invocation record, result round-trip.

Anthropic documents the same idea as Tool use, with input_schema in JSON Schema. The model emits a tool_use block of structured arguments; your app returns tool_result. Read Anthropic’s Tool use documentation

Tool Calling adds three engineering problems that JSON still has to carry:

• Parallel calls in one model turn. Decide which tools are safe to run together (reads) and which must be serial (lock stock, then charge).
• Approval for high-risk tools: return a JSON “pending” state instead of executing.
• Observability: traces must reconstruct tool name, argument summary, latency, and error code, not a paragraph of model prose.

Failures also change shape. The model can pick the right tool and still smuggle a dangerous payload inside a string field. JSON Schema catches types; it does not catch semantic attacks. Dangerous tools need allowlists, timeouts, and a sandbox in the execution layer.

MCP: make the JSON contract pluggable

Function and Tool Calling describe how a model talks to tools inside one API request. MCP (Model Context Protocol) describes how tools are discovered, reused by many hosts, and run in a separate process. The spec uses JSON-RPC 2.0 for initialize, list tools/resources/prompts, call a tool, and stream notifications. Read the MCP specification

The path you will debug looks like this:

1. A host (IDE, agent runtime, chat app) starts as an MCP client.
2. It connects over stdio, SSE, or Streamable HTTP.
3. initialize negotiates protocol version and capabilities.
4. tools/list returns each tool with an inputSchema that is still JSON Schema.
5. The model still does Tool Calling. When it selects an MCP tool, the client sends tools/call with JSON arguments.
6. The server returns content; the client turns it into a tool result for the model.

MCP does not replace JSON. It turns a hard-coded tools array into a runtime catalog. The same git server, browser automation, or ticket API can be shared by Cursor, Claude Desktop, or your own agent, as long as they speak MCP.

That is why “MCP is installed but tools never show up” is almost always a contract bug: initialize failed, list returned empty, the schema is incompatible with a provider’s strict mode, tools/call omitted a required field, or the server answered with prose instead of a JSON-RPC error.

End-to-end flow: one sentence to MCP execution

Stacked together, a real task looks like this:

1. The user types natural language.
2. The runtime builds messages plus local tools plus remote tools from MCP tools/list.
3. The model performs Tool Calling and emits JSON arguments.
4. The runtime routes: local function vs JSON-RPC tools/call.
5. Results return as JSON (or MCP content) into the conversation.
6. The model calls again, asks the user, or answers.

One boundary is easy to miss: the model never holds your database connection. It only emits JSON. Who parses it, which machine runs it, whether SSH or VNC is involved, and whether outbound network is allowed, are runtime policy. An agent or MCP server on a cloud Mac uses the same protocol as a laptop. The permission plane is different.

Ignore product names for a moment. If tools only serve one app, Function/Tool Calling is enough. If several hosts must reuse them, an MCP server is justified. Without schema and validation, MCP only moves the mess into another process.

When JSON breaks, the whole agent chain breaks

Production incidents are usually a deformed contract, not a “dumb” model. Debug hop by hop:

• Output is not valid JSON: strict mode off, the prompt asked for an explanation first, or arguments arrived inside a Markdown fence. Fix: use the structured tool channel; never regex arguments out of assistant prose.
• JSON parses but fails schema: missing required fields, bad enums, extra properties. Return the validation error as a tool result and cap retries.
• Schema passes, business fails: unknown order, low stock, permission denied. Return a stable error code, not a 500 HTML page.
• Retries double-write: a timeout plus a second model call. Require idempotency_key and dedupe in the executor.
• MCP list and call drift: list shows a tool, call uses another name, or the schema changed after a hot reload. Version the tool name plus schema hash and re-initialize on change.
• Secrets in JSON logs: full arguments landed in a shared trace. Log field names and hashes; redact tokens and personal data.

Turn those failures into one acceptance scenario: force a timeout, a schema error, and a permission deny. Watch whether the agent loops, mutates arguments at random, or stops for a human. Systems that fail this test should not touch payments, fulfillment, or production databases.

Ship only after this JSON contract checklist

• [ ] Every tool has a name, description, JSON Schema, and additionalProperties: false.
• [ ] High-risk fields use enums, formats, and max length, not free-form strings.
• [ ] Writes carry an idempotency key; timeout and retry live in runtime code, not in the prompt.
• [ ] Tool results use a stable shape: ok, error, data. No raw stack traces.
• [ ] Parallel tool calls are grouped by side effects; payments, email, and deletes default to serial.
• [ ] MCP initialize, tools/list, and tools/call have timeouts, auth, and error codes.
• [ ] Hosts refetch schemas after the catalog changes instead of caching stale JSON.
• [ ] Traces reconstruct the call chain without reconstructing secrets.
• [ ] A frozen prompt always hits the same tool name and the same required field set.

Only then consider multi-agent graphs or heavier workflows. An unstable protocol just copies bad JSON faster.

FAQ

Why not let the model write natural language for a program to parse?

Natural language has no required fields or types. A gateway cannot reliably map “make it urgent” to an enum, or reject extra fields. JSON Schema turns parsing into validation, and failures can be returned as structured errors instead of regex luck.

Is Tool Calling different from Function Calling in practice?

For application developers the core is the same: the model fills a JSON form, your code runs. Tool Calling is a wider surface: built-in search or code execution, parallel calls, and a unified result channel. When migrating, align schemas and message roles first; do not stall on legacy field names.

When do I need MCP instead of a hard-coded tools array?

When the same tools must be reused by multiple hosts, isolated in their own process, or published by another team. If a tool lives and dies with one internal service, Function/Tool Calling is simpler and avoids extra transport and auth.

The model sometimes emits bad JSON. Retry or switch protocols?

Enable strict structured output and stop extracting arguments from prose. If failures stay high, inspect schema depth, conflicting descriptions, and tool count. Moving to MCP will not fix this: MCP arguments are still JSON.

If you currently share a laptop for agents, schema regression tests break whenever the environment drifts, and MCP processes fight daily-dev ports. A Kvmjet bare-metal Mac cloud host keeps a fixed macOS image so you can reproduce the same Tool Calling or MCP failure over SSH or VNC. Public billing is daily through quarterly, which is enough to stabilize the JSON contract before you commit to a long lease.

Take the checklist to the Kvmjet help center, confirm always-on runtime, permission isolation, and remote access, then decide whether the MCP server belongs on a laptop, in a rack, or on a cloud Mac.