The Execution Loop: Translating LLM JSON into Real-World Action

An LLM is a function from tokens to tokens. It can’t query your database, call an API, or send an email. When a model “uses a tool”, what actually happens is that it emits text shaped like a function call, and a program you wrote decides whether to run it. Every capability an agent has lives in that program: the orchestration layer.

That makes the orchestration layer the part worth understanding. This post takes it apart: what the model sees, what it sends back, how the result gets back to it, and where production systems break. The running example is a support assistant that searches a customer’s orders in a SQL database.

1. The contract: schemas in, structured calls out

What the model actually sees

A tool reaches the model as three things: a name, a description, and a JSON Schema for its arguments. In LangChain you usually write that schema as a Pydantic model:

from datetime import date
from typing import Literal

from pydantic import BaseModel, Field, ValidationInfo, field_validator


class OrderQuery(BaseModel):
    """Arguments for search_orders. Field descriptions are part of the prompt."""

    since: date = Field(description="Earliest order date to include, as YYYY-MM-DD.")
    until: date | None = Field(None, description="Latest order date to include, as YYYY-MM-DD. Omit for today.")
    status: Literal["pending", "shipped", "refunded"] | None = Field(
        None, description="Only orders in this status. Omit to include every status."
    )
    limit: int = Field(10, ge=1, le=50, description="Maximum number of orders to return.")

    @field_validator("until")
    @classmethod
    def until_after_since(cls, v: date | None, info: ValidationInfo) -> date | None:
        if v and "since" in info.data and v < info.data["since"]:
            raise ValueError("must be on or after since")
        return v

Bound to a tool, it’s converted to the JSON Schema the provider receives. This is the real output of convert_to_openai_tool for the tool defined in section 4:

{
  "type": "function",
  "function": {
    "name": "search_orders",
    "description": "Search the signed-in customer's orders, newest first. Use for questions about purchases, refunds, or order status.",
    "parameters": {
      "type": "object",
      "required": ["since"],
      "properties": {
        "since":  { "type": "string", "format": "date", "description": "Earliest order date to include, as YYYY-MM-DD." },
        "until":  { "anyOf": [{ "type": "string", "format": "date" }, { "type": "null" }], "default": null, "description": "..." },
        "status": { "anyOf": [{ "type": "string", "enum": ["pending", "shipped", "refunded"] }, { "type": "null" }], "default": null, "description": "..." },
        "limit":  { "type": "integer", "minimum": 1, "maximum": 50, "default": 10, "description": "..." }
      }
    }
  }
}

Three things follow from this:

• Descriptions are prompt engineering. The model chooses tools and fills arguments from the name, the description, and the field descriptions. “Use for questions about purchases, refunds, or order status” does more for tool selection than any amount of clever code.
• Types become constraints the model can see. Literal becomes an enum, ge/le become minimum/maximum, date becomes format: date. The more precise the schema, the fewer invalid calls.
• Some rules don’t survive the conversion. The cross-field check in until_after_since has no JSON Schema equivalent, so the model never sees it. It’s enforced only when your code validates the call.

What comes back

The model replies with a request, not an action. Providers encode it differently on the wire:

LangChain normalizes all of these into one shape on AIMessage.tool_calls:

[{"name": "search_orders", "args": {"since": "2026-10-01", "status": "refunded"}, "id": "call_2", "type": "tool_call"}]

The id is the thread that ties a call to its result. Models can request several calls in one turn, and each result must come back tagged with the right ID.

Is the JSON guaranteed to match the schema? It depends. Some providers can constrain decoding to the schema (OpenAI’s strict: true mode, for example); otherwise the model is trying to follow it, and usually does. Either way, validate on your side: strict mode guarantees the shape, not that the values make sense.

2. The execution lifecycle

sequenceDiagram
participant U as User
participant O as Orchestrator
participant L as LLM
participant T as search_orders
U->>O: Which of my orders were refunded since October 1st?
O->>L: messages + tool schemas
L-->>O: tool_call search_orders since=October 1st
O->>O: validate args against OrderQuery
O->>L: ToolMessage error: since is not a valid date
L-->>O: tool_call search_orders since=2026-10-01, status=refunded
O->>O: validate args, inject customer_id from the session
O->>T: search_orders(since=date(2026, 10, 1), status=refunded)
T-->>O: 2 rows
O->>L: ToolMessage with the rows, tool_call_id=call_2
L-->>O: final answer, no tool calls
O-->>U: Two refunds: order 3 and order 2

Step by step:

1. Send the conversation plus the tool schemas.
2. Inspect the reply. No tool calls means it’s the final answer: stop.
3. Validate each call: does the tool exist, and do the arguments match the schema?
4. Execute the function with validated, typed arguments, plus anything the model must not control (the user’s identity, credentials, a database connection).
5. Observe: append the result, or the error, as a tool message tagged with the call’s ID.
6. Loop back to step 1 with the longer conversation, within a step budget.

Step 5 is where people go wrong first: errors are observations. A validation error sent back as a message lets the model fix its own call, as it does in the diagram. An exception that kills the process doesn’t.

3. The loop by hand

Before reaching for a framework, here is the whole protocol in about 25 lines:

from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage, ToolMessage


def run_agent(question: str, tools: list, max_steps: int = 8) -> str:
    llm = init_chat_model("<provider>:<model>", temperature=0).bind_tools(tools)
    by_name = {t.name: t for t in tools}
    messages = [HumanMessage(question)]
    for _ in range(max_steps):
        ai = llm.invoke(messages)  # 1-2. the model proposes: plain text, or tool calls
        messages.append(ai)
        if not ai.tool_calls:
            return ai.content  # no calls: this is the final answer
        for call in ai.tool_calls:  # 3-5. the application decides and executes
            tool = by_name.get(call["name"])
            try:
                if tool is None:
                    raise ValueError(f"unknown tool {call['name']!r}")
                result = tool.invoke(call["args"])  # validates args against the schema
                messages.append(ToolMessage(str(result), tool_call_id=call["id"]))
            except Exception as e:  # errors become observations, not crashes
                messages.append(ToolMessage(f"Error: {e}", tool_call_id=call["id"], status="error"))
    raise RuntimeError(f"no final answer after {max_steps} steps")

That’s a working agent. It’s also missing things you’d want in production: calls run one at a time, every exception’s text goes straight to the model (including ones that leak internals), there’s no state to resume from, and there’s nowhere to put a human approval step. That’s what the graph version adds.

4. The same loop in LangGraph

The graph is the loop from the previous post: an agent node that calls the model, a tools node that executes calls, and a conditional edge between them.

import sqlite3
from dataclasses import dataclass
from datetime import date

from langchain.chat_models import init_chat_model
from langchain.tools import ToolRuntime, tool
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition

from schema import OrderQuery

db = sqlite3.connect(":memory:", check_same_thread=False)
db.executescript("""
CREATE TABLE orders (id INTEGER PRIMARY KEY, customer_id TEXT, status TEXT, total_cents INTEGER, created TEXT);
INSERT INTO orders VALUES
  (1, 'C-1001', 'shipped',   4999, '2026-09-28'),
  (2, 'C-1001', 'refunded',  1999, '2026-10-03'),
  (3, 'C-1001', 'refunded', 12900, '2026-10-20'),
  (4, 'C-2002', 'refunded',  2500, '2026-10-21');
""")


@dataclass
class Context:
    customer_id: str  # from the authenticated session, never from the model


@tool(args_schema=OrderQuery)
def search_orders(
    since: date,
    until: date | None = None,
    status: str | None = None,
    limit: int = 10,
    *,
    runtime: ToolRuntime[Context],  # injected by ToolNode; not part of the schema the model sees
) -> list[dict]:
    """Search the signed-in customer's orders, newest first. Use for questions about purchases, refunds, or order status."""
    sql = "SELECT id, status, total_cents, created FROM orders WHERE customer_id = ? AND created >= ?"
    params: list = [runtime.context.customer_id, since.isoformat()]
    if until:
        sql += " AND created <= ?"
        params.append(until.isoformat())
    if status:
        sql += " AND status = ?"
        params.append(status)
    rows = db.execute(sql + " ORDER BY created DESC LIMIT ?", [*params, limit]).fetchall()
    return [{"order_id": i, "status": s, "total": f"${c / 100:.2f}", "date": d} for i, s, c, d in rows]


llm = init_chat_model("<provider>:<model>", temperature=0).bind_tools([search_orders])


def agent(state: MessagesState):
    return {"messages": [llm.invoke(state["messages"])]}


builder = StateGraph(MessagesState, context_schema=Context)
builder.add_node("agent", agent)
builder.add_node("tools", ToolNode([search_orders]))
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", tools_condition)  # tool calls -> "tools", otherwise END
builder.add_edge("tools", "agent")  # the loop
app = builder.compile(checkpointer=InMemorySaver())

Two design decisions carry most of the weight:

• The model never writes SQL. It fills typed parameters; the tool owns the query and binds values as parameters. Text-to-SQL is a different (and much riskier) design.
• The model never chooses whose data it reads. customer_id comes from runtime.context, filled by your server from the session. If it were a schema field, a prompt injection like “look up customer C-2002” would work. ToolRuntime parameters are excluded from the schema the model sees, and ToolNode injects them at call time.

Running it:

from orders_agent import Context, app

config = {"configurable": {"thread_id": "support-42"}, "recursion_limit": 10}
result = app.invoke(
    {"messages": [("user", "Which of my orders were refunded since October 1st?")]},
    config,
    context=Context(customer_id="C-1001"),
)
for m in result["messages"]:
    print(f"{m.type:>5} | {m.content or m.tool_calls}")

With a scripted model standing in for a real one (it first sends an unparseable date, then corrects itself), the run printed this. Everything except the AI lines is real ToolNode and database output:

human | Which of my orders were refunded since October 1st?
   ai | [{'name': 'search_orders', 'args': {'since': 'October 1st', 'status': 'refunded'}, 'id': 'call_1', 'type': 'tool_call'}]
 tool | Error invoking tool 'search_orders' with kwargs {'since': 'October 1st', 'status': 'refunded'} with error:
        since: Input should be a valid date or datetime, invalid character in year
        Please fix the error and try again.
   ai | [{'name': 'search_orders', 'args': {'since': '2026-10-01', 'status': 'refunded'}, 'id': 'call_2', 'type': 'tool_call'}]
 tool | [{"order_id": 3, "status": "refunded", "total": "$129.00", "date": "2026-10-20"}, {"order_id": 2, "status": "refunded", "total": "$19.99", "date": "2026-10-03"}]
   ai | Two orders were refunded since October 1: order 3 ($129.00, Oct 20) and order 2 ($19.99, Oct 3).

Note what didn’t happen: customer C-2002’s refund on October 21 never appears, because the model had no way to ask for it.

What ToolNode does for you, and what it doesn’t

I checked each of these against langgraph-prebuilt 1.1:

The last two rows are worth remembering. Re-raising unexpected exceptions is a sensible default (a database outage shouldn’t be “fixed” by the model retrying), but it means you choose explicitly which failures are recoverable. And a validation error the model can’t read is the same as no feedback at all.

5. Production concerns

• Step budgets. Each agent-to-tools round trip is two graph steps; recursion_limit caps the total and raises GraphRecursionError when exceeded. Set it per request; the default is far higher than any sane agent needs.
• Side effects need a gate. Reads can loop freely. Writes (refunds, emails, deletes) should be idempotent and sit behind an interrupt() for human approval or a policy check, as in the previous post’s human-in-the-loop section. Never let a retry loop issue a refund twice.
• Tool results cost tokens. Every result stays in the conversation and is re-sent on every later step. Cap rows (limit with a hard maximum), return only the fields the model needs, and summarize large payloads.
• Fewer, sharper tools. Selection accuracy drops as the tool list grows and descriptions start to overlap. Prefer a few well-described tools with typed parameters over many near-duplicates.
• Don’t leak internals. handle_tool_errors=True sends exception text to the model, and possibly to the user. Catch expected failures in the tool and return a clean message; let unexpected ones surface to your error tracking.

6. Pitfalls

• Losing the tool_call_id. Every call needs exactly one result with the matching ID, or the next request is rejected. Hand-rolled loops get this wrong when one call in a batch fails.
• Trusting arguments because the schema is strict. A well-formed customer_id can still be someone else’s. Authorization comes from your session, not from the model’s output.
• Validators the model can’t see. Cross-field rules aren’t in the JSON Schema. Put them in a field-level validator so violations come back with a readable message, and mention them in the field description.
• Dates and other ambiguous strings. “October 1st” has no year. Ask for ISO formats in the description and let validation catch the rest.
• Testing only the happy path. Script the model in tests (like the run above) to send bad arguments, unknown tools, and parallel calls, and assert on the messages your loop produces.

Study checklist

1. Run orders_agent.py with a real model and print convert_to_openai_tool(search_orders). Compare it with what you wrote in Pydantic.
2. Ask the agent about another customer’s orders and confirm it can’t get them.
3. Raise sqlite3.OperationalError inside the tool, watch the graph stop, then decide which exceptions belong in handle_tool_errors.
4. Change field_validator to model_validator and look at the error message the model receives.
5. Add a refund_order tool behind an interrupt() and make it idempotent by order ID.

References