LLM Agent Architecture in Depth

The agent loop from first principles

Strip away the frameworks and every agent is the same loop. Build a message list, call the model with the list and the tool definitions, inspect the reply. If the reply contains tool calls, validate and run each one, append the results as tool messages, and call again. If it contains only text, that is the answer. Everything else, planning, reflection, multi-agent hand-offs, is a variation on what goes into the list or what the runtime does between calls.

The sketch below is provider-neutral: model.complete stands in for whichever chat API you use, and each major provider exposes an equivalent tool-calling interface with its own field names. What matters is where the guards sit.

class BudgetExceeded(Exception): pass

def run_agent(goal, model, registry, policy, limits, trace):
    messages = [system_prompt(registry), {"role": "user", "content": goal}]
    spent = Spend()                       # steps, tokens, dollars, wall clock
    seen = Counter()                      # repeated identical calls = stuck
    while True:
        spent.check(limits)               # raises BudgetExceeded before the call
        reply = model.complete(messages, tools=registry.schemas())
        spent.add(reply.usage)
        trace.llm(messages, reply)
        messages.append(reply.as_message())
        if not reply.tool_calls:
            return reply.text             # the model chose to stop
        for call in reply.tool_calls:
            key = (call.name, canonical_json(call.args))
            seen[key] += 1
            if seen[key] > 2:
                result = error("repeated identical call; change approach")
            else:
                result = dispatch(call, registry, policy, trace)
            messages.append(tool_message(call.id, result))

Four properties of this loop matter more than any prompt. The budget check runs before the call, so the last step cannot overshoot the limit by a whole model invocation. Every tool result, including errors, goes back to the model as data, which lets it recover from a typo instead of crashing. Repeated identical calls are detected mechanically, because a model that is stuck rarely notices it is stuck. And every call is traced, because you cannot debug a non-deterministic system from its final output.

The agent loop from first principles

Strip away the frameworks and every agent is the same loop. Build a message list, call the model with the list and the tool definitions, inspect the reply. If the reply contains tool calls, validate and run each one, append the results as tool messages, and call again. If it contains only text, that is the answer. Everything else, planning, reflection, multi-agent hand-offs, is a variation on what goes into the list or what the runtime does between calls.

The sketch below is provider-neutral: model.complete stands in for whichever chat API you use, and each major provider exposes an equivalent tool-calling interface with its own field names. What matters is where the guards sit.

class BudgetExceeded(Exception): pass

def run_agent(goal, model, registry, policy, limits, trace):
    messages = [system_prompt(registry), {"role": "user", "content": goal}]
    spent = Spend()                       # steps, tokens, dollars, wall clock
    seen = Counter()                      # repeated identical calls = stuck
    while True:
        spent.check(limits)               # raises BudgetExceeded before the call
        reply = model.complete(messages, tools=registry.schemas())
        spent.add(reply.usage)
        trace.llm(messages, reply)
        messages.append(reply.as_message())
        if not reply.tool_calls:
            return reply.text             # the model chose to stop
        for call in reply.tool_calls:
            key = (call.name, canonical_json(call.args))
            seen[key] += 1
            if seen[key] > 2:
                result = error("repeated identical call; change approach")
            else:
                result = dispatch(call, registry, policy, trace)
            messages.append(tool_message(call.id, result))

Four properties of this loop matter more than any prompt. The budget check runs before the call, so the last step cannot overshoot the limit by a whole model invocation. Every tool result, including errors, goes back to the model as data, which lets it recover from a typo instead of crashing. Repeated identical calls are detected mechanically, because a model that is stuck rarely notices it is stuck. And every call is traced, because you cannot debug a non-deterministic system from its final output.

The tool registry and dispatch layer

Tools are the agent's only way to read or change the world, so the tool registry is the most important design surface. Each tool needs a name, a description the model reads to decide when to use it, and a JSON Schema for its arguments. The model sees the name, description and schema; it never sees your implementation. Models pick tools by matching the task against descriptions, so a description is effectively a prompt. Say what the tool returns, when not to use it, and what units and formats arguments take.

def dispatch(call, registry, policy, trace):
    tool = registry.get(call.name)
    if tool is None:
        return error(f"unknown tool {call.name}; available: {registry.names()}")
    problems = validate(call.args, tool.schema)        # jsonschema, not eval
    if problems:
        return error(f"invalid arguments: {problems}")
    decision = policy.check(tool, call.args)           # scope, tenant, risk tier
    if decision.deny:
        return error(f"not permitted: {decision.reason}")
    if decision.needs_approval:
        return pending_approval(tool, call.args)       # human in the loop
    try:
        out = with_timeout(tool.timeout_s, tool.fn, **call.args,
                           idempotency_key=call.id)
        return ok(truncate(out, tool.max_result_tokens))
    except TransientError as e:
        return retry_with_backoff(tool, call, e)
    except Exception as e:
        trace.tool_error(call, e)
        return error(f"{type(e).__name__}: {short(e)}")

Keep the tool set small and orthogonal. Twenty overlapping tools give the model twenty chances to pick the wrong one; five well-described tools with clear boundaries usually beat them. Prefer tools that return compact, structured results over raw dumps, and truncate large outputs with a note saying how to page, because every byte returned is re-read on every later turn. Make write tools idempotent by passing the tool-call id as an idempotency key, so a retried call cannot refund an order twice. Tool protocols such as the Model Context Protocol standardise how tools are described and invoked across processes, but they do not change any of these rules; the tool use guide covers schema design in more detail.

Planning: ReAct, plan-and-execute and workflows

Planning is how the agent decides which step comes next, and there are three common shapes. In the ReAct pattern the model interleaves reasoning and a single action per turn and adapts after every observation. It is simple and robust to surprises, but it plans no further than one step ahead and can wander. In plan-and-execute a first call writes an explicit list of steps, an executor works through them, often with a cheaper model, and a re-planner revises the list when a step fails. It gives you a visible plan to log, show the user or approve, at the price of extra calls and a plan that can go stale. In a router or workflow design the control flow is ordinary code, and the model is used only at specific decision points: classify the request, extract fields, draft the reply.

Start with the most constrained shape that solves the problem. If the steps are known in advance, write them as code and call the model inside them; that is cheaper, testable and easier to reason about than an open loop. Reach for a free-running ReAct loop only when the path genuinely depends on what the tools return. Multi-agent designs, a supervisor delegating to specialists, are mostly a context-management technique: each specialist gets a short, focused context and tool set. They add hand-off latency and new failure modes, so earn them with measurements rather than adopting them by default.

Memory: what stays in the context window

The context window is the agent's only working memory, and it is both finite and expensive: every turn re-sends the entire history, so cost per step grows with step count and a twenty-step task pays for the early observations twenty times. Memory design is therefore about deciding what stays in the window.

• Working memory is the message list for the current task. Trim it actively: replace bulky tool outputs that have been used with one-line summaries, and keep the goal and constraints pinned at the top.
• Compaction kicks in near a token threshold: summarise older turns into a structured state block (facts established, steps done, open questions) and drop the raw turns. Summaries lose detail, so keep identifiers and numbers verbatim.
• Long-term memory persists across sessions: user preferences, past resolutions, learned facts. It is usually a store queried by similarity or by key, with retrieved items injected into the context builder. Treat writes to it as a privileged tool, because a poisoned memory is a persistent prompt injection.
• Prompt caching, where the provider supports it, makes a stable prefix (system prompt, tool schemas) cheaper on repeated turns. Order the context so stable material comes first and volatile material last.

The agent memory systems article goes deeper on retrieval and consolidation strategies.

Guardrails enforced in code

Guardrails are the runtime's answer to the fact that the model will sometimes be wrong, confused or manipulated. Text a tool returns, a web page, an email, a support ticket, can contain instructions, and the model cannot reliably tell those apart from yours. So never rely on the prompt for safety. Enforce it in code at the dispatch layer, where the model's opinion does not count.

Scope credentials to the session's user, not to the agent: if the agent acts for customer 42, its database token should only be able to see customer 42. Then a successful injection can only do what the user could do anyway. Permission boundaries for agents covers this pattern in depth.

Worked example: a refund agent, turn by turn

Take a support agent with three tools: get_order(id), get_policy(topic) and issue_refund(order_id, amount_cents, reason). The first two are read-only. The refund tool is a reversible write, and refunds above 100 dollars sit in the approval tier. Limits: 8 steps, 40,000 tokens, 60 seconds. The customer writes: "Order A-1042 arrived broken, I want my money back."

1. Turn 1: the model calls get_order({"id": "A-1042"}). The tool returns the total (149.00), the delivery date (six days ago) and the line items. About 2,100 tokens in, 60 out.
2. Turn 2: the model calls get_policy({"topic": "damaged items"}) and gets back: damaged items within 30 days get a full refund; a photo is required above 50 dollars.
3. Turn 3: no photo is on the ticket, so the model answers in text and asks for one. The loop ends; the session state is saved.
4. Turn 4, after the customer replies with a photo: the model calls issue_refund for 14,900 cents. Dispatch validates the schema, the policy check sees an amount above the tier threshold and returns pending_approval. A human approves in the queue, the tool runs with idempotency key equal to the call id, and the result comes back.
5. Turn 5: the model confirms the refund to the customer. Total: 5 model calls, one human approval, roughly 11,000 input tokens, because each turn re-sent the order record and the policy text.

Two design choices did the work. The policy text came from a tool rather than the system prompt, so it can change without a deploy and is only loaded when needed. And the refund threshold was enforced by the dispatch layer, so a confident model, or a cleverly worded ticket saying "approval already granted", could not skip it.

Failure modes

• Infinite or circular loops. The model retries the same failing call or alternates between two tools. Fix: repeated-call detection, step budgets, and error messages that suggest a different approach.
• Hallucinated tools and arguments. Calls to tools that do not exist, or IDs that were never returned. Fix: strict schema validation and errors that list valid options.
• Context rot. Long sessions bury the goal under tool output and quality drops. Fix: pin the goal, summarise used outputs, compact early.
• Prompt injection through tool results. Retrieved text tells the model to call a tool. Fix: least privilege, approval tiers, and never granting tools more scope than the user has.
• Premature stopping. The model declares success before verifying. Fix: a verification step in code (did the refund row appear?) or an explicit checker call before the final answer.
• Duplicate side effects. A timeout followed by a retry runs a write twice. Fix: idempotency keys on every write tool.
• Silent cost growth. A prompt change doubles average steps and nobody notices for a month. Fix: per-session cost tracking with alerts on the distribution, not just the mean.

Operations, evaluation and trade-offs

Operate an agent like any other distributed system, with the extra twist that the same input can take different paths. Trace each session as a tree: one span per model call (prompt and completion tokens, latency, model name, stop reason) and one per tool call (arguments, result size, error). Agentic tracing shows how to structure those spans. Track success rate, steps per task, tokens and cost per task, approval rate, and the share of sessions that hit a budget limit; a rising budget-hit rate is often the first sign of a regression.

Evaluate before you ship. Build a set of a few hundred real tasks with checkable outcomes, run it on every prompt, tool or model change, and compare pass rate and cost side by side. Because runs vary, execute each task several times and look at the pass rate, not a single pass. Agent evaluation at scale covers harness design.

LLM agent architecture as a runnable loop

LLM agent architecture is the split between a model that proposes the next action and a runtime that validates, permits and executes it, then feeds the result back for another turn. The loop above is a sketch; the script below is the same architecture with the model replaced by a fixed script, so it runs with plain Python and no API key. The scripted model makes two mistakes real models make, repeating a call and inventing a tool, before it answers.

import json

TOOLS = {"get_order": lambda id: {"id": id, "items": 2, "status": "delivered"}}
SCHEMA = {"get_order": {"id"}}

def scripted_model(messages):            # stands in for a real LLM call
    script = [{"tool": "get_order", "args": {"id": "A-1042"}},
              {"tool": "get_order", "args": {"id": "A-1042"}},
              {"tool": "refund", "args": {"id": "A-1042"}},
              {"text": "Order A-1042 was delivered; 2 items."}]
    return script[sum(m["role"] == "assistant" for m in messages)]

def run(goal, max_steps=6):
    messages, seen = [{"role": "user", "content": goal}], set()
    for step in range(1, max_steps + 1):
        reply = scripted_model(messages)
        messages.append({"role": "assistant", "content": reply})
        if "text" in reply:
            print(step, "answer:", reply["text"]); return
        name, args = reply["tool"], reply["args"]
        key = (name, json.dumps(args, sort_keys=True))
        if name not in TOOLS:
            result = {"error": f"unknown tool {name}; available: {list(TOOLS)}"}
        elif set(args) != SCHEMA[name]:
            result = {"error": "invalid arguments"}
        elif key in seen:
            result = {"error": "repeated identical call; change approach"}
        else:
            seen.add(key); result = TOOLS[name](**args)
        print(step, name, "->", result)
        messages.append({"role": "tool", "content": result})
    print("stopped: step budget exhausted")

run("Where is order A-1042?")

Output on Python 3.13:

1 get_order -> {'id': 'A-1042', 'items': 2, 'status': 'delivered'}
2 get_order -> {'error': 'repeated identical call; change approach'}
3 refund -> {'error': "unknown tool refund; available: ['get_order']"}
4 answer: Order A-1042 was delivered; 2 items.

The two guards that fired each turned a model mistake into a tool message instead of an exception, and the loop finished in four steps under its budget of six. To use a real model, replace scripted_model with your provider tool-calling API and map its reply into the same tool-or-text shape; the guards do not change.

What to do next

1. Write your agent loop as plain code first, with budgets, schema validation and tracing, before adopting a framework.
2. List every tool and assign it a risk tier; put irreversible actions behind approval.
3. Rewrite tool descriptions as if they were prompts: purpose, return value, when not to use it.
4. Add idempotency keys to every write tool and test a timeout-then-retry path.
5. Collect 100 real tasks with checkable outcomes and run them on every change, several times each.
6. Dashboard steps, tokens, cost and budget-hit rate per task, and alert on shifts in the distribution.
7. Red-team one tool result with an injected instruction and confirm the dispatch layer, not the prompt, stops it.