Multi-Agent Orchestration Architecture in Depth

Choosing a control topology

The first decision is who decides what happens next. In a supervisor topology, one agent plans, delegates and merges results; it holds the full plan and workers see only their piece. In a graph or workflow topology, the order of steps is written in code and agents fill in nodes; the model never chooses the next step. In a handoff or swarm topology, each agent can transfer control to another, so the path emerges at run time. In an event-driven or choreographed topology, agents subscribe to events on a bus and react without a central coordinator.

These sit on a spectrum from predictable to flexible. Workflows are easiest to test and reason about, so use them whenever the steps are known. Supervisors suit tasks whose decomposition depends on the input. Handoffs suit conversational products where a user moves between specialists. Choreography suits long-running systems where agents are owned by different teams. Most production systems are hybrids: a workflow skeleton with a supervisor inside one flexible node. Frameworks such as LangGraph implement several of these; see LangGraph for agent orchestration. Whatever you pick, the control-plane components below are the same.

Choosing a control topology

The first decision is who decides what happens next. In a supervisor topology, one agent plans, delegates and merges results; it holds the full plan and workers see only their piece. In a graph or workflow topology, the order of steps is written in code and agents fill in nodes; the model never chooses the next step. In a handoff or swarm topology, each agent can transfer control to another, so the path emerges at run time. In an event-driven or choreographed topology, agents subscribe to events on a bus and react without a central coordinator.

These sit on a spectrum from predictable to flexible. Workflows are easiest to test and reason about, so use them whenever the steps are known. Supervisors suit tasks whose decomposition depends on the input. Handoffs suit conversational products where a user moves between specialists. Choreography suits long-running systems where agents are owned by different teams. Most production systems are hybrids: a workflow skeleton with a supervisor inside one flexible node. Frameworks such as LangGraph implement several of these; see LangGraph for agent orchestration. Whatever you pick, the control-plane components below are the same.

The capability registry and tiered routing

Routing by capability only works if capabilities are written down. A registry entry describes each agent: the skills it accepts, the model tier it runs on, the tools it may call, its maximum input size, its cost per call estimate and its current health. The supervisor proposes a skill for a subtask; the router, which is code, picks the cheapest healthy agent that has the skill and can accept the input size. Keeping this in code rather than in the supervisor's prompt means you can change models, add an agent or take one out of service without editing prompts, and the supervisor cannot route a task to an agent that does not exist.

REGISTRY = {
    "researcher": {"skills": {"web_research", "doc_lookup"}, "tier": "small", "tools": ["search", "fetch_doc"],
                   "max_input_tokens": 32_000, "healthy": True},
    "analyst":    {"skills": {"sql_analysis"}, "tier": "small", "tools": ["sql_readonly"],
                   "max_input_tokens": 32_000, "healthy": True},
    "writer":     {"skills": {"draft_report"}, "tier": "medium", "tools": [],
                   "max_input_tokens": 64_000, "healthy": True},
    "reviewer":   {"skills": {"review"}, "tier": "large", "tools": [], "max_input_tokens": 128_000, "healthy": True},
}
TIER_COST = {"small": 1, "medium": 4, "large": 12}       # relative cost units, set from your price sheet

def route(skill, input_tokens):
    candidates = [(TIER_COST[a["tier"]], name) for name, a in REGISTRY.items()
                  if skill in a["skills"] and a["healthy"] and input_tokens <= a["max_input_tokens"]]
    if not candidates:
        raise NoAgent(skill)                              # surfaces to the supervisor as a planning error
    return min(candidates)[1]

Tiering is where most of the cost saving in multi-agent systems comes from. Retrieval, extraction and formatting rarely need the largest model; synthesis and review often do. Escalation is a routing rule too: if a small-tier result fails validation twice, route the retry to the next tier rather than looping.

The handoff contract

When the supervisor delegates, it sends a handoff packet, and the shape of that packet decides how well the system works. Sending the whole conversation is the common mistake: the worker pays for tokens it does not need, sees instructions meant for other agents, and inherits any injected text the supervisor saw. Sending a one-line instruction is the opposite mistake: the worker lacks the constraints it needs and guesses.

A good packet is typed and small. It states the objective, the acceptance criteria, the constraints, a budget, and references to artifacts rather than their contents. The worker fetches only the artifacts it needs from the artifact store, and returns a result in the same shape: a status, a short summary, and references to the artifacts it produced.

from dataclasses import dataclass, field

@dataclass(frozen=True)
class Handoff:
    goal_id: str
    task_id: str
    skill: str
    objective: str                     # one or two sentences, written by the supervisor
    acceptance: list[str]              # checks the result must pass
    constraints: dict                  # e.g. {"region": "EMEA", "quarter": "2026-Q3"}
    inputs: list[str]                  # artifact ids, not artifact contents
    budget_tokens: int
    deadline_s: int
    trace_parent: str                  # propagated trace context
    scopes: tuple[str, ...] = field(default=())   # permissions delegated for this task only

@dataclass(frozen=True)
class Result:
    task_id: str
    status: str                        # "done" | "failed" | "needs_human"
    summary: str                       # under 150 words, what the supervisor reads
    outputs: list[str]                 # artifact ids
    tokens_used: int

The supervisor reads summaries, not artifacts, which keeps its context small over a long run. When a step needs a person, the same contract applies; agent-to-human handoff patterns describes the context packet for that case.

Shared state with clear ownership

Agents need shared state: facts discovered, intermediate tables, drafts. A blackboard, a structured store every agent can read, is the usual answer, but a blackboard where everyone writes everything turns into a race. Two rules keep it sane. Every key has exactly one owning agent or task, and only the owner writes it. And every write is versioned, with compare-and-set, so an agent working from a stale read fails loudly instead of overwriting newer work.

def write(board, key, value, writer, expected_version):
    # One conditional write in the store (a transaction or conditional put), never read-then-put in app code.
    ok = board.put_if(key, value, owner=writer, expect_owner=writer, expect_version=expected_version)
    if not ok:
        raise StaleWrite(key)                             # wrong owner or newer version: re-read, re-plan

Large outputs go in the artifact store, which is immutable: a new draft is a new artifact ID, and the blackboard key points at the latest one. That gives you a full history for free and makes retries idempotent, because writing the same artifact twice produces the same content hash.

Identity and least privilege per agent

In a single-agent system it is tempting to give the agent the user's credentials. In a multi-agent system that multiplies the problem: every worker can do everything the user can, and a prompt injection in the researcher's web results can drive the analyst's database tool. Give each agent its own workload identity and let tools be reached only through a gateway that checks two things: what this agent type may ever do, and what this task was delegated. The effective permission is the intersection of the agent's own rights, the user's rights and the task's scopes, which is the defence against the confused-deputy problem described in confused deputy, in depth.

Least privilege also shapes the topology. The researcher reads untrusted web pages, so it gets no write tools and no private data. The analyst touches private data, so it gets no internet. The writer gets no tools at all. An injected instruction in a web page can then corrupt a summary, which review can catch, but it cannot reach the database.

Tracing one goal across many agents

When a multi-agent run produces a wrong answer, the question is which agent introduced the error and what it saw. Answering it requires one trace per goal, a span for every agent step, model call, tool call and handoff, and the trace context carried inside the handoff packet so spans from different processes join up. OpenTelemetry's trace-context propagation does this across services, and its generative-AI semantic conventions are a reasonable starting point for attribute names. Record on each span the agent, tier, tokens in and out, artifact IDs read and written, and the validation outcome. Two dashboards pay for themselves immediately: cost per goal broken down by agent, and the rate at which each agent's results fail validation.

In process or as services

Agents can run in one process as functions sharing a scheduler, or as separate services behind a queue. In process is simpler and faster, and right for one team's workflow. Separate services let agents scale, deploy and fail independently, and let other teams own them; they need the handoff packet as a wire format, idempotent task IDs because queues redeliver, and a timeout on every await. Across organisational boundaries, open protocols such as A2A define agent discovery and task exchange so that the two sides need not share a framework.

Worked example: a regional sales report

Take a request for a Q3 regional sales report with a budget and a five-minute deadline. The supervisor, on the large tier, writes a plan of four tasks: gather market context (web_research), compute quarter-over-quarter and year-over-year deltas by region (sql_analysis), draft the report (draft_report), and review it (review). The first two have no dependency and run in parallel. To see why tiering matters, use illustrative prices of $0.25 and $1.25 per million input and output tokens for the small tier, $1 and $5 for medium, and $3 and $15 for large; substitute your own price sheet.

The same token volumes on the large tier throughout would cost about $1.14, roughly four times as much. In this run the reviewer step was skipped because the plan's acceptance checks, which are code, passed. During the run the analyst's first SQL result failed an acceptance check because the totals did not reconcile with the finance table; the router retried on the same tier with the error attached, and the second attempt passed. A researcher result carrying an instruction-like sentence from a web page was flagged in its summary, and the writer's packet referenced only the analyst's tables and the researcher's cited facts, so the instruction never reached an agent with tools.

Failure modes

• Context bleed. Whole transcripts are forwarded to every worker. Fix: typed packets with artifact references.
• Router in the prompt. The supervisor invents an agent or sends everything to the largest model. Fix: registry-based routing in code.
• Lost updates. Two agents overwrite the same blackboard key. Fix: ownership plus compare-and-set.
• Permission creep. Every agent holds the user's token. Fix: per-agent identity and task scopes at a tool gateway.
• Ping-pong handoffs. Agents keep transferring control to each other. Fix: a hop limit per goal and a handoff graph that forbids cycles without supervisor approval.
• Invisible failures. A worker returns done with an empty artifact. Fix: acceptance checks run by code on every result.
• Orphan traces. Spans from remote agents do not join. Fix: propagate trace context in the packet.

What to do next

1. Choose the topology per workflow: code-defined graphs wherever the steps are known.
2. Write a capability registry with skills, tier, tools and limits, and route from it in code.
3. Define the handoff and result packets as types; pass artifacts by ID.
4. Give every blackboard key an owner and enforce versioned writes.
5. Issue each agent its own identity, put tools behind a gateway and intersect agent, user and task permissions.
6. Propagate trace context through every handoff and build cost-per-goal and validation-failure dashboards.
7. Re-run your largest workflow with tiered routing and compare cost and quality against a single large-model agent.