Supervisor or Swarm: When Multi-Agent Systems Actually Beat One Agent

Multi-agent systems are easy to draw and expensive to run. A supervisor that delegates to specialists, or a swarm of peers that hand the conversation to one another, looks like a clean division of labour on a whiteboard. In production each arrow is a model call, each handoff is a chance to lose context, and each extra agent is another prompt to maintain. Sometimes that cost buys a large gain in accuracy or latency. Often a single agent with a well-chosen toolset would have done better for less.

This guide works through when splitting pays: the arithmetic of tool-selection accuracy as a catalog grows, the call and token overhead of a supervisor, the routing errors that make swarms brittle, and the latency gain from running independent subtasks in parallel. It covers how frameworks such as LangGraph and the OpenAI Agents SDK express these patterns, the failure modes that research on multi-agent systems has catalogued, and the metrics that tell you which architecture is earning its keep. The embedded lab runs synthetic tasks through all three architectures so you can find the cheapest one that meets a target.

The baseline: one agent and the cost of a big tool list

A single agent sends every tool definition with every model call. Definitions are not small: a name, a description and a JSON schema typically take a hundred tokens or more. At an illustrative 120 tokens each, a catalog of 120 tools adds about 14,400 tokens to every call before any conversation. That is a direct cost, and it also makes the choice harder. Model accuracy at picking the right tool falls as the list grows and as tools overlap in purpose; the companion article on tool routing covers the evidence.

The lab models tool-selection accuracy with an illustrative curve, a(t) = 0.995 − 0.08 · ln(t / 16), minus a small penalty for long contexts. At 24 tools that is about 0.96; at 120 tools about 0.83. A wrong pick is not always fatal: the lab assumes the agent notices and recovers 75% of the time, at the price of an extra step. The probability that one step ends the task is then (1 − a) × 0.25, about 4% at 120 tools, and a task of six steps succeeds with probability about (1 − 0.042)^6 ≈ 0.77. A specialist holding a sixth of those tools, 20, has a ≈ 0.98 and fails a step less than 1% of the time.

That gap is the whole case for splitting by domain. If your agent has a dozen tools, it does not exist, and a single agent is almost always the right answer. If it has a hundred or more, or its tools fall into groups with confusable names, the case becomes real.

What a supervisor costs and what it buys

In the supervisor pattern a lead agent sees short descriptions of each specialist, decides which one handles the next piece of work, passes it a task description, and receives a result. The specialist sees only its own tools. Per subtask, the overhead is at least one supervisor call to route, one specialist call to read the handoff and plan, and a share of the final call where the supervisor assembles the answer. For a task touching two domains with three tool steps each, a single agent makes about seven calls; a supervisor architecture makes about eleven.

What it buys is accuracy and smaller prompts. Each specialist call carries 20 tool definitions instead of 120, so even with more calls the tokens per task can fall: in the lab's two-domain, 120-tool setup the supervisor uses roughly 70k tokens per task against 160k for the single agent, and success rises from about 80% to about 92%. These are calculated by the model, not measured, but the mechanism is general: the supervisor pays a fixed coordination tax per subtask and saves a tax proportional to catalog size on every call.

The supervisor also adds its own failure: misrouting. If it sends a subtask to the wrong specialist, that specialist must notice and hand back, which costs two more calls. The lab assumes 97% routing accuracy with distinct specialist descriptions and 88% when descriptions overlap. Routing accuracy is a property of how you describe the specialists, so treat their descriptions as carefully as tool descriptions.

Handoff payloads: summary or full history

Every handoff must decide what the receiving agent sees. Passing the full history so far preserves every detail but makes each specialist call as expensive as a single agent's, and it grows with every subtask. Passing a short task summary keeps specialist prompts small but loses whatever the summary omits. In the lab, a 400-token summary drops a needed detail 2% of the time, while full history costs far more tokens; in the two-domain setup, full history raises success from about 92% to about 97% and cost by about 15%.

Anthropic's description of its multi-agent research system makes the same point from the other side: the lead agent has to give subagents a clear objective, an output format, guidance on tools and explicit task boundaries, because vague delegation caused duplicated work and gaps. In practice the best payload is usually a structured brief (goal, constraints, identifiers, what is already known, what to return) rather than either extreme.

Swarms: direct handoffs and compounding misroutes

In a swarm (also called a network or peer handoff pattern) there is no lead. Each agent can transfer the conversation to any peer it knows about, and the receiving agent continues with the shared history. This saves the supervisor's calls and lets the right specialist talk to the user directly. The OpenAI Agents SDK expresses it as handoffs, and LangGraph documents it as a network or swarm architecture.

The weakness is that every agent must route correctly, and errors chain. If each peer picks the right target 92% of the time, and a wrong target tries the work itself rather than handing on 35% of the time (both illustrative), a subtask fails at the handoff with probability about 0.08 × 0.35 ≈ 2.8%. When specialist descriptions overlap and peer routing falls to 72%, that becomes about 10% per subtask, and a three-domain task loses roughly a quarter of its runs to routing alone. A supervisor with the same overlap still routes at 88% and can retry, which is why mission four has a clear answer.

Swarms also accumulate shared history like a single agent, so they keep the context-growth cost while removing the tool-list cost. They shine when tasks are conversational and mostly stay in one domain after the first handoff, such as a triage agent passing a user to a billing specialist; they struggle with tasks that bounce between many domains.

Parallel fan-out: the latency win with a catch

The strongest argument for a supervisor is parallelism. If a task has four independent subtasks taking about eight seconds each, a single agent or a sequential supervisor takes over 30 seconds; a supervisor that dispatches all four at once takes roughly the slowest one plus a planning call and a synthesis call, perhaps 12 to 15 seconds. Anthropic reported that parallel subagents with parallel tool calls cut research time substantially on complex queries, and that token use, not architecture alone, explained most of the variance in performance on its browsing evaluation; both findings are specific to its system and evaluation, so measure your own.

The catch is dependencies. A subtask that needs another's result cannot start until it exists. Dispatch it in parallel anyway and it runs on incomplete information. The lab assumes 15% of later subtasks depend on an earlier one and fail 25% of the time when started in parallel. That is enough to make parallel fan-out with a thin summary handoff miss the success target in mission three, while parallel fan-out with a richer handoff meets both targets.

Comparing the three architectures

Property	Single agent	Supervisor	Swarm
Calls per two-domain task (lab)	about 8	about 11	about 9
Tool definitions per call	Whole catalog	One domain	One domain
Context growth	Shared, grows every step	Fresh per specialist	Shared, grows every step
Routing errors	None (tool choice only)	One router, can retry	Every agent routes
Parallel subtasks	Parallel tool calls only	Natural	Awkward
Best fit	Small or cohesive toolsets	Large catalogs, broad or parallel tasks	Conversational triage, mostly one domain per task

The rows interact. A supervisor with a 24-tool catalog pays its coordination tax and saves almost nothing on tool lists, which is mission two. A swarm with overlapping descriptions pays for misroutes at every hop, which is mission four. The right architecture is the cheapest one that meets your success and latency targets on your task mix, and the task mix (how many domains a typical request touches) matters as much as the catalog size.

How the frameworks express it

The patterns map to a small amount of code in current frameworks. The sketch below uses the OpenAI Agents SDK's handoff and agents-as-tools features; LangGraph offers equivalent supervisor and swarm helpers. APIs evolve quickly, so check the documentation for your version:

from agents import Agent, Runner

catalog = Agent(name="Catalog", instructions="Answer catalog and stock questions.", tools=catalog_tools)
orders  = Agent(name="Orders",  instructions="Look up and change orders.",        tools=order_tools)
billing = Agent(name="Billing", instructions="Invoices, refunds and payments.",   tools=billing_tools)

# swarm-style: the triage agent hands the conversation to a peer, which continues it
triage = Agent(name="Triage", instructions="Route the user to one specialist.",
               handoffs=[catalog, orders, billing])

# supervisor-style: specialists are tools; the lead keeps control and assembles the answer
lead = Agent(name="Lead", instructions="Split the request, call specialists, combine results.",
             tools=[catalog.as_tool(tool_name="ask_catalog", tool_description="Catalog and stock"),
                    orders.as_tool(tool_name="ask_orders",   tool_description="Order lookups and changes"),
                    billing.as_tool(tool_name="ask_billing", tool_description="Invoices and refunds")])

result = await Runner.run(lead, "Is order 1042 shipped, and refund the damaged copy?", max_turns=12)

Whatever the framework, set limits at every level (turns for the lead and for each specialist, and a total budget passed down), log which agent handled each step, and keep specialist descriptions as distinct as you would keep tool descriptions.

Two hybrids are worth knowing because they often beat all three pure forms. The first is a router plus specialists: one cheap classification call picks a specialist at the start, the specialist runs to completion, and there is no lead agent waiting to synthesise. It keeps the small tool lists and drops most of the coordination tax, which suits workloads where most requests touch a single domain. The second is tiered models: the lead or router runs on a strong model because its decisions are few and consequential, while specialists that execute well-scoped tool steps run on a smaller, cheaper and faster model. Anthropic describes a strong lead model with smaller subagent models in its research system. Whether the cheaper tier holds up is an empirical question for your tools, so evaluate per-step accuracy for each specialist before switching.

A worked example: Belgavi's Book Store assistant

Suppose a hypothetical assistant for Belgavi's Book Store started as one agent with 30 tools for catalog search, orders, billing and support, and has grown to 140 tools as teams added integrations. Most requests (say 70%) touch one domain; the rest touch two or three.

Using the lab's curve, a single agent with 140 tools selects correctly about 82% of the time per step, and carries roughly 17k tokens of tool definitions per call. For a one-domain request of three steps it still succeeds most of the time, but two- and three-domain requests fall toward 75% or below. A supervisor with six specialists of about 23 tools each raises per-step accuracy to about 96% at the cost of about three extra calls per subtask, and cuts tokens per call because each specialist sees a sixth of the catalog. For the one-domain majority, an even simpler option exists: a lightweight router that picks a specialist once and lets it run, which is essentially a one-hop swarm with no further handoffs.

The figures are calculated from the illustrative model. The decision procedure transfers: measure how many domains real requests touch, how many tools each step has to choose from, and how often subtasks are independent, then pick the architecture whose overhead you can afford for that mix.

Failure modes and what to measure

A study of multi-agent system failures (the MAST taxonomy) grouped observed failures into specification problems, inter-agent misalignment and task verification gaps, and found that many failures came from coordination rather than from the underlying model. The production versions are familiar:

Ping-pong handoffs between two agents that each believe the other owns the task. Measure handoffs per task and cap them.
Lost context at a handoff, so a specialist asks the user again for something already given. Measure repeated questions and task success by number of handoffs.
Duplicated work when parallel subagents overlap. Measure overlap in their tool calls.
Silent cost growth as agents are added. Track tokens and calls per task by architecture and task breadth, not just in total.
Nobody verifies the assembled answer. Give the lead or a separate checker an explicit verification step.

The lab's missions follow the decision points. Mission one asks you to fix a single agent drowning in tools. Mission two shows a supervisor wasting calls on single-domain tasks. Mission three puts four-domain tasks under a latency target, where parallel fan-out helps only with a good handoff. Mission four makes specialists overlap and asks which architecture tolerates it.

Run the lab: One agent, a supervisor or a swarm: pick the cheapest that works

The lab runs 600 synthetic tasks through the architecture you choose. The picture shows one sample task: for a single agent, the ring of tools it must choose from; for a supervisor or swarm, the agents and which ones were called, with a moving dot for each handoff. Tasks touch one or more of six domains, and each domain needs a few tool steps. Start with the single agent and 120 tools, then load mission one.

Prefer a full-width view? Open the standalone lab. Your stars are saved in this browser and shown on the series hub.

What to do next

Count the tools your agent sends per call and the tokens their definitions take; record per-step tool-selection errors from traces.
Measure how many domains real requests touch and how often subtasks are independent before choosing an architecture.
If you split, rewrite specialist descriptions to be mutually exclusive and test routing accuracy on a labelled set of requests.
Define a structured handoff brief (goal, constraints, identifiers, known facts, expected output) instead of passing either nothing or everything.
Set turn limits at every level and a total budget passed down from the lead; cap handoffs per task.
Track success, calls, tokens and p95 latency per architecture and per task breadth, and keep a single-agent baseline in your evaluations.

Sources and further reading

Related in this series:

Diagrams, examples and the lab model are original educational material. Lab numbers are calculated by a synthetic model, not measured on real hardware or services.

Views are my own and do not represent any employer, past or present.