Why architecture matters here
The architecture matters because a single large task exceeds what one model call can reliably do, and decomposition is how you buy back reliability. A model asked to satisfy a broad goal in one pass has to hold the entire problem in its working context, plan and execute simultaneously, and produce a long structured output without checkpoints — the conditions under which models drift, omit requirements, and confabulate. Cutting the goal into subtasks each sized to a single focused call keeps every model invocation inside its competence, gives each a clear success criterion, and localizes errors. The difference is the difference between an essay written in one breathless pass and one written section by section with an outline: the second is more complete, more consistent, and far easier to fix.
The second forcing function is latency through parallelism. Many real goals decompose into subtasks that are mutually independent — research three vendors, summarize five documents, check four data sources — and if they must run sequentially the total latency is the sum of them all. A dependency graph makes the independence explicit, so the scheduler can dispatch all the independent subtasks at once and the total latency collapses toward the longest single chain rather than the sum. For an agent that would otherwise feel unbearably slow chaining a dozen tool calls one after another, exploiting the parallelism the decomposition exposes is often the single largest user-facing speedup available.
The third reason is observability and recovery, which monolithic agent calls lack entirely. When an agent does everything in one opaque reasoning trace and the answer is wrong, you cannot tell which part failed or retry just that part. A decomposed plan is a graph of discrete steps, each with an input, an output, and a status, so progress is inspectable and a failure is attributable to a specific node. Recovery becomes surgical: re-run the failed subtask, or re-plan the affected subtree, while the successful subtasks' outputs are preserved in the result store. This is what makes long-running agents debuggable and resumable rather than all-or-nothing.
The fourth architectural payoff — and its central risk — is that decomposition must be bounded or it consumes itself. A planner that can break tasks into subtasks can also break subtasks into sub-subtasks without limit, spawning a combinatorial explosion of tiny steps that burns tokens and time while accomplishing little. So the architecture pairs the power to decompose with hard limits: a maximum recursion depth, a cap on fan-out per node, and a verifier that judges whether a subtask is 'done' rather than always splitting further. Getting these limits right is what separates a decomposition system that carves a goal into a right-sized handful of steps from one that recurses into a thousand trivial fragments and never converges on an answer.
A fifth, more strategic reason is that decomposition decouples planning from execution, and that separation is itself an architectural lever. The decomposer can be a stronger, more expensive model asked only to think — to produce a good plan — while the workers that execute individual subtasks can be cheaper, faster models or deterministic tools, since each subtask is narrow and well-specified. This split lets a system spend its most capable reasoning where it matters most (the plan) and its cheapest compute where the work is routine (the steps), a cost-and-quality trade-off that is only expressible once the goal has been broken into a planning phase and an execution phase. Teams that conflate the two — using one heavyweight call to both plan and execute every step — pay premium rates for routine work and get no independent checkpoint on the plan before committing to it.
The architecture: every piece explained
Top row: from goal to graph to schedule. The user goal is the high-level, often ambiguous request. The decomposer — the model acting as a planner — turns it into a set of subtasks and, critically, the dependencies among them, emitting a structured plan (often JSON) that names each step, its inputs, and which other steps must complete first. Those dependencies form the dependency graph, a DAG whose edges say 'B needs A's output'. The scheduler walks this graph by maintaining a ready set — every subtask all of whose predecessors have finished — and dispatching those, running independent subtasks in parallel while dependent ones wait for their inputs.
Middle row: execution, storage, and verification. A worker / tool executes one subtask — a model call for a reasoning step, an API or retrieval tool for an information step, a code run for a computation. Each subtask's output lands in the result store keyed by its task id, so downstream subtasks can pull exactly the inputs they depend on. A verifier checks each output against the subtask's subgoal — did the research step actually return the vendor's pricing, or an apology? If the check fails, the re-plan path adapts the graph: retry the step, substitute a different approach, or split the subtask further, rather than letting a bad intermediate silently poison everything downstream.
Bottom row: composition and the guardrails. The aggregator is the closing move — once the leaf subtasks are done, it composes their outputs into the final deliverable, typically a last model call that synthesizes the pieces (the three vendor summaries plus the comparison into a recommendation). Depth + fan-out limits are the guardrails that keep decomposition finite: a maximum recursion depth stops infinite splitting, and a cap on children per node stops a single step from spawning a swarm. The ops strip names the health signals that matter — plan quality, per-step success rate, how often re-planning fires, total tokens spent across the whole graph, and end-to-end latency.
End-to-end flow
Follow the competitive-analysis goal through the system. The user goal — 'compare three vendors and recommend one' — reaches the decomposer, which emits a plan: three research subtasks (one per vendor), one comparison subtask that depends on all three, and one recommendation subtask that depends on the comparison. The dependency graph is a fan-in: three parallel leaves, then a join, then a final node. The scheduler inspects the graph, finds the three research subtasks in the ready set (they depend on nothing), and dispatches all three at once to worker threads, each of which calls a web-search-and-summarize tool for its vendor.
The three research subtasks run concurrently. Two return clean summaries with pricing and features and their outputs are written to the result store under their task ids. The third comes back thin — the verifier checks it against the subgoal ('must include pricing') and finds no pricing, so it marks the subtask failed and triggers a targeted re-plan: retry the research with a refined query rather than aborting the whole goal. The retry succeeds, its output is stored, and now all three research subtasks are complete. Their completion satisfies the comparison subtask's dependencies, so the scheduler moves it into the ready set and dispatches it. The comparison worker pulls the three summaries from the result store, calls the model to build a feature-and-price matrix, and stores the result.
With the comparison done, the recommendation subtask becomes ready. Its worker pulls the comparison matrix and calls the model to weigh the trade-offs and pick a vendor with justification. That output flows to the aggregator, which composes the final deliverable — the recommendation, backed by the comparison, backed by the three vendor summaries — into a single coherent answer for the user. The whole run touched five subtasks; three ran in parallel, one was retried once, and the critical path was 'slowest research → comparison → recommendation', far shorter than running all five in sequence.
Now consider the guardrails firing on a harder goal. A vaguer request — 'improve our product' — tempts the decomposer to split endlessly: improve marketing, which splits into improve SEO, which splits into audit keywords, which splits into... The depth limit stops this: once a branch reaches the maximum allowed depth, the planner is forced to treat the current subtask as a leaf and execute it directly rather than decompose again. Likewise a node that tries to spawn fifty children is capped at the fan-out limit. These bounds, plus the verifier's willingness to call a subtask 'good enough', are what let the system converge on a finite plan and produce an answer instead of recursing forever into ever-smaller fragments — the operational cycle of decompose, schedule, execute, verify, re-plan, and aggregate running to completion.