Why architecture matters here
Architecture matters here because the alternative — control flow embedded in model reasoning — fails in ways that are almost impossible to fix after the fact. When the decision to stop, to retry, or to escalate is made by the model choosing a next token, you cannot enforce an invariant. You cannot guarantee that verification runs before a response, because the model might just decide to skip it. You cannot cap the number of tool calls, because the model might loop. Every reliability property you want becomes a probabilistic hope rather than a structural fact. A state machine makes them facts.
Consider the loop problem, which every free-form agent hits eventually. The model calls a search tool, gets a mediocre result, reasons that it should search again, gets the same mediocre result, and reasons that it should search again — a cycle that burns tokens and money and eventually times out. In a state machine, the transition out of the ACT/OBSERVE cycle is guarded: a retry counter and a no-progress detector live on the edge back to PLAN, and when the budget is exhausted the guard routes the agent to ESCALATE or RESPOND-with-what-we-have instead. The loop is impossible because the structure forbids it, not because the model remembered not to.
Testability is the second structural win. A free-form agent can only be tested end to end, by running whole conversations and hoping the model behaves; its internal decision points are invisible and non-deterministic. A state machine exposes its decision points as named transitions, each with a guard that is ordinary code. You can unit-test the guard: given this context, does the machine move from VERIFY to RESPOND or to ERROR? You can assert that no path reaches RESPOND without passing through VERIFY. The agent's control flow becomes verifiable in the way any state machine is.
Resumability is the third. Because the machine's entire control state is a single named node plus the context accumulated so far, you can serialize it after every transition. If the process crashes, is redeployed, or the agent is deliberately paused to wait on a slow human approval, it resumes from the persisted node with no ambiguity about where it was. A free-form loop has no such clean checkpoint — its position is buried in a growing message history and the model's implicit sense of what it was doing.
Auditability rounds it out. When something goes wrong in production, the state-machine agent hands you a transition log: INTAKE to PLAN to ACT to OBSERVE to PLAN to ACT to OBSERVE to VERIFY to ESCALATE, with the guard that fired at each step. That trace localizes the problem to a specific state and a specific guard, which you can then reproduce and fix. The free-form agent hands you thousands of reasoning tokens and a shrug. In any domain where an agent's actions have consequences, the ability to explain why it did what it did is not a luxury; it is a requirement, and the state machine gives it to you by construction.
The architecture: every piece explained
The top row is the working cycle. INTAKE parses the incoming request — normalizes it, extracts parameters, attaches session context — and if the request is valid, transitions to PLAN. PLAN is where the model reasons about what to do next and selects a tool or decides no tool is needed. ACT executes the chosen tool call, and OBSERVE reads and interprets the result. The edge from OBSERVE back to PLAN — the replan loop — is the heart of the agent's iterative work, and it is precisely the edge that a guard must protect from running forever.
The bottom row is the resolution and safety states. VERIFY checks the candidate output against whatever criteria the domain requires: is it grounded in the retrieved evidence, does it satisfy the schema, does it pass the policy filter. RESPOND emits the final answer only after VERIFY passes. ESCALATE hands the task to a human when a guard determines the agent cannot or should not proceed — low confidence, budget exhausted, a policy trip. ERROR handles tool failures and exceptions, deciding whether to retry, fall back, or escalate. These states exist so that every abnormal path has a named home rather than an implicit crash.
The guards are the intelligence of the control layer. A guard is a boolean condition on a transition edge: the agent may move from OBSERVE to VERIFY only if it has enough evidence; it may loop from OBSERVE back to PLAN only if the retry counter is under its cap and progress is being made; it must divert to ESCALATE if the token budget is spent or confidence is below a floor. Guards encode the policies you refuse to leave to the model's discretion — budget, retry limits, confidence floors, safety rules — and because they are code, they are deterministic and testable.
The persisted state box is what makes the whole thing operationally robust. After each transition, the machine writes down two things: the current node and the accumulated context (the request, the tool results so far, the running plan). That snapshot is the complete resumable position of the agent. A crash, a redeploy, or a deliberate pause loses nothing; the agent reloads the snapshot and continues from the exact node it was in. This is also the natural place to implement human-in-the-loop pauses: the machine enters a waiting state, persists, and does nothing until an external event wakes it.
The ops strip closes the loop. Transition counts show which paths are common and which are never taken (dead code in your machine). Dwell time per state reveals where the agent spends its time and cost. Dead-end rate and escalation rate tell you how often the agent fails to resolve on its own, and loops caught counts how often a guard stopped a runaway cycle. Because the states and transitions are named, all of these metrics come for free from the transition log — the observability is a byproduct of the structure.
End-to-end flow
Trace a request through the machine. A user asks the agent to reconcile two reports and flag discrepancies. The machine starts in INTAKE: it parses the request, confirms both reports are accessible, attaches the session, and — the guard being satisfied — transitions to PLAN. In PLAN the model reasons that it needs to fetch both reports, and selects the retrieval tool; the machine moves to ACT.
ACT invokes the retrieval tool for the first report, and OBSERVE reads the result. The OBSERVE guard asks: do we have everything we need to verify an answer? Not yet — only one report is loaded — so the guard routes the machine back to PLAN. This is the replan loop, and note the guard on it: a retry counter increments, and a no-progress check confirms that the loop is actually accomplishing something (a new report was loaded) rather than spinning. PLAN selects retrieval for the second report, ACT fetches it, OBSERVE now sees both reports present, and this time the guard routes forward to VERIFY.
VERIFY runs the domain checks: are the flagged discrepancies actually supported by the two reports, is the output in the required format, does it violate any policy. Suppose the check finds that the agent's draft cites a discrepancy that is not actually present in the data — a hallucinated finding. The VERIFY guard fails, and rather than shipping a wrong answer, the machine routes to ERROR, which decides to loop back to PLAN with the verification feedback attached so the model can correct itself. The retry counter guards this loop too: after a bounded number of failed verifications, the guard diverts to ESCALATE.
On the second attempt the model, now aware of the failed check, produces a corrected draft. ACT and OBSERVE gather any additional evidence, VERIFY runs again, and this time the checks pass. The guard routes to RESPOND, which emits the final reconciliation. At every one of these transitions, the machine persisted its node and context, so had the process crashed between the first and second VERIFY, it would have resumed at exactly the right place with the verification feedback intact — no re-doing the retrievals, no lost work.
Step back and count what the structure enforced. Verification ran before the response, unconditionally, because RESPOND is only reachable through VERIFY. The correction loop was bounded, because a guarded retry counter sat on every edge that could cycle. The escalation path existed and was reachable, so a genuinely stuck task had somewhere to go besides a timeout. And the entire run produced a legible trace of named states and fired guards. None of these properties depended on the model choosing to be well-behaved; they were structural consequences of drawing the agent as a state machine and letting the model work only inside the states.