Agent-era architecture · Practical technical guide · 20 October 2026
How to introduce an agent into an existing request path without a big-bang replacement: a routing facade owns the decision, the agent runs in shadow on read-only inputs, disagreements are classified, segments cut over one at a time with rollback, and the old code remains the fallback.
Replacing a working path is the risky part
Most agent projects inside established companies are not green-field. There is already a rules engine that triages support tickets, a form that routes expense claims, or a service that classifies incoming documents, and it works well enough that people depend on it. The proposal is to replace or augment it with an agent. Martin Fowler's strangler fig describes the safer shape for any such replacement: grow the new system around the edges of the old one, move behaviour across a piece at a time, and let the old system shrink until it can be removed. The Azure Architecture Center describes the same pattern with an explicit facade that routes each request to the old or new implementation.
Agents make the pattern more useful, not less. The new path is nondeterministic, costs money per call and can fail in ways the old one never did. A gradual migration with a facade gives you a place to compare the two, a switch to turn the agent off per segment, and a fallback for the cases the agent should not handle.
What stays deterministic
| Concern | Owner | Why it stays out of the model |
|---|---|---|
| Which path serves a request | The routing facade, from a versioned table | Must be auditable and instantly reversible |
| Whether the agent result is used | Validation in the facade | Schema and policy checks are exact |
| Fallback when the agent abstains, times out or fails validation | The facade calls the old path | Keeps the service level of the old path |
| Comparison of shadow results | An offline comparator | Must be repeatable over stored pairs |
| Cutover and rollback per segment | An operator with recorded criteria | A business decision with evidence |
| How the ticket is triaged inside the agent path | The agent | The judgement the migration is buying |
The agent owns only the decision it is being introduced for. Everything about whether that decision is used, and what happens when it is not, lives in code a reviewer can read.
The facade
The facade sits where the old path is called today. It keys each request to a segment, such as the ticket's product and language, looks the segment up in a routing table, and runs one of three modes: legacy only, shadow, or agent with legacy fallback. In shadow mode the legacy result is returned to the caller and the agent result is recorded beside it; nothing the agent produces reaches a user or a downstream system.
Illustrative python, checked for syntax; adapt names and storage to your system.
def handle(ticket, routes, legacy, agent, record, timeout_s=8):
segment = (ticket["product"], ticket["language"])
mode = routes.get(segment, "legacy")
if mode == "legacy":
return legacy(ticket)
if mode == "shadow":
result = legacy(ticket)
try:
proposal = agent(ticket, timeout=timeout_s, read_only=True)
except Exception as exc: # shadow failures never reach the caller
proposal = {"error": type(exc).__name__}
record(ticket["id"], segment, result, proposal)
return result
try: # mode == "agent"
proposal = agent(ticket, timeout=timeout_s, read_only=False)
if proposal.get("abstain") or not valid(proposal):
return dict(legacy(ticket), served_by="legacy_fallback")
return dict(proposal, served_by="agent")
except Exception:
return dict(legacy(ticket), served_by="legacy_fallback")Two details matter. The agent receives a read_only flag in shadow mode and its tool set must honour it, because a shadow run that sends an email or assigns a ticket is no longer a shadow. And the result records which path served it, so every metric downstream can be split by path.
Shadow parity without comparing prose
GitHub's Scientist library popularised the mechanics for refactoring critical paths: run the control and the candidate, return the control result, compare the two, and publish mismatches for inspection. The same structure works here with one change. An agent's free text will never equal the rules engine's output, so the comparison must be over a structured decision: the queue, the priority and the tags, for example. Make the agent return that structure through a schema, and compare field by field.
Raw disagreement rates are not enough to decide anything. Classify each disagreement by reading a sample: agent better, legacy better, both acceptable, or neither. A segment where the agent disagrees 30 percent of the time but is judged better in most of those cases is a stronger cutover candidate than one with 5 percent disagreement where every disagreement is an agent error. The classification needs people at first; a judge model can take over once its agreement with those people has been measured.
Worked example: four segments, two cutovers
The figures in this example are invented for illustration. A ticket triage service has four segments. After three weeks of shadow running, the comparator reports the following, with disagreements classified from a reviewed sample.
| Segment | Shadow tickets | Disagreement | Agent better | Legacy better | Agent timeouts | Decision |
|---|---|---|---|---|---|---|
| Billing, English | 4,200 | 22% | 61% of disagreements | 9% | 0.4% | Cut over |
| Billing, Spanish | 900 | 31% | 48% | 27% | 0.6% | Stay in shadow; fix prompt |
| Login problems | 6,800 | 3% | 20% | 35% | 0.2% | Keep legacy rules |
| Shipping damage | 2,100 | 40% | 72% | 6% | 1.1% | Cut over with a 10% sample first |
Billing in English cuts over: the agent wins most disagreements and the legacy path stays as the fallback for timeouts. Spanish billing stays in shadow because more than a quarter of disagreements are agent errors; the next prompt revision is shadowed again before anyone looks at cutover. Login problems are a well-understood domain where the rules already agree with the agent 97 percent of the time and win more of the rest, so the agent buys nothing there. Shipping damage gets a staged cutover: the routing table sends 10 percent of that segment to the agent, the rest stays in shadow, and the share grows week by week while the reopen rate holds.
Cutover, rollback and keeping the old code callable
Write the cutover criteria down before the shadow period ends, so the decision is not made by whoever is most enthusiastic on the day: a minimum number of shadow decisions, a ceiling on legacy-better disagreements with an interval around it, a timeout ceiling, and a business metric such as reopen rate that must not move. Rollback is a one-line change to the routing table and should be rehearsed once per segment.
Keep the legacy path deployable and tested for as long as any segment can fall back to it. That sounds obvious, but the old code tends to rot once traffic moves: its tests are skipped, its dependencies age, and the day the agent provider has an outage the fallback fails too. Retire the legacy path per segment only after the agent has served it through at least one provider incident and one prompt revision without needing the fallback.
Migration steps
- Put a facade in front of the existing path with every segment in legacy mode, and ship it with no behaviour change.
- Define the structured decision both paths return, and adapt the legacy output to it.
- Give the agent a read-only tool set and turn on shadow mode for one segment; record pairs, latencies and costs.
- Build the comparator and a review queue for classifying disagreements; measure a judge against the reviewers before using it.
- Agree the cutover criteria per segment, then cut over the best segment with legacy fallback.
- Grow the share per segment, rehearse rollback, and retire legacy code only per segment and only after an incident has been survived.
When the agent is unnecessary
The shadow data answers this question directly, which is one of the best reasons to run it. Segments where the rules agree with the agent almost always, or where the agent loses most disagreements, should stay on rules: they are cheaper, faster and fully explainable. Segments with very low volume may never produce enough shadow decisions to justify a cutover. A migration that ends with the agent serving two segments out of four, and the rules engine serving the rest more cheaply, is a successful migration.
Failure modes
- The agent path has write access in shadow mode and acts twice on every ticket; enforce read-only in the tool layer, not in the prompt.
- Comparing free text instead of a structured decision, which reports near-total disagreement and teaches people to ignore the comparator.
- Shadow cost doubles model spend for weeks; sample shadow traffic per segment instead of shadowing everything.
- The legacy fallback hides agent failures, so the agent looks healthy; alert on the fallback rate per segment.
- Traffic mix changes during the shadow period and an aggregate parity number moves for reasons that have nothing to do with the agent; compare per segment.
Sources and scope
Checked against these sources on 20 October 2026. Examples and figures are illustrative, not measurements.
Put a routing facade in front of the old path, shadow the agent read-only, compare structured decisions, classify disagreements, cut over one segment at a time with rollback, and keep the old code callable.