Rules First, Agent for the Residual: Adding an Agent to a Deterministic Pipeline Without Giving Up What Already Works

Start from the pipeline you have

Many business processes already run on rules that work: invoice matching, claims triage, refund eligibility, fraud screening. The rules handle the routine cases correctly and send everything else to a queue of people. The pressure to add an agent usually comes from that queue, not from the routine cases. Replacing the whole pipeline with an agent throws away the part that already works and makes it harder to explain any single decision.

The alternative is to keep the rules authoritative and give the agent only the residual: the cases no rule decides. Google’s Rules of Machine Learning make the same point from the other direction, recommending that a team launch with simple heuristics first; Anthropic’s guidance on building agents distinguishes predefined workflows from agents that direct their own process and recommends the simplest option that works. A residual router is a workflow with one agent step inside it.

What stays deterministic

The agent proposes from a closed set of actions. The validator encodes constraints that are true whatever the reasoning: no approval without a purchase order, no approval of goods not received, no agent approval above an amount. These are the same checks the rules already apply, written once and applied to both paths.

Worked example: invoice exceptions

An accounts-payable pipeline receives 1,000 synthetic invoices. Its rules reject duplicates and approve invoices that match their purchase order within two percent with all goods received. They decide 851 invoices, none wrongly. The other 149 used to go to people: large price differences, short shipments, missing purchase orders and small freight charges on large orders.

The agent in this example is a stub with a fixed error profile per exception type, so the arithmetic can be checked; its numbers describe the stub, not a model. On the 149 residual invoices:

Three things are visible. The validator refuses 20 proposals at every threshold: ten approvals without a purchase order, six approvals of goods not received and four approvals above the agent’s limit. All twenty are wrong answers, and confidence would not have caught them, because the stub is fairly confident when wrong. The threshold then trades automation against errors: at 0.6 four wrong credit decisions get through, at 0.8 none do, and at 0.9 almost nothing is automated. And the human queue falls from 149 to 44 at 0.8 without the rules changing at all.

The router in code

The runnable example reproduces the table. Its core is the order of the checks:

Illustrative python, checked for syntax; adapt names and storage to your system.

def route(inv, agent, threshold):
    action = rules(inv)                      # unchanged pipeline
    if action is not None:
        return action, "rules"
    proposed, confidence = agent.decide(inv)  # the only model call
    reason = validate(inv, proposed)          # hard constraints first
    if reason:
        return "human", "validator: " + reason
    if proposed == "escalate" or confidence < threshold:
        return "human", "below threshold"
    return proposed, "agent"

The validator runs before the confidence check so that a confident wrong answer cannot pass. Escalation is a first-class action, so the agent has a legitimate way to say it should not decide.

Choosing the threshold

Set the threshold on labelled residual cases, not on overall accuracy. Selective classification, where a classifier may abstain, is studied as a risk-coverage trade-off: for each threshold, measure how many cases are decided (coverage) and how many of those are wrong (risk), then pick the point your process can tolerate. Measure per exception type as well as overall; the stub here is reliable on freight charges and unreliable on missing purchase orders, and one overall number would hide that. Re-measure whenever the model, the prompt or the exception mix changes. After go-live, keep sending a random sample of accepted agent decisions to the same reviewers who handle the queue; their disagreement rate on that sample is the production estimate of the error rate, and it is the number that should move the threshold, not the agent’s own confidence.

Promoting decisions back into rules

Over time the agent will make the same decision for the same pattern again and again. In the example it approves small freight charges on large orders. Promoting that pattern into a rule removes model calls and makes the decision explainable, but the promoted rule is new code and needs the same evaluation as any change. Adding a rule that approves any invoice exceeding its order by at most 60 moves 13 invoices from the agent to the rules, and four of them are wrong: small orders whose excess was a price error, not freight. A rule that looked obvious from the agent’s decisions was not obvious on the full population.

Migration steps

1. Instrument the existing human queue: count the exception types and how people resolved them. That is the labelled set.
2. Write the validator from the constraints people already apply, and test it against historical cases.
3. Run the agent in shadow mode on the residual and compare its proposals with the human decisions.
4. Choose a threshold per exception type from the shadow results; start with types where the agent is reliable.
5. Route live, with a sample of accepted decisions still reviewed by people.
6. Review the agent’s repeated decisions monthly for patterns worth promoting into rules, and evaluate each promotion.

When the agent is unnecessary

If the residual is small and stable, the people already handling it may be cheaper than running, evaluating and auditing an agent. If most exceptions resolve by one or two patterns, write those rules instead. And if the cost of a wrong decision is high and errors are hard to detect afterwards, keep a person on every residual case and use the agent only to prepare a summary for them.

Failure modes

• Sending every case to the agent and letting it re-decide routine ones, which loses the explainability of the rules.
• A validator that only checks the response format, not the business constraints.
• One global threshold, tuned on the easy exception types and applied to the hard ones.
• Treating the agent’s confidence as calibrated without measuring it on your data.
• Promoting agent patterns to rules without evaluating the rule on the full population.

Practice and related reading

Further reading:

• Agents with a human in the loop
• Typed agent decisions for routing, moderation and extraction
• Load shedding

Sources and scope

Checked against these sources on 22 October 2026. Examples and figures are illustrative, not measurements.

• Google for Developers: Rules of Machine Learning (rule 1, do not be afraid to launch without machine learning)
• Anthropic engineering: Building effective agents (workflows versus agents)
• Geifman and El-Yaniv, Selective Classification for Deep Neural Networks (arXiv 1705.08500)