Change Budgets for Agent Writes: Capping What an Agent May Change per Window, Staged Expansion and Automatic Halt

Authorization says yes to each call; something has to bound the total

Per-call authorization answers one question: is this agent allowed to make this change to this object? It is necessary and it is not enough. An agent can be authorized for every individual write it makes and still, over a few minutes, change far more than anyone intended, because authorization has no memory of what the agent already did. A loop that deletes stale records can be correct on every single call and still empty a table if its filter is wrong. OWASP names this directly as excessive agency: the risk comes from too much functionality, autonomy, or permission exercised in aggregate, not from any one forbidden action.

A change budget is the aggregate limit. It caps how much an agent may change within a window, counts every committed change against that cap, and stops the agent when the cap is reached, regardless of whether each change was individually allowed. It is level-triggered on the total, the way an SRE error budget is level-triggered on reliability: you spend down a quantity and you stop when it is gone.

What stays deterministic

The budget is pure accounting. It never asks a model whether to continue; it compares a running count against a limit.

Because it is deterministic, the budget is testable and auditable: replay an agent run against the ledger and the decision to halt is reproducible.

Counting change by blast radius

A budget that counts requests treats deleting one row and deleting a million the same. Count the blast radius instead: the number of rows, the number of distinct customers touched, the monetary value moved, or whatever unit matches the harm you are bounding. A single statement can carry a large cost.

Illustrative python, checked for syntax; adapt names and storage to your system.

class ChangeBudget:
    """Sequential teaching counter; not a concurrent reservation ledger."""

    def __init__(self, limits):
        if not limits or any(type(v) is not int or v <= 0 for v in limits.values()):
            raise ValueError("limits must be positive integers")
        self.limits = dict(limits)       # e.g. {"rows": 500, "customers": 50}
        self.spent = {k: 0 for k in limits}

    def check(self, cost):
        """Validate costs; this does not reserve capacity."""
        if set(cost) - set(self.limits) or any(type(v) is not int or v < 0 for v in cost.values()):
            raise ValueError("unknown unit or invalid cost")
        projected = {k: self.spent[k] + cost.get(k, 0) for k in self.limits}
        if any(projected[k] > self.limits[k] for k in self.limits):
            return "halt"
        if any(projected[k] > 0.8 * self.limits[k] for k in self.limits):
            return "approve"            # near the cap: require a human to continue
        return "allow"

    def commit(self, cost, approved=False):
        decision = self.check(cost)
        if decision == "halt" or (decision == "approve" and not approved):
            raise ValueError("budget or approval required")
        for k in self.limits:
            self.spent[k] += cost.get(k, 0)

This sequential fixture validates costs and requires an explicit approval flag near the cap; the flag is not authentication. Its separate check/write/commit steps are not an atomic production cap. Production needs durable reservations before effects, trusted approval records, settlement and reconciliation after uncertain completion. Concurrent checks can otherwise oversubscribe. The separate runnable example illustrates the original counter; these article corrections do not update that repository.

Staged expansion as trust grows

A new agent, or a new version of one, should start with a small budget and earn a larger one from evidence, not from a calendar. Tie each stage to an observed outcome: the agent ran within budget across N windows, its changes passed review, no incident was attributed to it. Promotion widens the limits; a single attributed incident drops it back a stage. This is the same discipline as a staged rollout, applied to how much the agent may change rather than how much traffic it serves, and it means the automatic halt is the normal safety net during the riskiest early period.

Migration steps for an existing agent

1. Choose the unit of harm you most want to bound, and write a blast-radius function that returns the cost of a pending change in that unit.
2. Add the ledger in observe-only mode: record what each change would cost and what the running total is, but do not block.
3. Read the observed totals and set the window and the limit from them, leaving headroom for legitimate bursts.
4. Turn on enforcement at the approve threshold first, so near-budget runs go to a person before anything halts.
5. Turn on the halt, and wire it so a halted agent returns a clear reason rather than retrying the blocked change.
6. Define the promotion and demotion rules, and start every new agent or version at the lowest stage.

When a change budget is unnecessary

A read-only agent has no write blast radius, although query cost and load still need limits. Small, reversible, individually approved actions may justify a simpler policy; their aggregate can still exceed an acceptable change window. And a budget is not a substitute for authorization or for a spend cap: it bounds how much the system changes, not who may call it or how much money the provider will let you spend on tokens. Use it where an agent commits writes autonomously and the harm scales with how many it commits.

A budget also composes with the other patterns in this series. A desired-state reconciler, for instance, can price the diff it is about to apply and halt if the number of changes needed to converge exceeds the window budget, which catches a bad proposal before any write lands rather than partway through. The unit you choose is itself a design decision worth revisiting: a budget in rows protects a database, a budget in distinct customers protects your blast radius across tenants, and a budget in money protects the business, and an agent that touches all three may need more than one limit checked at once.

Failure modes

• Counting requests instead of blast radius, so one statement that changes a million rows costs the same as one that changes one.
• Measuring cost after the commit, which means the budget learns about the damage only once it is done.
• A window so long that a slow runaway never trips it, or so short that normal work constantly hits the approve threshold.
• A halted agent that retries the blocked change in a loop instead of surfacing the halt.
• A per-run budget with no per-principal budget, so many concurrent runs each stay within budget and together do not.

Verification

Unit-test the ledger: a sequence of costed changes must produce the exact allow, approve and halt decisions, and the halt must fire on the change that crosses the cap, not the one after. Replay a known runaway and assert the agent stops before exceeding the limit. In staging, run the agent with the budget in observe-only mode for a representative period and check that the limit you chose would not have blocked legitimate work. Keep the spent-versus-limit ratio per agent on a dashboard; an agent that regularly approaches its budget is either under-provisioned or misbehaving, and the ledger tells you which.

Practice and related reading

Further reading:

• Agents and authentication
• Rate limiting
• LLM error budgets
• Retry and backoff

Sources and scope

Checked against these sources on 23 October 2026. Examples and figures are illustrative, not measurements.

• OWASP Top 10 for LLM Applications 2025: LLM06 Excessive Agency
• Anthropic: Building effective agents
• Google SRE Workbook: Implementing SLOs and error-budget policies
• Kubernetes documentation: Controllers (control loops reconcile current to desired state)