Operate bounded agents without hiding exhaustion

Pattern 01 companion: Python example · Behavioral tests · Example license

Define an operational status vocabulary

The first operating mistake is counting every terminal run as success. A bounded investigator can terminate because it finished collection, exhausted its allowance, requested forbidden authority, repeated an observation, returned malformed output or encountered an adapter failure. The sample exposes these cases individually. Build dashboards around that vocabulary. A sudden increase in exhausted runs may reflect a changed prompt, a broader incident population or an inadequate budget; it is not automatically a reason to raise the limit.

Keep application recovery separate from investigator completion. An incident remains open until the owning operational process verifies recovery. A completion summary should state what evidence was read, which questions remain unanswered and why the investigator stopped. A reviewer can then continue manually without mistaking a safe stop for a healthy service. Preserve the terminal reason even if the user-facing summary is brief.

Architecture and request sequence

Component diagram for Operate bounded agents without hiding exhaustion — Original reference architecture by Sandeep Belgavi. Arrows show the responsibilities and transitions described below.

Request sequence for Operate bounded agents without hiding exhaustion — Illustrative sequence for this design; failure and recovery variants are explained in the article.

Inject failures at the contract boundaries

Begin with the supplied tests, then add a production-adapter test suite that delays a response, returns malformed JSON, proposes an unlisted tool and repeats a previously read object. Inject a valid reference pointing to an unavailable source. That final case checks whether the connector preserves missing evidence rather than quietly returning an empty successful result. Include a request to finish before any observation and a sequence that spends all its allowance without a final decision.

The teaching loop cannot interrupt a callable that never returns. A production acceptance exercise must therefore test the actual timeout mechanism in its adapter transport or worker process. Do not infer deadline enforcement from the decision counter. Record time until cancellation, whether the upstream operation continued and whether occupied resources were reclaimed. These measurements determine whether a stalled investigator can consume the service's entire worker pool.

Keep retry ownership explicit

A model client, an observation connector and an outer job runner can each retry independently. Without a shared policy, one incident can multiply into many calls while each component believes it is being conservative. Give one layer ownership of the end-to-end allowance. The adapter should return retry information and usage to that owner rather than invisibly resetting a fresh budget. The illustrative three-decision task might permit one transport retry, but that retry must still count toward the global request and elapsed-time limits.

AWS's retry guidance notes that retries can worsen overload and that a timeout does not prove side effects never occurred. Source: retry boundaries. This investigator remains read-only, but even reads occupy scarce connections and can incur charges. Backoff and admission control need to protect the downstream service, not merely improve the investigator's apparent completion rate.

Observe cost and policy outcomes together

Trace admission, adapter request, parsed proposal, authorization decision, evidence fetch and terminal status under one run identifier. Record timestamps and sanitized error categories; avoid collecting private model reasoning. Attach model and prompt revisions so a change in exhaustion can be compared with a release. Count denied actions separately from transport failures. A growing denied-action rate after a prompt change is useful evidence that the proposed capabilities and actual permissions no longer align.

Calculate cost per reviewed, useful investigation as well as cost per run. A cheap run that produces an unusable summary can shift work to the on-call engineer. Include unsuccessful calls, retries, evidence storage and reviewer effort in the analysis. Use a maximum concurrent-run count to prevent incidents arriving together from creating a surge. Reserve capacity for manual investigation so the assistant cannot starve the people it is supposed to help.

Test the security and recovery story

Place hostile instructions inside synthetic observation text: ask the investigator to restart a service, reveal another incident or skip its remaining checks. The adapter may respond badly, but the action gate must continue to deny operations outside its inventory. This is a capability test, not a test of whether the model writes a refusal sentence. Also test a legitimate operator request outside the current scope; authorization should remain explicit even when the request is well intentioned.

For a controlled trial, require that every completed collection has at least one retrievable observation, every denied action leaves a policy event, and every interruption preserves its terminal reason or an explicit interrupted status. Evaluate summaries against labeled incidents from more than one failure class. Roll back an adapter revision when it increases unsupported finishes or unexplained denials. A rollback should not discard the evidence needed to understand why that revision failed.

Evidence required before expansion

Use the Evals prerequisite collection for measurement terminology and the linked retry source for its narrow reliability principle. The operational controls above are original recommendations, not features already implemented in the downloadable module. Windows unit tests cover its deterministic states. A broader rollout still needs real transport deadlines, secure connectors, calibrated summaries, restart behavior and a practiced manual fallback.