Run the routing contract locally
Download pattern_13.py and test_pattern_13.py into the same directory. Use Python 3.11 or later. On macOS run python3 -B pattern_13.py and python3 -B -m unittest -v test_pattern_13.py. On Windows use py -3 -B pattern_13.py and py -3 -B -m unittest -v test_pattern_13.py. The program has no model SDK, network operation, or paid dependency.
The output selects deep first, reports overloaded for the concurrent proposal, then selects fast after settlement. A later deep request is budget_denied, with one unit remaining. This sequence demonstrates admission and accounting decisions only. It does not measure the quality or latency of any real model.
Architecture and request sequence
Read the architecture as text
- admit input: Identity, capability and deadline budget
- Admission filters: Expiry, active slots and eligibility
- Active reservation: Subtract synthetic units before work
- settle call: Refund unused units or retain discrepancy
- admit input → Admission filters: Validated input
- Admission filters → Active reservation: Bounded processing
- Active reservation → settle call: Recorded result
Read the sequence as text
- Caller → Boundary: Request required capability
- Boundary → Store: Reserve eligible units
- Store → Boundary: Return route decision
- Caller → Boundary: Submit measured usage
- Boundary → Store: Reconcile reservation
Inspect the route table and request identity
Route is an immutable dataclass with a name, capability level, and reserved-unit estimate. Router stores the remaining budget, active reservations, and identifiers already admitted. The two synthetic routes are ordered by cost only after eligibility filtering. The used-identifier set rejects a second admission for the same accepted request, preventing accidental double reservation in this sequential demonstration.
An overloaded request has not been admitted, so it can retry with the same identifier after capacity becomes available. An accepted request cannot use that identifier again after settlement. This is an explicit toy contract, not a complete idempotent API response cache. A production admission service should store the decision and settlement history durably and define what a caller receives after a lost response.
Follow admit in its actual order
The admit method validates the identifier and capability, rejects reused identifiers, checks the supplied deadline budget, and checks the number of active reservations. It then filters routes by minimum capability and available units. If none remain, it returns budget_denied without consuming units. Otherwise it selects the least expensive eligible route, subtracts the reservation, and records the active request.
The capability-floor test starts with only two units and requests capability two. It must refuse rather than select the affordable fast route. The overload test confirms that a rejected second request does not consume budget. The expired test confirms that a nonpositive deadline budget leaves the ledger unchanged. These assertions target consequences of decisions, not merely the string returned by a helper.
Settle usage without hiding inconsistent charges
Settlement looks up the active reservation and accepts only an integer charge between zero and the reserved amount. It refunds the unused portion and removes the active entry. An invalid charge raises an error while retaining the reservation for review. A second settlement fails because the active request no longer exists. Tests verify that neither path silently adds extra budget.
The upper bound is a deliberate simplification. Real providers can report usage above an estimate, so a production ledger needs an overage policy and a reconciliation state. Do not discard a genuine invoice because the local validator dislikes it. Preserve the reservation, measured usage, and discrepancy, then prevent further admission or escalate according to the account's policy. This module teaches the boundary but does not implement billing reconciliation.
Insert a real model adapter after admission
A future adapter receives the selected route, sanitized request, and deadline context. It invokes a pinned provider model, records measured usage, and returns a structured result or an explicit uncertain completion state. The router should not infer zero cost from a network exception because the remote provider might have processed the request. Keep unknown usage reserved until reconciliation can establish the charge.
A proposed API exposes admission decisions separately from generated content. Its storage adapter must atomically update global and tenant reservations under concurrent callers. The in-memory dictionaries are not sufficient for multiple workers or restart recovery. A reference deployment can package the gateway as a service backed by durable transactional storage, but this example does not deploy containers, configure provider credentials, or perform cloud load tests.
Use the tests to define the next integration work
Eight tests cover capability floor, least-cost eligible selection, overload without charge, expired admission, refunds, duplicate identifiers, inconsistent charges, and double settlement. Extend them with real adapter timeout behavior, cancellation, usage reconciliation, and concurrent admission once those components exist. Record which tests run locally and which contact an actual provider; the two kinds of evidence answer different questions.
The SRE overload discussion supports treating resource restrictions explicitly. Our route table and accounting model are original teaching choices. Production readiness requires measured task eligibility and a durable admission boundary, not simply replacing the word deep with a current model name. The operate article explains how to observe the ledger and refuse work honestly under pressure.