Agent tenants on a shared service: fair queues and re-plans

What this experiment demonstrates

A shared internal service has a fixed number of workers and a fixed time per call. Tenants A, B and C call it at a steady rate. Tenant D is an agent: it calls at the same rate until its loop runs away for a while, and every call that is refused at admission or abandoned after waiting too long is re-planned into new calls half a second later, for at most three rounds per original call. The service admits calls through one of three policies: one shared FIFO queue, the same queue with a cap on running calls per tenant, or one bounded queue per tenant served round-robin. The agent runtime can also enforce its own budget, a token bucket checked before each call leaves.

The page advances a clock in 10 ms steps for 120 seconds. Arrivals are evenly spaced and every call takes exactly the configured time, so every run is repeatable. Nothing calls a real service.

Run controlled comparisons

Start with the defaults: 8 workers at 200 ms per call, about 40 calls per second; three tenants at 8 per second; the agent at 8 per second, then 40 per second from 30 s for 40 s; two calls per re-plan; a shared FIFO queue of 200; callers give up after 2 seconds. Ordinary tenants get 84.2 percent of their calls served and the worst of them 76.7 percent, while the agent offers 14,092 calls, 11,852 of them re-plans.

Now change one thing at a time. Set re-plans to 0: the ordinary tenants still lose calls (88.5 percent), so the runaway alone hurts them. Choose the per-tenant cap: 89.8 percent, because the capped agent still fills the shared queue. Choose one queue per tenant: the ordinary tenants are back to 100 percent, but the agent now offers 24,816 calls as more refusals turn into more re-plans. Go back to the shared queue and set an agent budget of 10 calls per second: everyone is served, the budget holds back 1,191 calls before they leave, and there are no re-plans at all. Doubling the workers also makes the problem disappear, which is worth comparing with what it costs.

Inspect the trace and keep evidence

The tenant table lists, for each tenant, the calls offered, served, refused at admission and abandoned in the queue, the re-plan calls and the calls held back by the agent budget, with the mean queue wait of served calls. Metrics summarise the ordinary tenants, the agent, the queue and anything still waiting at 120 seconds. Start trace, Next step, Show all and the slider control how much of the decision trace is visible; the metrics always describe the whole run. Reset restores the defaults and Export JSON downloads the inputs, the table and the trace.

Limits and production translation

Real services have variable call times, slow down as they saturate, autoscale, and sit behind connection pools and client retries that this model leaves out. Real agents re-plan into different tools and services, not only into more calls to the same one. Shares and counts are meaningful relative to each other, not as predictions. What carries over is the structure: identify tenants from authenticated identity, bound each tenant in its own queue, rotate between tenants, and put a budget and a round limit in the agent runtime so failures cannot multiply its own load.