Metastable overload: a short slowdown, retries and a backlog

What this experiment demonstrates

The order-status service at Belgavi's Book Store receives 80 new requests a second and can complete 100. At 60 seconds its capacity drops by 60 percent for 20 seconds. Clients give up after 5 seconds and retry up to 3 times, with a backoff of 1 second doubling per attempt. The server works through a first-in, first-out queue; a request completed after its client gave up is wasted work.

The model steps one second at a time for 300 seconds with deterministic arrivals. Nothing is sent over a network.

Run controlled comparisons

Start with the defaults: goodput falls during the slowdown and never returns. Over the last minute it is 0 requests a second while the server stays busy, with about 4 attempts per new request.

Now change one thing at a time. Shorten the trigger to 11 seconds: the backlog is absorbed and goodput recovers. At 12 seconds it does not. Set retries to 0: recovery takes 19 seconds. Raise the backoff base to 8 seconds: still no recovery. Turn on a retry budget of 10 percent: recovery after 40 seconds. Let the server drop requests whose client gave up: recovery is immediate. Set a queue limit of 300: recovery within about a second. Raise the client timeout to 15 seconds: the system never collapses. Lower the load to 50, or raise capacity to 150: there is enough headroom to absorb the trigger.

Inspect the trace and keep evidence

The table shows goodput at selected seconds before, during and after the trigger. Start trace, Next step, Show all and the slider control how much of the decision trace is visible. Reset restores the defaults and Export JSON downloads the inputs, metrics and trace.

Limits and production translation

Real arrivals are bursty, servers slow down under contention rather than holding a fixed rate, and clients differ in timeouts and retry policies. The model has one queue, one timeout and no caches. What carries over: a backlog that pushes waiting time past the client timeout turns all work into waste, retries keep that state going after the trigger ends, and the fixes that work remove wasted work or bound retries rather than just delaying them.