Pooled connections and idle timeouts: the stale reuse race

What this experiment demonstrates

A client keeps one HTTP/1.1 connection in its pool and sends 2,000 requests with random gaps between them, drawn from a fixed seed so every run is repeatable. The server, or a load balancer in front of it, closes a connection that has been idle for its idle timeout. The client pool discards a connection that has been idle for its own idle timeout, and zero means it never does.

A close is not instant. It takes some time for the server's close to reach the client, and a request already on its way reaches the server after the server has decided to close. The model calls that span the close race window. A request whose idle gap falls inside it goes out on a connection the server has just closed: it fails with a reset or an empty response and is not applied.

Run controlled comparisons

Start with the defaults: a 60 s server idle timeout, a 90 s client idle timeout, a 250 ms race window, gaps averaging 45 s, 20 percent POST requests, and a client that retries GET once on a new connection. Five of the 2,000 requests land in the window between 59.75 s and 60.25 s. The two GETs are retried and succeed; the three POSTs reach the caller as errors.

Now change one thing at a time. Set the client idle timeout to 55 s, below the server's 60 s minus the window: the client always discards a connection first, and stale reuse disappears. Retry every method: the errors disappear, but three POSTs are retried without knowing whether they were applied, which is safe only when the request carries an idempotency key; turn that on and the blind POST retries drop to zero. Widen the race window to 1,000 ms and 25 requests are caught; narrow it to zero and none are. Raise the server idle timeout to 350 s and stale reuse disappears in this model; on a real path that only moves the problem to whichever device now has the shortest idle timeout.

Inspect the trace and keep evidence

The table lists the first eight requests that reused a closed connection, with their method, the idle gap before them and what the client did. The metrics count reused connections, new connections, stale reuse failures, failures seen by callers, requests retried on a new connection, and POSTs retried without an idempotency key. Start trace, Next step, Show all and the slider control the visible trace; the table and metrics always describe the full run. Reset restores the defaults and Export JSON downloads the inputs, the metrics, the examples and the trace.

Limits and production translation

Real gaps are not exponential, real pools hold many connections, and the race window depends on network delay, server load and how quickly a client notices a close. Some clients detect an already-closed connection before reusing it, which this model treats as a reconnect, and some retry automatically. The local measurement in the companion guide shows both behaviours. What carries over: make every client idle timeout shorter than the shortest idle timeout on the path, retry only what is safe to repeat, and give non-idempotent requests a key so a retry is safe.