Pooled HTTP Connections That Die Mid-Request: Idle Timeout Mismatches, the Reuse Race and Which Retries Are Safe

Core engineering · Practical technical guide · 26 October 2026

Why a client that keeps idle connections longer than the server or load balancer occasionally sends a request into a closed connection, what JDK HttpClient and Python http.client did in a local test, which requests may be retried, and how to set idle timeouts along a path so it stops happening.

The error that only happens sometimes

A service calls another over HTTP with a connection pool. Almost every request succeeds, but a few times a day one fails with connection reset, an empty response or, behind a load balancer, a 502. Retrying by hand always works. Load tests never reproduce it. The usual cause is not the network: it is two idle timeouts that disagree.

HTTP/1.1 keeps connections open between requests, and both ends are allowed to close an idle one. RFC 9112 warns that connections can be closed at any time, with or without intention, and that clients ought to anticipate recovering from asynchronous close events. The server, or a load balancer in front of it, closes a connection after its idle timeout. The client pool also has an idle timeout, after which it discards a connection rather than reuse it. When the client's idle timeout is longer than the server's, there is a window in which the client still believes a connection is usable and sends a request at the same moment the server closes it. The request is never processed, and the client receives a reset or end of stream.

The defaults do not agree

Component	Idle setting	Default per its documentation
AWS Application Load Balancer	Connection idle timeout	60 seconds (range 1 to 4,000)
AWS Application Load Balancer	HTTP client keepalive duration (maximum connection age)	3,600 seconds
nginx	keepalive_timeout	75 seconds
Java SE 23 HttpClient	jdk.httpclient.keepalive.timeout	30 seconds
Go net/http DefaultTransport	IdleConnTimeout	90 seconds

Put a Go client with the default transport behind an ALB with the default idle timeout and the client is willing to reuse a connection for 90 seconds while the load balancer closes it at 60. The ALB documentation also warns about the hop behind it: if the application's own idle timeout is shorter than the load balancer's, the load balancer may send a request on a connection the application has just closed and return 502 Bad Gateway, which is why it recommends making the application's idle timeout larger than the load balancer's. The rule is the same at every hop: each client must give up on an idle connection before the server it talks to does.

What happened in a local test

To see what two common clients do, a small Python server was run on Windows 11 over loopback, with a 2-second idle rule, and each client sent four requests 3 seconds apart, so every reuse crossed the server's idle timeout. The server ran in two modes. In the graceful mode it closed idle connections itself after 2 seconds. In the race mode it never closed an idle connection on its own, but dropped any request that arrived after 2 seconds of idleness with a reset and no response, which emulates a request crossing the server's close on the wire. JDK 23.0.2 HttpClient and Python 3.13.5 http.client were the clients.

Server mode	Client and method	Succeeded	Errors seen by caller	Server resets (not applied)
Graceful close	JDK HttpClient, GET	4 of 4	0	0
Graceful close	Python http.client, one reused connection, GET	2 of 4	2	0
Race emulation	JDK HttpClient, GET	4 of 4	0	3
Race emulation	JDK HttpClient, POST	2 of 4	2	2
Race emulation	Python http.client, GET or POST	2 of 4	2	2
Race emulation	JDK HttpClient, POST, keepalive.timeout=1	4 of 4	0	0

Three things stand out. The JDK client noticed connections the server had already closed and opened new ones, while Python's http.client wrote into the closed connection and failed; on Windows the error was ConnectionAbortedError, and the exact exception differs by platform. In the race, the JDK client retried each GET once on a new connection without telling the caller, the server log shows three resets followed by three successful GETs, but it did not retry POST and surfaced the error, header parser received no bytes. Lowering the client idle timeout to 1 second, below the server's 2, removed the failures, because with 3-second gaps the client then discarded every idle connection and opened a new one for each request. This is one machine, one run per case and an emulated race, so treat it as an illustration of behaviour, not a benchmark.

Which requests may be retried

RFC 9110 states the rule. A request with an idempotent method, which means PUT, DELETE and the safe methods such as GET, can be repeated automatically if the connection fails before the response is read, because repeating it has the same intended effect. A client should not automatically retry a non-idempotent request such as POST unless it knows the request is idempotent or can tell that the original was never applied, and a proxy must not retry non-idempotent requests at all. The RFC notes that some clients take the riskier approach of retrying a POST when an idle persistent connection closed before any response arrived.

In the idle-close race the request usually was not applied, but the client cannot prove that, because the same symptoms appear when a server processes a request and then crashes before replying. So the safe options for POST are to avoid the race, or to make the POST idempotent with a key the server checks, and then retry.

Illustrative Python; pool is a placeholder with get, new and put. The idle-timeout setting is the real fix.

import http.client

IDEMPOTENT = {"GET", "HEAD", "OPTIONS", "PUT", "DELETE"}

def send(pool, method, path, body=None, headers=None):
    """Retry once on a fresh connection, but only when repeating is safe."""
    headers = dict(headers or {})
    safe = method in IDEMPOTENT or "Idempotency-Key" in headers
    for attempt in (1, 2):
        conn = pool.get() if attempt == 1 else pool.new()
        try:
            conn.request(method, path, body=body, headers=headers)
            resp = conn.getresponse()
            data = resp.read()
            pool.put(conn)
            return resp.status, data
        except (ConnectionResetError, ConnectionAbortedError,
                http.client.RemoteDisconnected, BrokenPipeError):
            conn.close()
            if attempt == 2 or not safe:
                raise

Setting idle timeouts along a path

List every hop: client pool, any proxy or sidecar, load balancer, and the application server, and record each idle timeout and maximum connection age.
Make each client idle timeout shorter than the idle timeout of the server it talks to, with a margin of a few seconds for delay and clock granularity.
Behind a load balancer, make the application idle timeout longer than the load balancer idle timeout, as the ALB documentation recommends.
Retry idempotent requests once on a new connection, and give non-idempotent requests an idempotency key before allowing any retry.
Count connection resets and empty responses per client, separately from timeouts, so this failure is visible rather than buried in a generic error rate.

Failure modes

NAT and firewall idle timers. A middlebox can drop an idle mapping without telling either end. The next request then waits for a timeout rather than failing fast. TCP keepalive probes or a shorter client idle timeout keep the mapping alive or retire the connection first.
HTTP/2 PING does not help everywhere. The ALB documentation states that it does not support HTTP/2 PING frames and that they do not reset its idle timeout.
Maximum connection age. Some servers close a connection after a number of requests or an age limit, which produces the same race on a busy connection; the ALB closes gracefully after its keepalive duration with Connection: close on HTTP/1.x.
Automatic retries in libraries. A library that retries POST on a stale connection hides the race and risks duplicates. Read what yours does and test it, as above.

Verification

Reproduce the race locally with a server whose idle timeout is short, as in the test above, and a client configured as in production. Assert that idempotent requests succeed, that non-idempotent requests without a key fail loudly instead of being repeated, and that the failures disappear when the client idle timeout is lowered. The companion lab models the same race statistically over 2,000 requests, so you can see how rarely it occurs at realistic request rates, and why it still matters.

Sources and scope

Checked against these sources on 26 October 2026. The client behaviour table is a local measurement on 26 October 2026 (Windows 11, loopback, JDK 23.0.2, Python 3.13.5, one run per case, emulated race); other figures are documented defaults.

Make each client idle timeout shorter than the shortest server or load balancer idle timeout on its path, retry only idempotent requests or requests with an idempotency key, and count connection resets separately so the rare stale reuse becomes visible.