MCP Streamable HTTP in Production: Sessions, Resumption and Load Balancers

Two revisions, two shapes of one transport

Streamable HTTP arrived in MCP revision 2025-03-26 to replace the older HTTP+SSE pair of endpoints. Through revision 2025-11-25 it kept a recognisable shape: a single MCP endpoint that accepted POST for client messages and GET for a standalone server-to-client SSE stream, an optional Mcp-Session-Id minted in the response to initialize, and optional stream resumption through SSE event IDs and the Last-Event-ID header. Revision 2026-07-28, the latest published revision when this was written, removed all three. MCP is now explicitly stateless: there is no initialize handshake, every request carries its protocol version and client capabilities in _meta, the GET endpoint is gone, and a broken response stream simply loses the in-flight request.

A modern-only server answers GET and DELETE with 405 Method Not Allowed and ignores Mcp-Session-Id and Last-Event-ID; a dual-era server keeps the older behaviour for older clients.

Everything below is based on the published specification pages for revisions 2025-11-25 and 2026-07-28. SDKs implement revisions at different speeds, so check which revision your client and server SDK versions actually negotiate before relying on any behaviour described here.

Why in-memory sessions and round-robin fight

In the 2025-11-25 model a server may mint a session ID when it answers initialize, and the client must send that ID on every later request. The spec says nothing about where the session lives. The path of least resistance in most server frameworks is a dictionary in process memory, which works perfectly on a laptop and breaks the moment you run more than one replica behind a load balancer that does not know about sessions.

The arithmetic is unforgiving. With R replicas behind round-robin, each later request lands on the replica that holds the session with probability 1/R. A session that needs N requests survives only if all N-1 follow-ups hit home, so the probability is (1/R)^N-1. With three replicas and a five-call agent turn that is (1/3)⁴, about 1.2%. Every miss returns 404 Not Found, which obliges the client to start a new session with a fresh initialize; that new session is minted on whichever replica took the re-initialization, and the next request is again a one-in-three bet. In the lab, mission one starts here: three replicas, sessions in memory, round-robin, and thousands of re-initializations in ten simulated minutes.

There are three ways out, and they have different costs:

• Session affinity. Route by the session header or a cookie so every request in a session lands on the replica that minted it. Cheap, but it turns every restart into a mass session loss and ties load to session popularity: if one client sends 35% of calls, the busiest of two replicas receives 0.35 + 0.65 / 2 = 67.5% of them, and adding replicas never helps that client. Mission four in the lab shows the result, one replica pinned at its concurrency limit while the others idle.
• Externalised session state. Keep session data and any SSE event buffers in a shared store such as Redis, so any replica can serve any request. You pay a store lookup per request (a few milliseconds in the lab's model) and you must secure and expire that store.
• No session at all. Revision 2026-07-28 makes this the protocol's answer: state that has to span calls is an explicit handle the server returns from a tool and the model passes back as an argument, so any replica can look it up.

Resumption in 2025-11-25: where does the buffer live?

Revision 2025-11-25 lets a server attach an id to SSE events. The ID must be unique across all streams in the session and should encode enough to identify its stream, so that when a client reconnects with an HTTP GET carrying Last-Event-ID, the server can replay what was sent after that point on that stream only. A server may also close the connection deliberately after an event ID, with a retry field, turning a long call into short polls. Disconnection is explicitly not cancellation: to cancel, the client sends a notifications/cancelled message.

Resumption only works if the replica that receives the GET can see the event buffer. With an in-memory buffer and round-robin routing, the reconnect lands elsewhere with probability (R-1)/R, and that replica does not know the session either. With affinity, the reconnect reaches the right process as long as it is still alive. With a shared buffer, any replica can replay. The buffer costs memory: 2,000 concurrent streams that each retain 50 events of about 1 KB need roughly 100 MB plus a retention policy. And resumption cannot recover work that died with its process: a replay buffer helps when the connection breaks while the server keeps working, not when the server itself is killed.

Revision 2026-07-28: stateless requests and explicit handles

The 2026-07-28 revision removes protocol-level sessions, the GET endpoint and resumption in one move, and replaces the handshake with metadata on every request. Each POST carries MCP-Protocol-Version plus the standard request headers Mcp-Method and, for tools/call, resources/read and prompts/get, Mcp-Name. The server must reject a request whose headers disagree with the JSON body with 400 Bad Request and error -32020 (HeaderMismatch), which is what makes it safe for a gateway to route or rate-limit on headers without parsing bodies.

POST /mcp HTTP/1.1
Accept: application/json, text/event-stream
Content-Type: application/json
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: reserve_stock

{"jsonrpc": "2.0", "id": 41, "method": "tools/call",
 "params": {"name": "reserve_stock",
            "arguments": {"basket_id": "bsk_7f3c", "isbn": "9780000000001",
                          "idempotency_key": "turn-88-call-3"},
            "_meta": {"io.modelcontextprotocol/protocolVersion": "2026-07-28",
                      "io.modelcontextprotocol/clientCapabilities": {}}}}

Three consequences matter for operations. First, any replica can serve any request, so plain round-robin or least-connections routing is correct and affinity becomes optional. Second, state that used to hide in a session moves into explicit identifiers: the spec's non-normative guidance on stateful tools recommends a creation tool that returns a handle (the basket_id above), authorization checked against that handle on every call, a stated lifetime and a clear error for expired handles. Multi round-trip requests follow the same pattern: the server returns an InputRequiredResult with an opaque, integrity-protected requestState that the client echoes on its retry. Third, the stream is now a pure function of one request: closing it is cancellation, the server should stop work and must not send more messages for it, and a client that lost a stream must re-issue the request with a new ID.

The silent stream and the proxy idle timeout

Most connection failures in an MCP fleet are not crashes. They are intermediaries closing a connection that looked idle. nginx's proxy_read_timeout defaults to 60 seconds between two reads, and many cloud load balancers default to an idle timeout of about the same order (check your provider's documentation for the exact value). An SSE response that is waiting on a slow tool sends nothing, so a tool that runs for four minutes and reports progress every 90 seconds gets cut at 60 seconds, every time.

In revision 2025-11-25 the client could resume and the server kept working, so the cost was a reconnect. In revision 2026-07-28 the cut cancels the request, and because the silence is a property of the tool, the re-issued request is cut at the same point. Mission two in the lab reproduces this: 2% of calls run for one to four minutes with progress every 45 to 90 seconds behind a 60-second idle timeout, and the setup loses those calls no matter how many retries the client makes.

The fix the spec itself suggests is cheap: for long-lived streams, servers are encouraged to emit an SSE comment line (a line starting with a colon) as a keep-alive, and clients must ignore it. A comment every 20 seconds costs a few bytes per stream per interval, keeps every intermediary's idle timer reset and survives a timeout as short as 40 seconds. The spec also says servers should send X-Accel-Buffering: no on SSE responses so that nginx-style proxies forward events immediately instead of buffering them; a buffering proxy turns real progress events into the same silence.

location /mcp {
    proxy_pass          http://mcp_backends;
    proxy_http_version  1.1;
    proxy_buffering     off;          # or rely on X-Accel-Buffering: no from the server
    proxy_read_timeout  60s;          # keep-alive comments must arrive well inside this
}
# server side, per SSE response: emit ":\r\n" every 15-30 s while the tool runs

Set the keep-alive interval to between a quarter and half of the shortest idle timeout on the path, including the client's own HTTP library and any corporate proxy you do not control. Raising the timeout also works, but only on the hops you own.

Re-issue means re-execute: duplicates and idempotency

Whenever a client has to re-issue a call, through a cancelled stream in 2026-07-28 or a lost session in 2025-11-25, the server may already have done part of the work. For a read-only tool that costs only time. For a mutating tool it can be a second payment, a second email or a second stock reservation. The probability is easy to estimate: if 40% of tools mutate state and the commit point is spread across the middle of the execution, a call cut off at a random moment has already committed in roughly half of the mutating cases, so about one in five broken calls risks a duplicate write unless the tool can recognise the retry.

Because 2026-07-28 has no session, deduplication cannot rely on connection identity, and the JSON-RPC request ID changes on re-issue. The key has to be part of the call itself: an idempotency key argument the model or client derives from the conversation turn and call position, which the tool stores with the result of the first execution and returns on a repeat. The lab's idempotency toggle models exactly that: a re-executed mutating call returns the stored outcome instead of writing again. It is the only fix that holds regardless of why the stream broke.

Rolling deploys: drain windows and long-running work

A rolling deploy stops each replica in turn. A graceful shutdown stops routing new requests to the replica, waits for in-flight requests up to a drain window, then kills what is left. With R replicas, a drain window D and a boot time B, a one-at-a-time rollout takes about R × (D + B). In the lab, four replicas with a 15-second boot and a 20-second drain finish in 140 seconds; a 60-second drain stretches that to 300 seconds.

Drain windows have a hard limit: you cannot drain a four-minute tool call inside a three-minute deploy budget. Anything still running at the kill is lost on both revisions, so the client re-issues it, and without idempotency the re-execution can write twice. On 2025-11-25 with in-memory sessions the kill also drops every session the replica held, and the next request from each of those clients returns 404 and forces a re-initialization. Mission three asks for a deploy under three minutes with slow tools in flight, zero duplicate writes and at least 99.5% success; the efficient answers keep the drain short and make retries safe rather than trying to outwait the longest call.

For work that genuinely runs for minutes, a request-scoped stream is the wrong container. Revision 2026-07-28 moves long-running tasks into the official io.modelcontextprotocol/tasks extension, which returns a task handle and lets the client poll with tasks/get. Check whether your SDK version implements the extension before designing around it.

Worked example: sizing the Belgavi's Book Store tool fleet

Take a hypothetical MCP server for Belgavi's Book Store that exposes catalog search, stock reservation and order tools to an assistant. Illustrative traffic: 3 tool calls per second from 24 client processes, median call time 1.2 seconds with a long tail capped at 30 seconds, mean about 1.8 seconds, so about 5.4 calls in flight on average (Little's law: arrival rate times time in system). A small slice, 2% of calls, are bulk inventory reconciliations that run for one to four minutes, adding about 0.06 × 150 ≈ 9 more in flight. Each replica runs up to ten concurrent tool executions.

1. Capacity: about 15 calls in flight on average. Two replicas (20 slots) run at roughly 75% utilisation, enough for queues to form whenever a few long calls overlap; three replicas (30 slots) run near 50%.
2. Routing: round-robin on 2026-07-28; with affinity and a 35% client, the busiest of three replicas would get 0.35 + 0.65 / 3 ≈ 57% of calls.
3. Streams: reconciliation calls are silent for up to 90 seconds behind a 60-second idle timeout, so the server emits a keep-alive comment every 20 seconds.
4. Writes: stock reservations carry an idempotency key; the store keeps keys for 24 hours.
5. Deploys: 20-second drain, 15-second boot: about 105 seconds per rollout.

In the lab this configuration (stateless, round-robin, three replicas, keep-alive 20 seconds, idempotency on) holds p95 latency under eight seconds with both the hot client and slow tools active, which is the three-star answer to mission four. All figures here are calculated from the lab's synthetic model, not measured.

Detecting transport failures in production

Transport problems hide inside agent failures: the model sees a tool error or a timeout and improvises. Instrument the layer directly, at the gateway and in the server:

Common mistakes come from single-process habits: in-memory sessions on three replicas, retrying mutating tools blindly, raising retry counts to hide a deterministic idle-timeout cut, and enabling affinity and then wondering why more replicas do not cut latency. A subtler one in mixed fleets is letting a gateway route on Mcp-Name for requests that declare an older protocol version: the 2026-07-28 text says intermediaries that enforce policy on mirrored headers should reject requests whose version does not require header and body validation instead of trusting the header.

Run the lab: MCP fleet: sessions, silent streams and rolling deploys

The lab simulates ten minutes of tool calls from 24 clients against a fleet of MCP server replicas behind a load balancer. Each replica row shows the calls it is running right now as small cells and, shaded behind them, how many calls it held over the whole run; amber and red rows are draining or rebooting during a deploy. The strip underneath marks broken streams in red and failed calls in pink. Each mission loads a broken fleet and asks you to fix it the way you would in production.

Prefer a full-width view? Open the standalone lab. Your stars are saved in this browser and shown on the series hub.

What to do next

1. Find out which MCP protocol revisions your client and server SDK versions negotiate, and whether any older clients still send Mcp-Session-Id, GET or DELETE.
2. If you run 2025-11-25 sessions on more than one replica, move session state and event buffers to a shared store or route with affinity, and track the 404 rate on session requests.
3. Measure the idle timeout on every hop (client library, proxy, load balancer) and emit SSE comment keep-alives at a quarter to half of the shortest one; send X-Accel-Buffering: no.
4. Add idempotency keys to every mutating tool and alert on key hits, so re-issued calls cannot write twice.
5. Time a rolling deploy end to end and set the drain window from your p99 call duration, not your longest call; move multi-minute work to task handles.
6. Add gateway metrics for stream durations, HeaderMismatch errors and the busiest replica's request share.

Sources and further reading

• MCP specification 2026-07-28: Streamable HTTP transport
• MCP specification 2026-07-28: key changes since 2025-11-25
• MCP specification 2025-11-25: transports (sessions and resumability)
• MCP specification 2026-07-28: multi round-trip requests
• WHATWG HTML Living Standard: server-sent events
• nginx ngx_http_proxy_module documentation

Related in this series:

• MCP Authorization Explained: OAuth 2.1, Resource Indicators and Token Audience
• Retrying Agent Tool Calls Without Executing the Same Action Twice
• MCP Tool Poisoning: How One Malicious Description Hijacks an Agent

Diagrams, examples and the lab model are original educational material. Lab numbers are calculated by a synthetic model, not measured on real hardware or services.

Views are my own and do not represent any employer, past or present.