MCP fleet: sessions, silent streams and rolling deploys

Evidence label: deterministic synthetic model. This page runs a small JavaScript model of an MCP server fleet. Synthetic clients send tool calls with random durations, some of which mutate state part-way through; a load balancer routes each attempt to a replica with ten execution slots; and the model applies the session, stream and cancellation rules of MCP revision 2025-11-25 or 2026-07-28. Nothing here connects to a real MCP server, proxy or network. Every number is a calculated estimate that makes the transport mechanics visible.

Play the lab

How to play

Choose a protocol revision and state model, a routing mode and the number of replicas, then set the proxy idle timeout, the SSE keep-alive interval and the deploy drain window. The idempotency toggle makes mutating tools recognise a re-executed call. Every change re-runs the same ten minutes of synthetic traffic with the same seed, so differences come from your change. Inject Slow tools, Rolling deploy or Hot client at any time, and use New random seed to check that a fix survives different traffic.

Missions unlock in order. Load mission setup loads the broken fleet; meet the goal for one star and meet it efficiently, for example with round-robin instead of affinity or fewer replicas, for three.

What the model assumes

Calls arrive as a Poisson process at three per second from 24 clients; durations are log-normal with a median of 1.2 seconds, capped at 30 seconds. With slow tools on, 2% of calls run 60 to 240 seconds and emit progress only every 45 to 90 seconds. Forty percent of calls mutate state at a random point between 20% and 80% of their run. In the 2025-11-25 modes a request that reaches a replica without its session gets a 404 and the client re-initializes; a stream cut by the idle timeout is resumed with Last-Event-ID if the reconnect reaches a replica that can see the buffer. In the 2026-07-28 mode any replica serves any request and a cut stream cancels the call, which the client re-issues up to three times. A deploy drains each replica for the chosen window, kills whatever is still running and reboots it in 15 seconds.

What it cannot tell you

The model ignores network latency variation, TLS and connection reuse, client libraries with their own timeouts, the cost of the shared store, partial progress delivery and the tasks extension. Routing under round-robin is modelled as a random pick per attempt. Real SDKs differ in how they retry, resume and deduplicate, so treat the lab as intuition for the shape of each failure, then measure your own fleet with the signals listed in the article.

Read the full explanation

All AI in Production labs · Lab catalog

Views are my own and do not represent any employer, past or present.