Why architecture matters here
Long polling matters because compatibility is sometimes worth more than efficiency. WebSockets are more efficient for real-time bidirectional communication, but they require a protocol upgrade that some network intermediaries block, they need fallback handling, and they add complexity. For a use case where updates are occasional (not high-frequency), where the audience's networks are unpredictable (corporate proxies, restrictive firewalls), or where simplicity matters, long polling's 'works everywhere HTTP works' is a decisive advantage. Notification delivery, status updates, chat with modest message rates, and any server-push need where reaching every client reliably beats squeezing out efficiency — these are long polling's domain, and choosing it deliberately (rather than reaching for WebSockets reflexively) is sometimes the right architectural call.
The reliability challenge long polling must solve is the gap between responses. Between the server responding (with data) and the client's next request arriving, there's a window where new data could arrive with no held connection to deliver it on — and a naive implementation loses those messages. The solution is a cursor (or since-token): the client tracks what it's received (a message ID, timestamp, or sequence number), sends it with each request, and the server returns everything since that cursor — so messages arriving during the gap are delivered on the next request rather than lost. This cursor discipline is what makes long polling reliable; without it, the gap between requests is a message-loss window, and the reliability difference between a correct long-polling implementation and a naive one is entirely in the cursor.
And the scaling reality is that held connections are the cost. Each long-polling client holds a connection open (waiting for data), so a server with many clients holds many connections — and if the server uses thread-per-connection (blocking IO), it exhausts threads quickly (each held connection parks a thread doing nothing but waiting). The answer is async servers (non-blocking IO, event loops, or virtual threads) that hold many connections cheaply — the same scaling requirement as WebSockets and SSE, because they all hold connections. A long-polling deployment that uses a blocking server hits a connection wall fast; one that uses async IO holds tens of thousands of waiting clients on modest resources. Understanding that held connections are the scaling constraint (and async servers the answer) is essential to deploying long polling at scale.
The architecture: every piece explained
Top row: the cycle. The client request asks for updates (typically with a cursor indicating what it's already seen). The server holds the request — instead of responding immediately, it waits (registering interest, parking the request asynchronously) until there's data for this client or a timeout is reached. Data or timeout: when data arrives (a new message, an update), the server responds with it; if no data arrives within the timeout (say 30 seconds), the server responds empty (a keepalive, preventing the connection from being killed by intermediaries and letting the client re-request). Client re-requests: on receiving a response (data or timeout), the client immediately makes a new request — so there's almost always a held request ready to receive the next push. The immediate re-request is what keeps the push channel continuous.
Middle row: reliability and tuning. Held connections are the resource: one per client, waiting — the scaling constraint, requiring async servers. Cursor / since-token is the reliability mechanism: the client tracks its position (last message ID/timestamp), sends it with each request, and the server returns everything since — so messages arriving in the gap between responses aren't lost; they're delivered on the next request. Timeout tuning balances latency and load: longer timeouts (fewer reconnections, less overhead, but connections held longer and intermediaries may kill idle ones); shorter timeouts (more reconnection overhead but more resilient to intermediary timeouts) — typically 20-60 seconds, tuned to the environment's intermediary timeouts. vs WebSocket/SSE: WebSocket for high-frequency bidirectional, SSE for server-to-client streaming, long polling for compatibility-critical or occasional-update cases — and long polling as the fallback when the others fail.
Bottom rows: scaling and role. Scaling held connections: async servers (non-blocking IO, event loops, or virtual threads) hold many waiting connections cheaply; the backend must support registering interest and notifying held requests when data arrives (pub/sub, a notification mechanism) rather than each held request polling the data source (which would be a database-hammering disaster). Compatibility is the key advantage: long polling works through every HTTP intermediary, in every client, with no upgrade — the reason it's the reliable fallback and the choice for restrictive environments. The ops strip: connection limits (held connections bounded by server capacity, sized for the client count), cursor durability (the cursor mechanism reliable across reconnections and server restarts, so no messages are lost), and fallback role (long polling as the graceful degradation when WebSocket/SSE are unavailable, with the client detecting failure and falling back).
End-to-end flow
Trace a notification system using long polling for compatibility. The application serves a corporate audience behind proxies that sometimes block WebSocket upgrades, so long polling is chosen for reliable reach. A client requests GET /notifications?since=1042 (cursor = last notification ID seen). The server checks: any notifications after 1042? No — so it holds the request (registering this client's interest with the notification pub/sub, parking the request asynchronously on the event-loop server). 20 seconds later, a notification (ID 1043) is created; the pub/sub notifies the held request; the server responds with notification 1043; the client displays it and immediately re-requests GET /notifications?since=1043. The push arrived in near-real-time (the moment the notification was created, the held request delivered it), over plain HTTP, through the corporate proxy that would have blocked a WebSocket.
The reliability vignette shows the cursor's importance. Between the server responding with 1043 and the client's re-request arriving, notification 1044 is created — a message in the gap. Because the client re-requests with since=1043, the server sees 1044 is after the cursor and returns it immediately (no holding needed — there's already data) — the gap message delivered on the next request, not lost. Had the implementation used no cursor (just 'give me new notifications'), 1044 would have been created with no held connection and lost. The cursor turned a message-loss window into reliable delivery — the difference between correct and naive long polling.
The scaling and fallback vignettes complete it. The system has 50,000 concurrent clients, each holding a connection — a blocking thread-per-connection server would need 50,000 threads (impossible); the async event-loop server holds all 50,000 cheaply (each is just a parked request awaiting a pub/sub notification, no thread). The backend uses pub/sub (not per-request database polling — 50,000 requests polling a database every second would melt it), so held requests are notified on data arrival efficiently. And long polling serves as a fallback in a different part of the product: a real-time feature primarily uses WebSockets, but when a client's WebSocket fails to connect (blocked by a proxy), it falls back to long polling — degraded efficiency but reliable delivery, the graceful degradation that keeps the feature working for everyone. The consolidated discipline: cursors for reliable gap-free delivery, async servers to hold many connections cheaply, pub/sub backends to notify held requests efficiently, timeout tuning for the environment's intermediaries, and long polling chosen deliberately for compatibility-critical cases or as the fallback for richer protocols — because it's less efficient than WebSockets but works everywhere HTTP does, and sometimes that reach is worth the overhead.