Why architecture matters here
Bidi streams promise a nice model — write messages in either direction — but production surprises come from HTTP/2 internals. A slow reader consumes flow-control window; the sender stalls silently. A load balancer forces GOAWAY; streams drop mid-conversation. Keepalive misconfigured means half-open connections linger for hours.
The architecture matters because the fix is not "use gRPC"; it is understanding how HTTP/2 works and configuring the client, server, and load balancer to cooperate.
With the pieces mapped, a bidi service becomes a durable long-lived resource, not a fragile session.
The architecture: every piece explained
The top strip is the framing. Client stream sends messages via HTTP/2 frames (HEADERS, DATA). Multiplexed streams share one TCP connection; each stream has its own ID and flow state. Server stream writes back on the same connection.
The middle row is the control mechanisms. Flow control uses window updates at both connection and stream level; when the receiver's window is zero, the sender must stop until a WINDOW_UPDATE arrives. Backpressure is how the application observes this — the async sender pauses on write when the wire is full. Deadlines are per-call timeouts; deadlines propagate downstream through child calls. Keepalive + ping detect dead connections; misconfigured they cause noise, correctly configured they save you.
The lower rows are resilience. Reconnect + retries use exponential backoff with jitter; hedging and retry policies must be written for idempotent semantics. Load balancer is critical — L4 balancers pin to one backend; L7 (or client-side name resolution + subchannels) enables real load balancing across backends. Observability tracks per-stream latency, flow control stalls, and GOAWAY events.
End-to-end flow
End-to-end: a chat client opens a bidi stream to the server. Client sends the first message; HTTP/2 frames flow. Server responds. Both stream for hours. Server needs to drain for a deploy; it sends GOAWAY with last-stream-id. Existing streams continue until natural close; new streams route to a fresh backend. Meanwhile, a slow client stops reading; flow control window drops to zero; server's write blocks; backpressure propagates. Keepalive pings run every 60s; a dead peer is detected within 2 pings and connection closed. Observability shows median stream age, GOAWAY count, and reconnect success rate. Load balancer is L7 gRPC-aware; new streams distribute properly.