Why architecture matters here
Stream multiplexing matters because it enables concurrent requests over a single connection -- eliminating the one-connection-per-request cost and connection-level head-of-line blocking of HTTP/1.1 -- a foundational efficiency of modern protocols (HTTP/2, HTTP/3, gRPC). The old model (a connection per concurrent request, or serialized requests) is inefficient (many connections -- each with handshake and resource cost) or slow (serialized -- head-of-line blocking at the connection level). Multiplexing (many streams over one connection, concurrent) fixes both: concurrent requests share one connection (efficient -- one connection, one handshake -- versus many) and progress in parallel (concurrent -- not serialized). This is a foundational efficiency of modern protocols (HTTP/2, HTTP/3, gRPC all use it) -- enabling efficient, concurrent communication. For modern networked applications (web, APIs, RPCs -- all using multiplexed protocols), understanding multiplexing (how many streams share a connection) is understanding a foundational modern-protocol technique, and it's why HTTP/2+ and gRPC are efficient.
The interleaved-frames insight is the core mechanism, and it's how many streams share one connection. The key mechanism: each logical stream (a request/response or bidirectional flow) is broken into frames (chunks of the stream's data), and each frame is tagged with a stream id (identifying which stream it belongs to). The single connection carries the frames of all the streams, interleaved (frames from different streams mixed on the wire -- e.g., a frame of stream 1, then stream 3, then stream 1 again -- interleaved) -- and at each end, the frames are demultiplexed by their stream id (each frame routed to its stream -- reassembling the streams). So the one connection carries many streams concurrently (their frames interleaved -- progressing in parallel -- not one stream at a time) -- the multiplexing. This interleaving (frames of many streams mixed on the one connection, demultiplexed by stream id) is how many streams share one connection concurrently (versus HTTP/1.1 -- one request at a time per connection). This is the core mechanism -- the frames/stream-ids/interleaving enabling the multiplexing. Understanding the interleaved-frames core (streams broken into frames tagged with stream ids, interleaved on one connection, demultiplexed) is understanding how stream multiplexing works.
And the HTTP/2-still-has-TCP-HOL-blocking-but-HTTP/3-fixes-it reality is a crucial subtlety, because HTTP/2's multiplexing doesn't fully eliminate head-of-line blocking. HTTP/2 multiplexing eliminates the application-level head-of-line blocking (many streams concurrent over one connection -- not serialized). But there's a subtler problem: TCP-level head-of-line blocking. Because all the HTTP/2 streams share a single TCP connection, and TCP delivers data in order (a reliable, ordered byte stream), a lost TCP packet blocks all the streams: TCP won't deliver any subsequent data (of any stream) until the lost packet is retransmitted (since TCP must deliver in order -- the lost packet's data must come first) -- so even though the streams are logically independent, a packet loss affecting one blocks all of them (at the TCP level). So HTTP/2 multiplexing has TCP-level HOL blocking (a lost packet blocking all streams -- since they share the ordered TCP connection). HTTP/3 (over QUIC) solves this: QUIC (built on UDP) implements independent streams (each stream's data delivered independently -- so a lost packet only blocks its own stream -- the other streams unaffected -- no cross-stream HOL blocking). So HTTP/3's independent streams eliminate the TCP-level HOL blocking that HTTP/2 has. This is a crucial subtlety: HTTP/2 multiplexing helps (application-level concurrency) but still has TCP-level HOL blocking; HTTP/3 (QUIC) fully solves it (independent streams). Understanding the HTTP/2-TCP-HOL-blocking-but-HTTP/3-fixes-it subtlety (HTTP/2's shared TCP connection has HOL blocking; HTTP/3's independent QUIC streams don't) is understanding a crucial nuance of stream multiplexing.
The architecture: every piece explained
Top row: the problem and mechanism. The problem: one connection per concurrent request (HTTP/1.1 -- many connections, or serialized requests -- inefficient/slow). Multiplexing: many concurrent logical streams over one connection (sharing the connection -- concurrent). Frames + stream ids: each stream broken into frames tagged with a stream id, interleaved on the wire (the one connection carrying many streams' frames -- demultiplexed by stream id). Flow control: managing the data rate -- per-stream (so one stream doesn't hog the connection) and connection-level (so the receiver isn't overwhelmed) -- flow control at both levels.
Middle row: HOL blocking and priority. HTTP/2 HOL blocking: HTTP/2's multiplexing still has TCP-level head-of-line blocking (a lost TCP packet blocks all the streams -- since they share the ordered TCP connection). HTTP/3 (QUIC): HTTP/3 over QUIC has independent streams (a lost packet only blocks its own stream -- no cross-stream HOL blocking -- solving HTTP/2's TCP-level HOL blocking). Prioritization: which stream's frames go first (prioritizing important streams -- so they aren't delayed behind others) -- managing the streams' relative priority. Connection reuse: the one connection reused for many streams (amortizing the handshake -- versus many connections' handshakes) -- efficiency.
Bottom rows: gRPC and limits. gRPC + HTTP/2: gRPC builds on HTTP/2's multiplexing (many RPCs -- streams -- over one connection -- multiplexed RPCs -- efficient, concurrent RPCs). Concurrency limits: a max number of concurrent streams per connection (limiting the streams -- so a connection isn't overwhelmed -- and the client/server managing within the limit). The ops strip: stream limits (the max concurrent streams -- configured -- balancing concurrency against the connection's capacity -- and the client managing within it), flow control (the flow control -- per-stream and connection -- tuned so streams share the connection well -- no starvation or overwhelm), and monitoring (monitoring the multiplexing -- the streams, their concurrency, flow control, and any HOL blocking -- for performance).
End-to-end flow
Trace multiplexed concurrent requests. A client needs to make several requests concurrently. With HTTP/1.1, it would open several connections (one per concurrent request -- each with a handshake) or serialize them (slow). With HTTP/2 multiplexing: the client opens one connection and sends the several requests as concurrent streams (each request a stream -- broken into frames tagged with the stream id). The frames of the different streams are interleaved on the one connection (the requests progressing concurrently -- their frames mixed on the wire), and the responses come back the same way (interleaved response frames -- demultiplexed by stream id at the client). So the several requests are done concurrently over one connection (efficient -- one connection, one handshake -- and concurrent -- the streams in parallel) -- versus HTTP/1.1's many connections or serialization. The multiplexing (many streams over one connection) made the concurrent requests efficient and concurrent.
The HOL-blocking and gRPC vignettes show the subtlety and application. A HOL-blocking case: with HTTP/2, a TCP packet is lost (a network issue). Because all the streams share the one TCP connection (ordered delivery), the lost packet blocks all the streams (TCP won't deliver subsequent data of any stream until the lost packet is retransmitted -- TCP-level HOL blocking) -- so even the streams not directly affected by the lost packet are stalled. With HTTP/3 (QUIC), the streams are independent (a lost packet only blocks its own stream -- the other streams continue) -- so the packet loss doesn't stall all the streams (only its own) -- HTTP/3 solving the TCP-level HOL blocking. The HTTP/3 independent streams avoided the cross-stream HOL blocking. A gRPC case: the team uses gRPC (built on HTTP/2 multiplexing) -- so many gRPC RPCs share one connection (multiplexed RPCs -- concurrent, efficient) -- leveraging the multiplexing for efficient RPCs (versus a connection per RPC). The gRPC multiplexing gave efficient concurrent RPCs.
The flow-control and limits vignettes complete it. A flow-control case: one stream sends a lot of data (a large upload) -- and without flow control, it could hog the connection (starving the other streams) or overwhelm the receiver. The per-stream and connection-level flow control manages this (limiting each stream's rate -- so it doesn't hog the connection -- and the connection's total -- so the receiver isn't overwhelmed) -- so the streams share the connection fairly (no starvation) and the receiver isn't overwhelmed. The flow control balanced the streams. A limits case: the team configures the max concurrent streams per connection (limiting the streams -- so a connection isn't overwhelmed by too many -- and the client managing within the limit -- opening a new connection if it needs more concurrency than the limit) -- managing the stream concurrency within the limit. The consolidated discipline the team documents: use stream multiplexing (many concurrent streams over one connection -- their frames interleaved, demultiplexed by stream id) for efficient concurrent communication (versus HTTP/1.1's connection-per-request), understand the frames/stream-ids/interleaving mechanism, use flow control (per-stream and connection-level -- so streams share the connection fairly), recognize HTTP/2's TCP-level HOL blocking (a lost packet blocking all streams) and use HTTP/3 (QUIC's independent streams) to solve it where the HOL blocking matters, leverage prioritization (important streams first) and connection reuse (amortizing handshakes), use gRPC (multiplexed RPCs on HTTP/2), manage the concurrency limits (max streams per connection), and monitor the multiplexing -- because stream multiplexing enables concurrent requests over a single connection (eliminating the connection-per-request cost and connection-level HOL blocking of HTTP/1.1), a foundational efficiency of modern protocols (HTTP/2, HTTP/3, gRPC), with HTTP/3 (QUIC) fully solving the TCP-level HOL blocking that HTTP/2 retains.