HTTP/3 and QUIC Architecture in Depth: streams over UDP, the handshake, QPACK, migration and deployment

The problem HTTP/2 could not solve

HTTP/2 fixed HTTP/1.1's one-request-per-connection problem by interleaving many streams as frames on a single TCP connection. But TCP delivers bytes strictly in order. If segment 7 is lost and segments 8 to 20 arrive, the kernel holds all of them until 7 is retransmitted, even when 8 to 20 belong to completely different responses. On a mobile link with 1 to 2 percent loss, HTTP/2 can do worse than several parallel HTTP/1.1 connections.

Fixing that requires the transport to know about streams. TCP lives in kernels and is inspected by countless middleboxes, so changing it takes a decade. QUIC builds the new transport on UDP, implements it in user space where it ships with the application, and encrypts almost all of the header so middleboxes cannot come to depend on its details.

QUIC packets, frames and packet numbers

A QUIC packet sits inside a UDP datagram, and several packets can be coalesced into one datagram. Long-header packets are used during the handshake; short-header packets carry almost all application data and expose little more than a destination connection ID. Inside each packet are frames: STREAM frames carry data for one stream at an offset, ACK frames acknowledge ranges of packet numbers, CRYPTO frames carry TLS handshake bytes, and control frames adjust flow control, connection IDs and path validation.

Packet numbers are never reused. When data is lost, QUIC resends the frames in a new packet with a new number, instead of retransmitting the same packet. That removes TCP's retransmission ambiguity: an acknowledgement always identifies exactly which transmission arrived, so round-trip time samples are clean.

There are three packet number spaces, Initial, Handshake and Application data, each with its own keys and acknowledgements, so handshake loss is recovered independently of application loss. Many fields in the wire format use a variable-length integer encoding, and stream IDs encode who opened a stream and in which direction. Both are small enough to decode by hand:

def read_varint(buf, i=0):
    """QUIC variable-length integer (RFC 9000 section 16).
    The top two bits of the first byte give the length: 1, 2, 4 or 8 bytes."""
    first = buf[i]
    length = 1 << (first >> 6)
    value = first & 0x3F
    for b in buf[i + 1:i + length]:
        value = (value << 8) | b
    return value, i + length

def stream_kind(stream_id):
    """The two low bits of a stream ID encode initiator and direction."""
    initiator = "server" if stream_id & 0x1 else "client"
    direction = "uni" if stream_id & 0x2 else "bidi"
    return initiator, direction

assert read_varint(bytes([0x25])) == (37, 1)
assert read_varint(bytes([0x7B, 0xBD])) == (15293, 2)
assert stream_kind(0) == ("client", "bidi")     # first request
assert stream_kind(2) == ("client", "uni")      # e.g. client control stream
assert stream_kind(3) == ("server", "uni")

HTTP/3 uses client-initiated bidirectional streams (IDs 0, 4, 8 and so on) for requests, one request and response per stream. Each side also opens unidirectional streams: a control stream carrying SETTINGS and GOAWAY, and two QPACK streams for header compression state.

The handshake: transport and TLS in one round trip

Over TCP, a new HTTPS connection costs one round trip for the TCP handshake and at least one more for TLS 1.3 before a request can be sent. QUIC carries the TLS 1.3 handshake inside its own CRYPTO frames, so transport and cryptographic setup happen together: the client's first flight contains the TLS ClientHello plus its transport parameters, the server answers with its handshake, and the client can send its request after one round trip.

The client's first datagram must be padded to at least 1,200 bytes. That tests that the path carries QUIC's minimum packet size, and it limits abuse: until a server has validated the client's address, it may send at most three times the bytes it has received from that address. A server under load can go further and answer with a Retry packet containing a token, which the client must echo, proving it can receive at its claimed address at the cost of one extra round trip.

On a resumed connection, a client may send 0-RTT data in its very first flight, using keys from a previous session. The request arrives with no round trip at all, but 0-RTT data can be replayed by an attacker who captures it, because the server has not yet seen fresh input from this handshake. Only idempotent requests belong in 0-RTT. Servers that accept it should reject anything unsafe, and HTTP defines status 425 Too Early (RFC 8470) for asking the client to retry after the handshake completes.

Streams, flow control and congestion control

Every stream is an independent ordered byte sequence. A lost packet carrying stream 8's data delays only stream 8; streams 0 and 4 continue to be delivered. Streams are cheap to open, and the number each side may open is itself flow controlled with MAX_STREAMS, which is how a server limits concurrent requests.

Flow control has two levels: MAX_STREAM_DATA limits each stream's buffered data and MAX_DATA the whole connection, so one unread download cannot starve the rest if the connection window exceeds any stream's window.

QUIC has congestion control, just as TCP does. RFC 9002 describes a NewReno-style baseline, and implementations commonly ship CUBIC and BBR. Because the controller lives in the QUIC library rather than the kernel, it changes with a deploy, and behaviour depends on which library you run, so benchmark the one you ship. QUIC can also carry Explicit Congestion Notification; see ECN, in depth.

QPACK: header compression without head-of-line blocking

HTTP/2's HPACK compresses headers with a dynamic table that both sides update in the order header blocks arrive. That works over TCP because order is guaranteed. Over QUIC, request streams arrive in any order, so a header block that references a table entry inserted by a delayed packet could not be decoded correctly.

QPACK, RFC 9204, moves table updates onto a dedicated encoder stream and acknowledgements onto a decoder stream. A header block can reference only entries it knows are inserted; if it references an entry that has not arrived yet, that request stream is blocked until it does. The decoder bounds this with the setting SETTINGS_QPACK_BLOCKED_STREAMS, and SETTINGS_QPACK_MAX_TABLE_CAPACITY bounds its memory. An encoder that never references unacknowledged entries never blocks, trading compression ratio for latency.

Connection IDs, migration and load balancing

A TCP connection is identified by its four-tuple of addresses and ports; when a phone moves from Wi-Fi to cellular, the connection dies. A QUIC connection is identified by connection IDs chosen by each endpoint. When the client's address changes, packets keep arriving with a recognised connection ID, the server validates the new path with PATH_CHALLENGE and PATH_RESPONSE frames, and the transfer continues. Endpoints switch to spare IDs on migration so observers cannot link the paths.

Connection IDs are also what makes QUIC hard for infrastructure. A layer-4 load balancer hashing on the four-tuple will send a migrated connection, or even one whose NAT rebinding changed its port, to a different backend that knows nothing about it. Robust deployments encode a server identifier in the connection IDs they issue so that the load balancer can route on it; the IETF QUIC-LB draft described such an encoding but expired without becoming an RFC. Within one machine, nginx's quic_bpf directive uses an eBPF program so that packets reach the worker that owns the connection.

Worked example: loading a page on a lossy network

A phone on a network with an 80 ms round trip and 2 percent loss opens a news page that needs the HTML plus 40 resources. Over HTTP/2 on TCP, setup costs about 160 ms: one round trip for TCP and one for TLS 1.3. Over a first-visit HTTP/3 connection it costs about 80 ms, and for a returning visitor sending an idempotent GET in 0-RTT, the request leaves with the first packet.

Over the load, roughly 8 of about 400 packets are lost. With TCP each loss blocks delivery of every response behind it for at least one round trip, so all forty resources inherit the stalls. With QUIC each loss delays only the streams whose frames were in that packet, typically one or two resources, while the rest render.

If the user then walks out of Wi-Fi range, TCP requests fail and restart with new handshakes, while a QUIC connection whose server and load balancer support migration validates the new path and continues. On a clean wired network these gains shrink to the saved handshake round trip, which is why datacentre benchmarks often show HTTP/3 no faster than HTTP/2.

Deploying it: discovery, fallback and the kernel

A browser cannot know in advance that a server speaks HTTP/3, because the first contact is usually over TCP. Servers advertise it in an Alt-Svc response header, or more directly in an HTTPS DNS record (RFC 9460) that lists h3 among its ALPN values. Browsers then race QUIC against TCP and use whichever works, in the spirit of Happy Eyeballs. Some enterprise and public networks block UDP 443, so TCP fallback is not optional: always serve HTTP/2 on the same name.

nginx supports HTTP/3 through its ngx_http_v3_module, which appeared in version 1.25.0 and is documented as experimental (http2 on needs 1.25.1 or later):

server {
    # same port for HTTP/3 (UDP) and HTTP/1.1 + HTTP/2 (TCP)
    listen 443 quic reuseport;
    listen 443 ssl;
    http2 on;

    ssl_certificate     /etc/nginx/tls/example.crt;
    ssl_certificate_key /etc/nginx/tls/example.key;
    ssl_protocols       TLSv1.2 TLSv1.3;  # QUIC needs 1.3; keep 1.2 for TCP fallback

    quic_retry on;    # validate client addresses with a Retry token
    quic_gso   on;    # batch UDP sends with generic segmentation offload

    location / {
        # tell TCP clients that HTTP/3 is available on UDP 443 for a day
        add_header Alt-Svc 'h3=":443"; ma=86400' always;
        proxy_pass http://app_backend;
    }
}

Kernel tuning matters more than it did for TCP. QUIC sends and receives many UDP datagrams from user space, so per-packet system call cost dominates CPU. Generic segmentation offload lets one call send a train of equal-sized datagrams, and receive offload does the reverse; without them, HTTP/3 can use noticeably more CPU per byte than TCP with TLS. Undersized socket receive buffers show up as drops under bursts. Packet size is limited by the path, so QUIC's own path MTU probing is relevant here; see path MTU discovery.

# Does the server speak HTTP/3? (needs a curl built with HTTP/3 support)
curl --http3-only -sI https://example.com/ | head -1

# Is it advertised to browsers that arrive over TCP?
curl -sI https://example.com/ | grep -i alt-svc

# Is UDP 443 open on the path? A timeout here, with TCP working, means a
# middlebox drops UDP and clients will fall back.
curl --http3-only -sI --max-time 5 https://example.com/ || echo "no h3 path"

# Linux receive buffers: QUIC is bursty UDP, defaults are often too small
sysctl net.core.rmem_max net.core.wmem_max

Failure modes

Trade-offs

HTTP/3 buys faster setup, independent streams and connections that survive network changes. It pays with more CPU per byte, less visibility for network tools, a dependence on UDP and connection-ID-aware load balancers. Gains are largest for mobile and lossy clients and smallest inside a datacentre, where HTTP/2 or gRPC over TCP remain good defaults; see CDN architecture for where most sites terminate HTTP/3 first.

What to do next

1. Measure your traffic's real loss and round-trip distribution by client network before promising a speed-up.
2. Terminate HTTP/3 at your CDN or edge first, keeping HTTP/2 over TCP as the fallback on the same hostname.
3. Open UDP 443 through every firewall and verify from an external network with a HTTP/3-capable curl.
4. Advertise with Alt-Svc and, if your DNS provider supports it, an HTTPS record listing h3.
5. Confirm your load balancer routes by connection ID before relying on migration.
6. Enable GSO, raise UDP buffer limits, and compare CPU per request against HTTP/2.
7. Restrict 0-RTT to idempotent requests and test that unsafe ones are refused.
8. Track the HTTP/3 share of requests and p95 page timings per network type after rollout.