WebSocket subprotocols architecture, in depth: negotiation, designing and versioning your own, routing by name, and the auth-token trick

Subprotocols versus extensions

RFC 6455 defines two negotiation mechanisms in the opening handshake, and they are easy to confuse. Extensions, negotiated with Sec-WebSocket-Extensions, change how frames are carried: permessage-deflate compresses payloads and is the only widely deployed one; see permessage-deflate. Several extensions can be active at once and the application usually does not see them.

Subprotocols, negotiated with Sec-WebSocket-Protocol, sit above frames. They define the application-level conversation: which message types exist, their encoding, how ids correlate them, and what each close code means. Exactly one subprotocol, or none, is active per connection. Examples registered with IANA or in common use include mqtt for MQTT over WebSockets, graphql-transport-ws for GraphQL subscriptions, the STOMP versions such as v12.stomp, and wamp.2.json for WAMP.

The negotiation, rule by rule

The client lists the subprotocols it can speak, in order of preference, in the Sec-WebSocket-Protocol request header. The server either selects exactly one of them and echoes it in the same header of its 101 Switching Protocols response, or omits the header entirely to indicate that it selected none. A server must not select a value the client did not offer, and RFC 6455 requires the client to fail the connection if the response names a subprotocol that was not in its request.

GET /ws HTTP/1.1
Host: chat.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
Sec-WebSocket-Protocol: chat.v2.example.com, chat.v1.example.com

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Protocol: chat.v2.example.com

Three consequences are easy to miss. First, the client's order is a preference, not a command; the server decides. Second, if the server picks none, the connection still opens. Nothing in the protocol forces a server to reject a client whose offers it cannot speak, so your server must decide whether to refuse the upgrade or to accept and immediately close. Third, values are tokens: no spaces or commas inside a name, and in the browser the constructor throws a SyntaxError for invalid or duplicate values.

In a browser, the offer is the second argument to the constructor and the result is the protocol property after the open event. An empty string means none was chosen. A careful client checks it before sending anything.

const ws = new WebSocket("wss://chat.example.com/ws",
                         ["chat.v2.example.com", "chat.v1.example.com"]);
ws.addEventListener("open", () => {
  const codec = CODECS[ws.protocol];          // "" if the server chose none
  if (!codec) { ws.close(4000, "no common subprotocol"); return; }
  session = new ChatSession(ws, codec);
});

Server architecture: route by name, once

The cleanest server design treats the subprotocol name as a routing key. At upgrade time, a selection function compares the offered list with a registry of supported names, picks the best one according to server policy, and binds the connection to that protocol's codec and handler for its whole life. Message handlers never inspect payloads to guess the format, which is the root of many subtle bugs when two versions share an endpoint.

With the Node ws library, the selection is the handleProtocols option, which receives the offered names as a Set together with the upgrade request and returns the chosen name or false to select none. Choose by server preference, not by the client's order, so that you can steer clients toward the newest version.

import { WebSocketServer } from "ws";

const REGISTRY = new Map([
  ["chat.v2.example.com", { codec: v2Codec, handler: handleV2 }],
  ["chat.v1.example.com", { codec: v1Codec, handler: handleV1, deprecated: true }],
]);
const SERVER_PREFERENCE = ["chat.v2.example.com", "chat.v1.example.com"];

const wss = new WebSocketServer({
  port: 8080,
  handleProtocols(offered /* Set<string> */, req) {
    for (const name of SERVER_PREFERENCE) {
      if (offered.has(name)) return name;
    }
    return false;                               // none: header omitted from 101
  },
});

wss.on("connection", (ws, req) => {
  const entry = REGISTRY.get(ws.protocol);
  if (!entry) { ws.close(1002, "subprotocol required"); return; }
  metrics.increment("ws.connect", { subprotocol: ws.protocol });
  if (entry.deprecated) metrics.increment("ws.deprecated", { ua: req.headers["user-agent"] ?? "" });
  entry.handler(ws, entry.codec);
});

Python's websockets library has the same shape: pass a subprotocols list to the server and read websocket.subprotocol in the handler. Whatever the stack, emit a metric tagged with the negotiated name on every connection, because it is the only reliable way to know when an old version can be removed.

Designing your own subprotocol

A name is only useful if it stands for a written specification. The specification should answer, at minimum, these questions:

• Name and version. Put the version in the name and use a domain you control, for example chat.v2.example.com, so names never collide with registered ones.
• Encoding. Text frames with JSON, or binary frames with a schema format. Say which, and whether both are allowed.
• Envelope. Every message has a type field and, for anything that expects a reply, an id. Unknown types get a defined response.
• Lifecycle. What must happen first (an init message with credentials or capabilities), how long the server waits for it, and how the session ends.
• Correlation. How a response or stream of results finds its request, and who allocates ids.
• Errors and close codes. Which errors are messages and which end the connection. Application-specific close codes come from the 4000 to 4999 private-use range.
• Liveness and flow control. Whether you rely on protocol-level ping and pong frames or define application-level ones, and what a peer does when it falls behind.

The graphql-transport-ws protocol, specified in the graphql-ws project, is a good reference design. The client must send ConnectionInit first; the server answers ConnectionAck or closes. Operations are started with Subscribe carrying a client-chosen id, results arrive as Next messages with that id, and Error or Complete end the operation; Complete can be sent by either side. Ping and Pong are application messages usable in both directions. Its close codes are precise: 4400 for an invalid message, 4401 for an operation before acknowledgement, 4403 for a forbidden connection, 4408 when ConnectionInit does not arrive within the server's connectionInitWaitTimeout, 4409 for a duplicate subscriber id, and 4429 for repeated initialisation requests.

Notice how much of the specification is about the edges rather than the happy path. That is where interoperability problems appear, and a well-specified close code turns a vague disconnect into a diagnosis.

Versioning and migration

Because exactly one name is chosen per connection, versioning is clean: a new incompatible version gets a new name. Clients offer every version they speak, newest first, and the server picks by its own preference. Deploy servers that speak both versions before shipping clients that prefer the new one, and remove the old name only when its connection metric has stayed near zero for an agreed period.

Do not let a connection with no negotiated subprotocol fall back silently to some default grammar. That fallback is how an unversioned v1 ends up living forever, because you can never tell which clients depend on it. Serve legacy unversioned clients on their own endpoint path if you must, so they are counted separately and can be retired.

Additive, backward-compatible changes, such as a new optional field or a new message type that old peers ignore, do not need a new name if your specification says unknown fields and types are ignored. Write that rule down on day one; it is the difference between a minor release and a migration.

Worked example: moving a chat service from v1 to v2

Consider an illustrative chat service that has spoken an unnamed JSON format for two years, with about 200,000 concurrent connections from web, iOS and Android clients. Version 2 adds message ids, acknowledgements and resumable sessions, which change the meaning of existing messages, so it needs a new grammar.

Step one registers chat.v1.example.com as a name for the existing format and changes no behaviour: the server accepts it, and also accepts connections with no subprotocol on the legacy path. Step two ships clients that offer the v1 name. Within a few weeks the metric shows most web traffic negotiating explicitly, while old mobile builds still send nothing. Step three deploys chat.v2.example.com on the server. Step four ships clients that offer v2 first and v1 second, and enables v2 for a percentage of users by server preference, which the selection function can read from a feature flag keyed on the request. Step five watches error rates, reconnection rates and the deprecated counter, then raises the minimum app version for mobile. Step six, months later, removes v1 when its connections are near zero, and returns an explicit close code with an upgrade message to the stragglers instead of an unexplained failure.

At no point did a handler need to guess which format a message was in, and at every point the team knew exactly how many connections each version carried.

Proxies, load balancers and the token trick

Sec-WebSocket-Protocol is an ordinary request header on the upgrade request, so standards-following proxies and load balancers forward it unchanged. Some layer-7 proxies can route on it, which lets different versions land on different backend pools during a migration. Check your own proxy and CDN documentation for header size limits and logging behaviour rather than assuming; the general load-balancing concerns are in the WebSocket guide.

Browsers do not let JavaScript set arbitrary headers such as Authorization on a WebSocket handshake. Because the subprotocol list is the one header a page can control, some systems smuggle a bearer token inside an offered value. The Kubernetes API server, for example, accepts a token encoded in a specially prefixed subprotocol entry. It works, but it has costs: the token appears in a header that many proxies and servers log, the server must not echo that value back as the selected protocol, and token characters must be encoded into valid token syntax. A safer pattern is a short-lived, single-use ticket: the page fetches it over an authenticated HTTPS request, passes it in the init message or the query string, and the server redeems it once. Whatever you choose, authenticate during the init phase and close with a specific code on failure.

Failure modes

What to do next

1. List every message format your WebSocket endpoints accept today and give each a versioned name under a domain you control.
2. Implement server-side selection by server preference, bind each connection to its codec at upgrade, and refuse or close when no name matches.
3. Make clients check the negotiated protocol on open and refuse to send anything if it is empty or unexpected.
4. Write the specification: encoding, envelope with type and id, lifecycle with an init timeout, error messages and close codes in the 4000 to 4999 range.
5. Emit a connection metric tagged with the negotiated name and alert when a deprecated name grows.
6. Replace any token-in-subprotocol scheme with short-lived tickets, or at least scrub the header from logs.
7. Read related designs such as MQTT over WebSockets and message ordering before extending your own protocol.