Why architecture matters here
Architecture matters here because the difference between fire-and-forget and store-and-forward is the difference between a chat that loses messages and one that does not, and that difference is entirely structural — it lives in whether a durable per-recipient log exists, not in how reliable the network is. You cannot make a pushed message survive a disconnected socket by retrying harder; the socket is gone. The only way a message survives the recipient's absence is if it was written somewhere durable that the recipient reads from when they return. The queue is that somewhere, and its presence or absence is a design decision made before the first message flows.
The core guarantee the queue provides is gap-free, ordered delivery across disconnects. Because every message a recipient should see is appended to their inbox with a strictly increasing sequence id, and because the recipient tracks the last id it processed, the set of messages the recipient still owes is unambiguous: everything with a sequence id greater than its cursor. On reconnect the server replays exactly that set, in id order, so the client's view is identical to what it would have been had it never disconnected. The sequence id is the linchpin — it makes 'what did I miss' a precise, cheap query rather than a guess.
The second structural reason is that it decouples producers from consumers' connectivity. A sender publishing a message should not need to know or care whether each recipient is online; it publishes once, and the fanout writer takes responsibility for durably recording the message for every recipient. This decoupling is what lets a group message to fifty people succeed instantly even if forty of them are asleep with their phones off — their inboxes accept the write, and their clients will collect it whenever they next connect. Without the queue, the sender's success would depend on the recipients' presence, which is nonsensical.
The third is that the queue is where at-least-once delivery is made safe. Exactly-once delivery over an unreliable network is famously impossible; what you can build is at-least-once delivery plus idempotent consumption, which together give the effect of exactly-once. The queue provides the at-least-once half by replaying anything past the cursor — including, unavoidably, messages the client processed but had not yet acked when it dropped. The client provides the idempotent half by deduplicating on message id. The architecture must make both halves explicit; a queue that assumes acks always arrive, or a client that assumes it never sees a duplicate, will corrupt the conversation.
Finally, the architecture has to confront the cost of durability: storage that grows with absence. Every message for every recipient is written and held until that recipient acks it or a retention policy expires it. A user who uninstalls the app but whose account still receives group messages would otherwise accumulate an unbounded inbox forever. So retention — a TTL, a max depth, a policy that stops fanning out to long-dead recipients — is not a nice-to-have; it is what keeps the queue's storage bounded and its replay-on-reconnect from returning a month of backlog. The tension between 'never lose a message' and 'do not store forever' is the central operational trade the queue's design must resolve deliberately.
The architecture: every piece explained
Delivery starts at the fanout writer. A sender publishes one message — a chat line, an event, a notification — and the writer's job is to resolve who should receive it (the members of a room, the followers of a channel, a single direct recipient) and to durably record it for each of them. For every recipient it appends the message to that recipient's inbox log with the next sequence id for that inbox. The fanout is where one logical publish becomes N durable writes, and it is the component that makes the sender independent of recipients' connectivity.
The per-recipient inbox log is the heart of the design: a durable, ordered, append-only sequence of messages, one log per recipient, each entry stamped with a monotonic sequence id. Ordering within a recipient's stream comes for free from the append order, and the sequence id gives every message a stable position that the cursor can reference. This log is what persists a message across the recipient's absence — it exists whether or not the recipient is connected, and it is the source of truth for what the recipient is owed.
The delivery cursor is the small but crucial companion: per client, the sequence id of the last message it successfully processed and acknowledged. The cursor advances only on an ack, and it is what makes reconnect precise — the server replays exactly the entries whose sequence id exceeds the cursor. Cursors are per client rather than per user because one user may have several devices, each at a different point in the stream; each device carries its own cursor so each catches up independently.
The live socket path is the fast lane. When a recipient is connected, the fanout writer (or a notifier watching the log) pushes the just-written message straight down the open socket, so an online client sees messages in real time with no polling. The durable write still happens — the push is an optimization layered on top of the log, not a replacement for it — so if the push fails or the socket drops mid-delivery, the message is still safely in the log for replay. Online delivery is 'write the log and also push'; offline delivery is 'write the log, skip the push.'
Two supporting components keep the queue correct and bounded. Dedup and idempotency live on the consumption side: because replay can re-deliver a processed-but-unacked message, each message carries a stable id and the client (or an idempotency layer just before it) discards ids it has already applied, so at-least-once delivery is safe. Retention and TTL live on the storage side: entries are trimmed once acked, and inboxes are capped by age or depth so a permanently-absent recipient does not accumulate forever — past the cap, the oldest messages are dropped or the recipient is dropped from fanout entirely. The ops strip tracks the health of all of this: inbox depth per user (how much backlog is accumulating), replay size on reconnect (how far behind clients fall), ack lag, how many messages are being dropped by TTL, and the duplicate rate the dedup layer is catching.
End-to-end flow
Follow a message to a client that is offline and back. Alice sends a message to a group of four. The fanout writer resolves the four recipients and appends the message to each of their inbox logs — Bob's inbox gets it as sequence 108, Carol's as 212, and so on, each inbox having its own independent numbering. Three of the four are connected, so the writer also pushes the message down their live sockets and they see it instantly. Bob, however, is offline — his phone is in his pocket, screen dark, socket long since dropped. For Bob there is no push; the message simply sits in his inbox at sequence 108, durably, waiting.
Time passes and more messages accumulate in Bob's inbox — replies from the others land as 109, 110, 111. None of this depends on Bob being anywhere; his inbox is a log that grows as the group talks. His delivery cursor is still at 107, the last message he processed before going offline.
Bob's phone wakes and the app reconnects. As part of the handshake, the client presents its cursor: 'my last-acked sequence is 107.' The server compares this to the head of Bob's inbox and sees entries 108 through 111 are newer than the cursor. It enters replay: it sends 108, 109, 110, 111 in order down the freshly opened socket. Bob's client applies each — deduplicating by message id in case any were partially processed before — and acks them, advancing the cursor to 111. The gap is filled, in order, exactly as if Bob had never disconnected.
Once the replay is caught up to the head of the log, the server switches Bob to the live stream. From this point, new messages are pushed to Bob in real time like any online client, and the durable write continues underneath. The transition from replay to live is a careful moment: the server must ensure no message falls in the seam — a message that arrives while replay is in flight must either be included in the replay or delivered live, never both-dropped nor delivered out of order. Handling that boundary correctly (for example, replay up to a snapshot point, then live from there) is what keeps the stream gap-free across the handoff.
Step back and note the guarantees the structure delivered. Bob lost nothing despite being offline, because every message was durably logged the instant it was sent, independent of his connectivity. He saw everything in order, because the sequence ids and the ordered log made 'what did I miss' an exact range query. He may have seen a duplicate at the seam, but it did no harm because his client deduplicated on message id. And his inbox will not grow without bound, because once he acked up to 111 the server can trim the acked prefix and retention caps the rest. Every one of those properties is a direct consequence of the log-plus-cursor design; none of them exist in a fire-and-forget push.