Why architecture matters here
Presence architecture matters because the naive version doesn't just occasionally glitch — it is structurally incapable of being correct. The core problem is that a TCP or WebSocket connection dying is often invisible: a client that loses power, drops off Wi-Fi, or gets NAT-timed-out sends no FIN, so the server holds a half-open socket that looks perfectly alive until it tries to write and fails minutes later. If 'online' means 'we have a socket,' every one of those becomes a ghost. The only robust fix is to invert the logic: presence must be affirmatively renewed. The client (or the gateway on its behalf) proves liveness with periodic heartbeats, and the truth lives in a store where the record expires unless refreshed. Absence of a heartbeat, not presence of a disconnect, is what marks a user offline — and that single design choice eliminates the entire ghost class by construction.
The second reason architecture matters is that presence is a fan-out problem in disguise, and fan-out cost is combinatorial. If you have a team chat where 500 people can all see each other's status, one person coming online is potentially 499 notifications; a morning where everyone logs in within a few minutes is hundreds of thousands of status pushes in a burst. Get the subscription model wrong — broadcast every change to everyone — and presence traffic dwarfs the actual application traffic and collapses under its own weight. The architecture has to answer 'who needs to know about this change?' precisely and cheaply, which means a real subscription/interest model, not a firehose.
Third, presence is a place where eventual consistency and latency are in direct tension and both matter. Users forgive a status that's a second or two stale, but not one that's minutes wrong, and not a dot that strobes. That tolerance is actually your budget: it's what lets you debounce, batch fan-out, and accept a small grace window on disconnect. A good presence system spends that tolerance deliberately — a short grace period to survive reconnects, a debounce to suppress flap — rather than either chasing impossible real-time exactness or letting staleness drift into ghosts.
A fourth, easily overlooked reason is that presence is write-heavy and hot in a way most application data is not. Every online user refreshes a key every 20–30 seconds forever, so a million concurrent users generate tens of thousands of writes per second just to say 'still here' — before a single status change is even fan-out. That steady background load shapes the whole storage choice: presence belongs in a fast, sharded, in-memory store with native TTL, not in your primary transactional database, where heartbeat churn would compete with real work and bloat the write-ahead log. The architecture matters because getting the storage tier wrong doesn't just make presence slow — it can drag down the systems presence sits next to.
The architecture: every piece explained
Top row: the connection and truth layer. A client holds a persistent connection (WebSocket, long-lived gRPC stream, or MQTT) and sends application-level heartbeats on an interval — say every 20–30 seconds — because transport keep-alives alone don't prove the application is healthy. The gateway node terminates that socket and is the authority for one thing: this specific session is alive. On each heartbeat it refreshes a key in the presence store — typically Redis or a similar fast KV store — set with a TTL a bit longer than the heartbeat interval (e.g. 45–90 seconds) so one missed beat doesn't drop a user but a truly dead session self-expires. The key is per-session (user + device), not per-user, and often carries metadata: last-seen timestamp, device type, declared status. The pub/sub bus carries presence deltas — 'session S of user U went online/offline' — between nodes, because the gateway that owns a connection and the gateways that own the watchers are usually different machines.
Middle row: aggregation and interest. Multi-session handling is the reconciliation step: a user is 'online' if any of their sessions is alive, 'away' if all are idle, 'offline' only when the last session expires — so the user-level status is a fold over the device-level keys. The state machine classifies each session and the aggregate: online (active), idle (connected but no user activity for N minutes), away (explicitly set), offline (no live session). The subscription set is the interest graph — for each observer, which users' presence they need — derived from friend lists, channel membership, or the roster currently on screen. The fan-out engine consumes deltas off the bus, looks up who subscribes to the changed user, and pushes the update to exactly those observers, batching and coalescing where it can.
Bottom rows: smoothing and bootstrap. Grace and debounce are the flap absorbers: a disconnect starts a short grace timer rather than immediately publishing offline, so a reconnect within the window is invisible to observers; rapid up/down transitions are debounced so only the settled state propagates. The snapshot API handles the cold-start problem — when an observer opens a chat, they need the current roster immediately, so they fetch a snapshot of all subscribed users' statuses once and then receive incremental deltas, rather than waiting for changes to learn the initial state. The ops strip lists what dominates real operation: heartbeat/TTL tuning, reaping ghosts whose keys or nodes died uncleanly, keeping fan-out fair so one hot user doesn't starve others, and bounding subscription cardinality.
End-to-end flow
Trace a user through a busy morning. Priya opens the app on her phone and, seconds later, her laptop. Each client connects to whichever gateway the load balancer chose — different nodes. The phone session sends its first heartbeat; gateway A writes presence:priya:phone with a 60-second TTL and, seeing this is the first live session for Priya, publishes a user_online(priya) delta to the bus. The laptop's first heartbeat on gateway B writes presence:priya:laptop; gateway B checks and sees Priya was already online, so it publishes only a session-added delta, not a redundant user-online. Aggregation is doing its job: two devices, one green dot.
The fan-out engine picks up user_online(priya), looks up Priya's subscribers — the 40 people whose rosters include her — and pushes the update to the gateways currently holding those observers' sockets, which deliver it down the wire. Within a second, Priya's contacts see her come online. Meanwhile 5,000 other people are logging in during the same ten-minute window; the fan-out engine batches deltas per destination gateway and coalesces multiple changes into single frames, so an observer watching 200 people who all just came online receives a handful of batched updates rather than 200 individual pushes.
Now the messy reality. Priya walks into an elevator; her phone's network dies with no FIN. Gateway A's socket is now a ghost, but the heartbeats simply stop arriving. The presence:priya:phone key isn't refreshed and, 60 seconds later, expires on its own — no disconnect event required. But her laptop is still beating, so the aggregate stays online: her contacts never see a blip. This is the TTL design paying off. Contrast the flap case: Priya's train enters a tunnel and her laptop reconnects every few seconds. Without protection, observers would see her strobe. Instead, each disconnect starts a 10-second grace timer; because she reconnects within it every time, no offline delta is ever published — the debounce swallows the noise, and observers see a steady online.
Finally the cold start and the storm. A new observer, Sam, opens the team channel. His client calls the snapshot API, which reads the current aggregate status for all 200 channel members in one shot and returns the roster; from then on Sam receives only deltas. Later, a gateway node crashes, dropping 50,000 sockets at once. All those clients reconnect within seconds — a thundering herd — and if every reconnect immediately republished online, the bus would spike hard. The system dampens this: reconnects land within the grace window of the sessions' brief absence where possible, heartbeats re-establish keys that may not have even expired yet, and the fan-out engine rate-limits and batches the resulting deltas so the recovery is a swell, not a tidal wave. Every mechanism in the architecture earns its place in this one narrative.
One subtlety worth drawing out from this walk-through: the user-level status was never stored directly — it was always derived, on demand, by folding over the live per-session keys. When Priya's phone session expired, nothing had to explicitly recompute her status; the next reader simply saw one live session instead of two and concluded 'online.' This derive-don't-store discipline is what keeps multi-device presence correct under partitions and races: there is no separate aggregate to fall out of sync with the sessions it summarizes. The sessions in the TTL store are the truth, and the status is a pure function of them — which is exactly why two gateways that briefly disagree about one device don't corrupt Priya's overall dot, since both compute the same answer from the same shared keys.