Why architecture matters here
The architecture matters because the alternative — updating rows in place on disk — is exactly the workload disks are worst at. An update-in-place engine must find the existing record, read the page it lives on, modify it, and write it back, which for a random write pattern means a seek per operation and a read-modify-write cycle that no amount of caching fully hides under heavy write load. Cassandra sidesteps this entirely by never updating anything on disk. Every mutation is an append: to the commitlog sequentially, and to the memtable in memory. The disk only ever sees large sequential writes — the commitlog append and, later, the flush — which are the one access pattern that stays fast as volume grows. This is why Cassandra's write throughput scales the way it does.
It matters because durability and latency are decoupled, which is the whole trick. If Cassandra had to write each mutation to its final sorted location on disk before acknowledging, write latency would be bound by random disk I/O. Instead the commitlog gives durability with a single sequential append, and the memtable gives the sorted structure a read needs, in memory, with no I/O. The client is acknowledged once both accept the write — memory-speed latency with on-disk safety. The flush is what later reconciles these two views, moving the memtable's contents to their permanent home in the background, off the critical path, so the client never waits for it.
The architecture also matters because immutability downstream is a gift that keeps giving. Because a flushed SSTable is never modified, it needs no locking, it can be memory-mapped and shared freely across reader threads, it can be safely copied for backup while live, and its checksums never go stale. The cost of immutability is that updates and deletes accumulate as new records — a delete is a tombstone, a later value shadows an earlier one — and reads must merge across multiple SSTables. That cost is paid down by compaction, but the point is that the flush's decision to write immutable files is what makes every other part of the storage engine simpler and safer.
Finally, it matters because the flush is where memory pressure meets disk throughput, and mismanaging that meeting point is a classic production failure. Flush too rarely and memtables balloon, garbage-collection pauses lengthen, and a crash means a long commitlog replay. Flush too eagerly and you produce a flood of tiny SSTables that overwhelm compaction and degrade reads. Let the commitlog outrun flushing and the log disk fills, stalling all writes. The flush sits at the center of the write path's health, and its tuning — thresholds, sync mode, disk headroom, compaction pacing — is precisely where a Cassandra operator earns their keep.
The architecture: every piece explained
Top row: the durable, fast write. A client write is a single mutation aimed at a partition. Cassandra does two things with it. First, it appends the mutation to the commitlog — a sequential, on-disk log whose only job is durability. This append is cheap because it never seeks; it just extends the current log segment. Second, it inserts the mutation into the memtable, an in-memory sorted structure (per table) that holds recent writes in the same partition/clustering order a read expects. Only once both have accepted the write does Cassandra acknowledge the client. The data is now durable (on the commitlog) and visible to reads (in the memtable) without a single random disk write having occurred.
Middle row: the flush itself. A memtable cannot grow without bound, so a threshold triggers a flush — the memtable reaching a configured size, an age limit elapsing, or Cassandra's global memory manager deciding that total memtable memory across tables is too high and picking the largest to evict. The flush writes the entire memtable out, in one sequential streaming sweep, as a new immutable SSTable — data file, partition index, and bloom filter. Crucially, this does not stop writes: a new memtable is swapped in and begins accepting incoming mutations the instant the old one is marked for flushing, so the write path never blocks. Once the SSTable is safely on disk, the commitlog segments that only protected data now living in that SSTable are no longer needed and are recycled for reuse.
Bottom-left: recovery. The commitlog is the reason a flush can be lazy and asynchronous. If the node crashes before a flush completes, the memtable's contents are gone from memory — but they were all appended to the commitlog first. On restart, Cassandra replays the commitlog, re-applying every mutation that had not yet been flushed into a fresh memtable, reconstructing exactly the state that was lost. This is why the commitlog can only be recycled after the corresponding flush succeeds: until the data is in an SSTable, the log is its sole durable copy.
Bottom-right and ops: the long tail. Flushing produces many SSTables over time, and reads must merge across them, so compaction runs in the background to merge SSTables, discard shadowed values, and drop expired tombstones — keeping read amplification bounded. The ops strip names the levers that keep the write path healthy: tuning the flush threshold so memtables are neither too large nor too small, choosing the commitlog sync mode (periodic vs batch) that matches your durability appetite, keeping disk headroom so the commitlog and flushes never run out of space, and giving compaction enough throughput to keep pace with the flush rate.
End-to-end flow
Trace a burst of writes into a Cassandra node under steady load, from a single mutation through flush, crash recovery, and compaction.
The write arrives: a client sends an insert for a partition. The coordinator routes it to the replica; on the replica, the mutation is appended to the tail of the active commitlog segment — a sequential write of a few hundred bytes — and simultaneously inserted into the table's memtable, slotting into its sorted position in memory. With the periodic sync mode, the commitlog append is buffered and fsynced on a short interval; with batch mode it is fsynced before the ack. Either way, once the write is in the memtable and recorded in the log, the replica acknowledges. Total latency is dominated by memory operations, not disk seeks.
The memtable fills: thousands of writes later, this table's memtable crosses its size threshold. Cassandra marks it as flushing and immediately swaps in a fresh, empty memtable that begins accepting new writes — the burst never pauses. A flush thread takes the frozen memtable, which is already sorted, and streams it to disk as a new SSTable: the data file in partition order, a partition index for seeks, and a bloom filter so future reads can skip this SSTable when a key is absent. No seeks, one big sequential write.
The commitlog is freed: once the SSTable is durably on disk and its checksums verified, every commitlog segment whose mutations are now fully captured in an SSTable is no longer protecting any un-flushed data. Cassandra marks those segments clean and recycles them — the log is a bounded ring, not an ever-growing file. The disk footprint of the commitlog stays flat regardless of how many writes have flowed through, because its only job is to bridge the window between a write and its flush.
A crash intervenes: suppose the node is killed after the fresh memtable has accumulated writes but before it flushes. Those in-memory writes vanish. On restart, Cassandra scans the commitlog, finds the segments still marked dirty, and replays every mutation in them into new memtables, re-applying the exact writes that were in flight. Because every acknowledged write was appended to the log before the ack, nothing acknowledged is lost. Once replay finishes, the node flushes the recovered memtables and resumes normal service — the commitlog has done its one job.
Compaction cleans up: over hours, the flushes have produced many SSTables, and a read for a hot partition might have to merge fragments from several of them, plus the live memtable. Background compaction merges overlapping SSTables into fewer, larger ones: it keeps the newest value for each cell, physically drops values shadowed by later writes, and removes tombstones whose grace period has passed. Read amplification falls back toward one, and the space held by obsolete data and expired tombstones is reclaimed. The write path's append-only simplicity is paid for here, in the background, off the client's critical path.