Why architecture matters here
The commit log matters because it is the single point at which Cassandra converts a fast in-memory write into a durable one, and its behavior defines the exact meaning of a successful write acknowledgment. When a client writes at a given consistency level and the replica acknowledges, what has actually been made durable depends entirely on the commit log's sync policy. Understanding that is the difference between believing your data is safe and discovering, after a power outage, that the last few milliseconds of acknowledged writes evaporated.
The architecture exists to avoid random writes on the hot path. If Cassandra updated its SSTables in place on every mutation, each write would require seeking to the right place in a large on-disk structure and modifying it — a random-access pattern that destroys throughput. Instead the log-structured design writes sequentially in two places: the commit log for durability and the memtable for queryability, and only later rewrites data into sorted SSTables during a flush. The commit log is the durability half of that bargain, and its sequential-append nature is precisely why it is cheap enough to be on the critical path of every write.
It also matters for recovery time. A node that restarts after a crash must replay every commit log segment that had not been fully flushed to SSTables. If flushes are infrequent or memtables are huge, the backlog of unflushed segments grows, and replay on restart takes longer — extending the window before the node can rejoin and serve. The commit log thus ties together three operational concerns that operators often treat separately: write latency, data-loss risk on power failure, and restart time. All three are governed by how the log is synced and how flushes are scheduled.
Finally, the commit log interacts with the cluster's replication model in a subtle way. Cassandra's overall durability comes from writing to multiple replicas, so the loss of one node's unflushed writes is often recoverable from peers via hints and repair. That is why the default sync mode is periodic rather than fsync-on-every-write: the design leans on replication for cross-node durability and uses the commit log for per-node crash recovery, accepting a small local data-loss window in exchange for much higher throughput. Deciding whether that trade-off is acceptable for a given table is an architectural decision, not a default to accept blindly.
The architecture: every piece explained
Segments are the physical form of the commit log. The log is not one ever-growing file but a series of fixed-size segment files. Writes append to the current active segment; when it fills, Cassandra rolls over to a new one. Old segments are kept only as long as they contain data that has not yet been flushed to an SSTable. Once every mutation in a segment has been flushed, the segment is recycled or deleted, which is how the commit log stays bounded rather than growing without limit.
The memtable is the in-memory companion. Each table has a memtable holding its recent writes in a sorted structure keyed by partition and clustering columns. A write updates the memtable immediately after the commit log append, so reads can see recent data without touching disk. When a memtable grows past a threshold — or when commit log pressure demands it — it is flushed to an immutable SSTable on disk, and a fresh empty memtable takes its place.
The sync policy is the durability knob. In periodic mode (the default), the commit log is fsynced to disk on a fixed interval — commonly around ten seconds — and writes are acknowledged as soon as they are written to the log's in-memory buffer, before that buffer is guaranteed on disk. This is fast but means a power failure can lose up to one sync interval of acknowledged writes. In batch mode, a write is not acknowledged until the log has been fsynced, so no acknowledged write is ever lost to a crash, at the cost of higher and burstier write latency. A related group mode batches fsyncs over a short window to amortize the cost while keeping the guarantee tight.
Replay is the recovery mechanism. On startup Cassandra scans the commit log directory, identifies segments whose mutations were not yet flushed (tracked via per-table flush positions), and replays those mutations back into fresh memtables. Each mutation carries enough information — table, partition, and a checksum — for replay to apply it correctly and to skip records corrupted by a torn write at the moment of the crash. After replay the node's in-memory state matches what it had acknowledged, minus any writes lost inside an un-synced periodic window.
Sizing and placement round out the architecture. The total commit log space is capped by configuration; when the cap is approached, Cassandra forces memtable flushes to free the oldest segments. Placing the commit log on a separate physical device from the SSTable data directories was historically important on spinning disks so that sequential log writes were not contending with random SSTable I/O; on modern SSDs the separation matters less but the principle — keep the log's sequential write stream unobstructed — still guides high-throughput deployments.
End-to-end flow
Trace a single write. A coordinator routes a mutation to a replica; the replica's storage engine takes the mutation and appends it to the active commit log segment's buffer. In periodic sync mode the write is now considered logged and the engine applies it to the appropriate memtable, updating the in-memory sorted structure. With both the log append and the memtable update done, the replica acknowledges the write back to the coordinator. The actual fsync of the log buffer to stable storage happens on the periodic timer, up to a sync interval later.
Writes keep flowing, filling the active segment. When it reaches its fixed size, Cassandra rolls over: it starts a new active segment and the old one becomes a completed segment, still retained because it contains mutations not yet flushed. Segments accumulate as long as their data lives only in memtables.
Eventually a memtable crosses its flush threshold. Cassandra flushes it to a new immutable SSTable and records, per table, the commit log position up to which that table's data is now safely on disk in SSTable form. When every table's flush position has advanced past a given segment — meaning all of that segment's mutations are now in SSTables — the segment is no longer needed for recovery and is recycled. This is the loop that keeps the commit log bounded: writes append, flushes retire old segments.
Now suppose the node loses power. On restart, Cassandra opens the commit log directory and finds segments that were never fully retired. It reads them sequentially, verifying each record's checksum and skipping any trailing torn record from the crash, and replays every valid mutation into fresh memtables. Once replay finishes, the node's memtable state is reconstructed to match what it had acknowledged before the crash — with the one caveat that in periodic mode any writes acknowledged but not yet fsynced within the last sync window are gone, which is why cluster-level replication and hints exist to backfill those from peers.