Why architecture matters here
The architecture matters because the durability level is the exact definition of what a successful HBase write acknowledgment means, and that definition varies by orders of magnitude across the four settings. When a client's Put returns success, whether the data survives a RegionServer crash depends entirely on the Durability that Put used. A team that does not know which level their tables run at does not actually know whether their acknowledged writes are safe — a gap that stays invisible until a server dies and takes recent data with it.
It matters because the trade-off it governs — write latency versus crash safety — is fundamental and unavoidable. Pushing an edit further toward stable storage before acknowledging it makes the write safer and slower; acknowledging sooner makes it faster and riskier. There is no setting that is both fastest and safest, so the architecture forces an explicit choice. The value of HBase's design is that it lets you make that choice at fine granularity: a critical financial table can run FSYNC_WAL while a bulk analytics ingest on the same cluster runs SKIP_WAL, each paying only for the guarantee it needs.
It matters because the WAL is not just about durability; it is also about recovery time and cluster-wide behavior. When a RegionServer dies, its regions are reassigned to other servers, and before those regions can serve, the dead server's WAL must be split and its edits replayed into the new servers' MemStores. A large WAL — the result of infrequent flushes or heavy write volume — makes that replay slow, extending the window during which the affected regions are unavailable. The durability level, the flush cadence, and the recovery time are all coupled, so treating durability as a purely local write-speed knob misses its impact on availability during failures.
Finally it matters because HBase's overall durability story, like Cassandra's, leans partly on the underlying replicated storage. The WAL lives on HDFS, which replicates each block across multiple DataNodes, so the log itself is protected against the loss of any single disk or node even before considering fsync. This is why ASYNC_WAL and SYNC_WAL are viable defaults for many workloads: the HDFS write pipeline already provides strong protection, and forcing a physical fsync on every edit (FSYNC_WAL) is often more durability than the workload's risk tolerance actually requires. Understanding where HDFS replication ends and where fsync begins is what lets you pick a level deliberately rather than superstitiously.
The architecture: every piece explained
The WAL itself is a per-RegionServer, append-only log stored on HDFS. Every edit (a Put or Delete) destined for any region on that server is appended to the same WAL before being applied to the region's MemStore. Being on HDFS means the log is replicated across DataNodes, so it survives individual disk and node failures; being append-only and shared per server means WAL writes are sequential and cheap relative to random HFile updates.
The MemStore is the in-memory companion the WAL protects. Each region (more precisely, each column family in a region) has a MemStore holding recent edits in sorted order. Writes go to the MemStore after the WAL append, so reads see fresh data immediately. When a MemStore fills, it is flushed to an immutable HFile on HDFS, and the WAL entries it covered become eligible for cleanup. Between flushes, those edits exist durably only in the WAL.
SKIP_WAL is the fastest and most dangerous level: the edit is written to the MemStore but never to the WAL at all. Writes are as fast as memory, but if the RegionServer crashes before the MemStore flushes, every unflushed edit is gone with no way to recover it. It is appropriate only for data that can be regenerated or re-ingested — for example a bulk load you can simply re-run — and never for data of record.
ASYNC_WAL appends the edit to the WAL but does not wait for the log to be flushed to the datanodes before acknowledging; the sync happens asynchronously on a short interval. This is fast and, because the append is on HDFS, reasonably safe, but a crash in the small window before the async sync completes can lose the most recent edits. SYNC_WAL (the historical default) appends and performs an hflush, which pushes the data into the HDFS write pipeline and to the DataNodes' memory, before acknowledging — so an acknowledged write survives a RegionServer crash, since the log data is already on other nodes. FSYNC_WAL goes further with hsync, forcing the DataNodes to fsync the data to physical disk before acknowledging, protecting even against a simultaneous power loss of the storage nodes — at the highest latency cost.
The Durability setting is how a level is selected, and its flexibility is the point. It can be set at the table (column family) level as a default, and overridden per operation on an individual Put or Delete, or per batch via the mutation's durability. This lets a single cluster mix guarantees: strict durability for tables that need it, relaxed for throughput-oriented ingest, all without separate infrastructure.
It helps to see the four levels as a ladder of how far an edit is pushed before the RegionServer says 'done', because the cost climbs at each rung. SKIP_WAL stops at memory: nothing is logged, so there is nothing to wait for. ASYNC_WAL appends to the log buffer and returns, letting a background thread flush it moments later. SYNC_WAL waits for hflush, which does not necessarily hit spinning platters but does get the bytes into the memory of multiple HDFS DataNodes — enough to survive the loss of the RegionServer process or machine, because the log data now lives on other hosts. FSYNC_WAL waits for hsync, which additionally forces those DataNodes to flush their page cache to durable media, closing the last gap: a simultaneous power loss across the storage nodes. Each rung buys protection against a strictly larger failure class and charges a strictly higher latency, and knowing exactly which failure class each rung defends against is what turns the durability setting from a guess into an engineering decision.
End-to-end flow
Trace a single Put with SYNC_WAL durability. The client sends the Put to the RegionServer hosting the target region. The RegionServer constructs a WAL edit describing the mutation and appends it to the active WAL, then calls hflush, which streams the appended bytes into the HDFS write pipeline so they land in the memory of the replica DataNodes. Only after hflush returns does the RegionServer apply the edit to the region's MemStore and acknowledge success to the client. At the moment of ack, the edit exists in the MemStore and, durably, replicated across HDFS DataNodes in the WAL.
Writes continue and MemStores grow. When a region's MemStore crosses its flush threshold, the RegionServer flushes it to a new HFile on HDFS. It records the point in the WAL up to which that region's data is now safely in HFiles. Once all regions' edits in an old WAL file are covered by flushes, that WAL file is no longer needed for recovery and is archived or deleted — the mechanism that keeps the WAL from growing without bound.
Now the RegionServer crashes. The HBase Master detects the failure (via its ZooKeeper session expiring) and begins recovery. The dead server's WAL files are split: a process reads the shared per-server log and partitions its edits by region, producing per-region recovered-edits files. The Master reassigns the dead server's regions to surviving RegionServers.
As each region comes up on its new RegionServer, before it serves any request, the server replays that region's recovered-edits — reapplying every WAL edit that had not been flushed to an HFile — into a fresh MemStore. Once replay completes, the region's in-memory state matches exactly what it had acknowledged before the crash, because the SYNC_WAL hflush had guaranteed those edits were on HDFS. The region opens for reads and writes. Had those same writes used SKIP_WAL, there would be nothing to replay and every unflushed edit would simply be lost; had they used FSYNC_WAL, the edits would have been not just on the DataNodes but fsynced to their disks, surviving even a correlated power event. The durability level chosen at write time is precisely what determines how much of the acknowledged history survives this recovery.