Why it matters

Almost every HBase performance question comes down to RegionServer tuning. Write latency, read latency, GC pauses, memory pressure — all of these are RegionServer concerns. Getting RegionServer configuration right is the highest-leverage HBase ops task.

Memory allocation between memstore, block cache, and JVM heap determines whether the RegionServer is write-optimized or read-optimized. Getting the split right for your workload matters more than any other tunable.

Advertisement

The architecture

The write-ahead log is a single append-only file per RegionServer (or per region in newer versions), stored on HDFS. Every mutation is appended to the WAL before it is applied to the memstore. If the RegionServer crashes, the WAL is replayed on the recovering RegionServer to reconstruct in-memory state.

The memstore is an in-memory sorted structure holding writes not yet flushed to HDFS. There is one memstore per column family per region. When the memstore fills (default 128 MB), it flushes to HDFS as a new HFile. Flushes happen per-region: when total memstore across all regions exceeds a global threshold, the RegionServer flushes the biggest.

RegionServer — the workhorse of HBaseWALdurability logMemStore per CFin-memory writesBlock cache (LRU)hot HFile blocksServes many regions concurrently — each region has its own memstore + HFile set
RegionServer internals: WAL for durability, MemStore for writes, Block Cache for reads.
Advertisement

How it works end to end

Writes flow like this: client sends a put to the RegionServer; RegionServer appends to WAL; on WAL sync, the mutation is applied to the memstore; the client gets an acknowledgment. Because the WAL sync is the durability boundary, this is where write latency is dominated.

Reads flow the other way: RegionServer checks the memstore first for the latest values, then consults HFiles on HDFS. HFile lookups are accelerated by the block cache, which holds recently-read HFile blocks. Bloom filters on each HFile let the RegionServer skip HFiles that provably do not contain the target row.

Compaction is the background process that merges HFiles. Minor compactions merge a few small HFiles into a bigger one; major compactions merge all HFiles for a region into one, honoring versions and tombstones. Compaction is what keeps read latency bounded as writes accumulate.