Why it matters
Compaction is the single largest source of background I/O in most HBase clusters. It reads existing HFiles, sorts and merges them, and writes new ones — often doubling or tripling the disk I/O compared to the raw write rate. Running compactions during peak hours slows every operation.
Under-compacted clusters accumulate HFiles until read latency spikes; over-compacted clusters spend so much I/O on rewriting data that little bandwidth is left for actual work. Getting the balance right is core HBase ops.
The architecture
Minor compactions merge a small set of small HFiles into one larger HFile. They are triggered whenever the number of HFiles per region exceeds a threshold (default 3). They are relatively cheap because they touch only a few HFiles at a time.
Major compactions merge every HFile in a region into a single HFile. They apply tombstones (delete markers) by dropping the deleted data during merge, and they drop expired versions per the column family's TTL and max-versions settings. Major compactions are expensive — they rewrite every byte of the region — but they are the only way to reclaim space from deletions.
How it works end to end
Compaction algorithm walks the input HFiles as a merge-sort, producing a single sorted stream of cells. Duplicate cells (same row+column+timestamp) are deduplicated. Tombstoned cells are dropped if this is a major compaction. Cells past the TTL or max-versions limit are also dropped.
Output is written to a new HFile with the merged content. When the new file is durably written, the RegionServer atomically swaps its store file list to reference the new file and remove the old ones. The old files are then deleted (with a delay so in-flight reads can finish).
Automatic major compactions run periodically (default every 7 days). Many production clusters disable automatic major compaction and run them manually during scheduled maintenance windows to control I/O impact.