Why it matters
HBase read latency is largely determined by HFile behavior. If the block cache hits, reads are microseconds. If they miss and go to HDFS, reads are milliseconds. Bloom filters can skip entire HFiles that provably do not contain the target row. HFile encoding and compression affect both storage cost and CPU cost per read.
Getting HFile configuration right — block size, compression, encoding, bloom filter type — is one of the most impactful HBase tuning choices.
The architecture
An HFile has four logical sections. The data section contains sorted key/value cells packed into blocks (default 64 KB uncompressed). The index section holds a hierarchical index that maps row keys to the block containing them, with root + intermediate + leaf indices for large files. The bloom filter section holds probabilistic membership filters for rows or row+column combinations. The metadata and trailer sections hold version info, block encoding options, and pointers to every other section.
The trailer is read first because it contains offsets and lengths of all other sections. From the trailer, the RegionServer can jump directly to the index, then to a specific data block, without reading the whole file.
How it works end to end
Point lookup flow: check the bloom filter first; if it says the row is absent, skip this HFile entirely. Otherwise consult the index to find which data block might contain the row, load that block (from block cache or HDFS), and scan within it. All this typically requires zero to two HDFS reads.
Scan flow: consult the index to find the first block containing the start row, then stream blocks sequentially until the end row is reached. Block prefetching hides HDFS latency: while the RegionServer processes one block, the next block is being fetched in the background.
Data block encoding (like DIFF or FAST_DIFF) compresses cell keys within a block by only storing differences from the previous key. This saves 40 to 70 percent of space for sorted keys with common prefixes. Compression (Snappy, ZSTD, GZ) further compresses each block. Encoding is fast; compression trades CPU for storage.