Why it matters

Exact cardinality at scale impractical. HLL is standard approximate solution. Understanding shapes analytics.

Advertisement

The architecture

Hash each item.

Leading zeros of hash indicates rarity.

Max leading zeros correlates with distinct count.

HLL algorithmHash itemuniform bitsCount leading zerosper bucketEstimate from maxharmonic meanBucket count m controls memory + accuracy; 2^14 gives ~0.8% error in ~16KB
HLL mechanics.
Advertisement

How it works end to end

Buckets: partition hash space into m buckets. Each stores max leading zeros seen.

Estimation: harmonic mean of (2^bucket) values, scaled.

Merge: HLL sketches merge cleanly by taking max per bucket. Distributed friendly.