Why it matters

HLL is standard cardinality estimator. Foundation for analytics scale.

Advertisement

The architecture

Hash items to bit strings.

Bucket by first bits.

Track max leading zero in each bucket.

Harmonic mean estimator.

HyperLogLogHash + bucketfirst bitsLeading zerosper bucket maxEstimatorharmonic mean1.5KB estimates billions with 2% error; mergeable across shards
HLL.
Advertisement

How it works end to end

m = 2^b buckets.

Standard error ~= 1.04/sqrt(m).

m = 16384: ~0.8% error, 16KB.

Mergeable: union sketches.