Why it matters

Streaming cardinality is core analytics op. HLL is standard.

Advertisement

The architecture

Sketch per window / partition.

Merge across partitions.

Query estimates cardinality.

HLL streaming stackPer-partition HLLsketch streamMergeunion sketchesQuerydistinct estimateMergeable + associative sketches parallelize naturally across shards
HLL streaming.
Advertisement

How it works end to end

Sketch per shard or window.

Union: bitwise max per bucket.

Query: harmonic mean estimator.

Redis: PFADD, PFCOUNT, PFMERGE.