Why it matters

Wrong partition count is a common Kafka mistake. Getting it right shapes throughput and cost.

Advertisement

The architecture

Partition selection: producer hashes record key modulo partition count.

Ordering: within partition, order preserved. Across partitions, no ordering.

Kafka partitioningProducer keyedhash → partitionPartition logordered subsetConsumer per partitionparallel consumptionIncreasing partition count is easy; decreasing requires topic recreation
Partition assignment.
Advertisement

How it works end to end

Partition count: rough starting point is target throughput / per-partition throughput. Broker limit ~4000 partitions.

Rebalancing: adding brokers doesn't move partitions automatically. Manual reassignment needed.

Keyed vs keyless: keyed for ordering per key; keyless for even distribution.