Kafka partitioning determines parallelism, ordering, and skew. The partition key is one of the highest-leverage decisions in any Kafka deployment. Get it wrong and you face cluster rebalancing, hot partitions, or broken consumer ordering.
Advertisement
Default: hash by key
Same key → same partition. Preserves per-key ordering. Right for entity-keyed events (user_id, account_id). Skew if keys are non-uniform (one user 10x more events than median).
Round-robin (null key)
Even distribution, no per-key ordering. Right for telemetry where order across keys doesn't matter and even consumer load does.
Advertisement
Custom partitioner
Override for specific cases: locality-aware (route to consumer's region), priority (high-priority events to dedicated partitions), composite keys. Use sparingly; reduces ability to rebalance cleanly.
Key-hash for ordered streams, round-robin for telemetry, custom only when justified. The key choice is hard to change later.