Kafka partitioning strategy is not a detail, it is your thread model. Every message landing on the same partition stays ordered within that partition. Every message on different partitions can race. Get the partition key right and your consumers move in lockstep; get it wrong and you trade correctness for throughput or rebuild the entire pipeline later. This article covers the three main strategies: hash by key (the default, standard choice), round-robin (null key), and custom partitioners for specific workloads.
Default: hash by key
Same key → same partition. Preserves per-key ordering. Right for entity-keyed events (user_id, account_id). Skew if keys are non-uniform (one user 10x more events than median).
This is the default partitioner and works well when your messages have a natural key that groups related events together. The hash ensures that all events with the same key land on the same partition, which means a single consumer will process them in order. If you are tracking user actions, orders, or account state, hash by key is almost always the right choice because the sequence of events for a single entity matters.
The catch is skew. If one key produces far more messages than others, that partition becomes a bottleneck while other partitions sit idle. You can mitigate this by increasing partition count, but there is no free lunch: more partitions mean more rebalancing overhead, more state to track in consumers, and more complexity in coordinating work across them.
Round-robin (null key)
Even distribution, no per-key ordering. Right for telemetry where order across keys doesn't matter and even consumer load does.
When you send a message with a null key, Kafka cycles through all available partitions in round-robin order. This guarantees that load spreads evenly across all partitions and all consumers, which is ideal for high-volume telemetry or metrics where you care about throughput over ordering.
The trade-off is clear: you lose all ordering guarantees between messages. Two events with different keys might be processed in any order, even if they arrived milliseconds apart. This is fine for metrics where you only care about the aggregate count. But if you need to track state changes for an entity, round-robin will create bugs because a later event might be processed before an earlier one, leaving your state inconsistent.
Custom partitioner
Override for specific cases: locality-aware (route to consumer's region), priority (high-priority events to dedicated partitions), composite keys. Use sparingly; reduces ability to rebalance cleanly.
When the default strategies do not fit your workload, you can implement a custom partitioner by extending org.apache.kafka.clients.producer.Partitioner and passing it to the producer. This is powerful but dangerous. A common use case is geo-aware partitioning: route all messages for EU customers to partitions that are consumed by EU-based services, and messages for US customers to partitions consumed by US-based services. This reduces cross-region network traffic and keeps compliance boundaries tight.
Another pattern is priority-based partitioning: reserve dedicated partitions for high-priority or critical events so they do not get stuck behind bulk data. Or partition by a composite key like region + timestamp_bucket to co-locate events that belong to the same operational window.
The caveat: custom partitioners couple your producer to your partition topology. If you add partitions, rename them, or rebalance your partition assignment, your custom logic has to adapt or messages will be misrouted. Keep the logic simple and document the assumptions it encodes.