Kafka Exactly-Once Semantics — Idempotence + Transactions

Why it matters

Exactly-once was long considered impossible in distributed systems. Kafka's implementation is a real win — worth understanding for critical pipelines.

The architecture

Idempotent producer: producer ID + sequence per partition. Broker dedupes retries.

Transactions: producer.beginTransaction() ... commitTransaction() atomic across partitions.

How it works end to end

Transactional isolation: consumers configured 'read committed' see only committed data.

Kafka Streams: EOS via read-process-write pattern in transaction.

Cost: ~10-20% throughput overhead, more state on brokers, more complex config.