Why it matters
Wrong queue choice locks in architecture for years. Kafka for a workflow that needs RabbitMQ semantics is painful; SQS for a use case requiring replay is worse. Learning the choices is core system design.
The architecture
Kafka: append-only log partitioned across brokers. Consumers track their own offset. Consumers can replay from any point. Optimized for throughput (millions of msg/s), retention (days to weeks).
RabbitMQ: broker manages message state, routes messages to consumers, acks remove from broker. Rich routing (topic exchanges, headers). Lower throughput but richer semantics.
How it works end to end
SQS: AWS managed queue. Standard (at-least-once, unordered) or FIFO (exactly-once, ordered but limited throughput). No replay, but reliable and cheap.
Delivery semantics: at-most-once (may lose messages), at-least-once (may duplicate), exactly-once (hard, often achieved via idempotent consumers).
Ordering: Kafka within partition, RabbitMQ within queue, SQS FIFO within message group.