All 54 articles, sorted alphabetically
Change Data Capture in 2026
Debezium Decodable native CDC.
Read article →Dead Letter Queues
Pattern retention and re-processing.
Read article →Debezium CDC to Kafka
Capturing Postgres MySQL and MongoDB changes as event streams.
Read article →Event Sourcing vs Event Streaming
When the same word means different architectures.
Read article →Exactly-Once Semantics Across Systems
What actually gets guaranteed and where the gaps are.
Read article →Exactly-Once Semantics in Streaming
What 'exactly-once' actually means in Kafka, Flink, and Spark Structured Streaming.
Read article →Flink Checkpoint Design
Frequency, alignment, and what gets lost.
Read article →Flink State Backends Compared
RocksDB vs in-memory and when each one fails.
Read article →Kafka KRaft vs ZooKeeper
Migrating to the new Raft-based metadata quorum.
Read article →Kafka Partitioning Strategies
Pick the key wrong, regret it later.
Read article →Kafka Tiered Storage
S3-backed cold tier in production.
Read article →Kafka vs Pulsar vs Redpanda
Architecture differences between the three top streaming brokers + when each one wins.
Read article →Materialize for Streaming SQL
Incremental view maintenance over Kafka with Postgres-style queries.
Read article →Redpanda Tiered Storage
Offloading cold data to S3 while keeping hot writes local.
Read article →Schema Registry Patterns
Avro Protobuf JSON Schema in 2026.
Read article →Apache Flink Overview
What Flink is, its stateful streaming model, and how it differs from Spark Streaming.
Read article →Flink State
How Flink manages state per-key and per-operator, backed by RocksDB for durability.
Read article →Flink Windowing
How Flink windowing groups events into time or count-based windows for aggregation.
Read article →Kafka Consumers and Consumer Groups
How consumer groups coordinate to distribute partition consumption, and how offsets track progress.
Read article →Kafka Exactly-Once Semantics
How Kafka achieves exactly-once processing via idempotent producers and transactional writes.
Read article →Apache Kafka Overview
What Kafka is, its append-only log design, and how it powers streaming and event-driven architectures.
Read article →Kafka Partitions
How partitions distribute topic data across brokers and enable parallel consumption.
Read article →AWS Kinesis
How Kinesis Data Streams provide Kafka-like functionality as a managed AWS service.
Read article →Apache Pulsar
What Pulsar is, how its architecture (compute-storage separation) differs from Kafka, and when to use it.
Read article →Spark Streaming
How Spark Streaming (and Structured Streaming) process streams as micro-batches on the Spark engine.
Read article →Stream-Table Duality
Same data two views in Kafka Streams and Flink SQL.
Read article →Streaming watermark architecture
Deep-dive on streaming watermarks: event time, watermark, trigger, late data, allowed lateness, side output, retract.
Read article →Backpressure architecture
Deep-dive on backpressure in stream processing: why unbounded buffering causes stateful crashes, credit-based flow control between operator subtasks, …
Read article →Apache Beam architecture
Deep-dive on Apache Beam: PCollection, transforms, runners, windowing, triggers, watermark, state + timers, side inputs.
Read article →CDC architecture
Deep-dive on Change Data Capture: log-based capture (WAL/binlog), snapshot, schema evolution, delivery semantics, ordering, sinks.
Read article →Kafka consumer group rebalancing architecture
Deep-dive on Kafka consumer group rebalancing: the group coordinator and heartbeat liveness, revoke-and-assign phases, offset commits at the rebalance…
Read article →Exactly-once stream processing architecture
Deep-dive on end-to-end exactly-once semantics: replayable Kafka sources, Flink barrier-aligned snapshots, two-phase-commit sinks bound to checkpoint …
Read article →Apache Flink Architecture in Depth
A 2500-word walkthrough of Apache Flink architecture: JobManager, TaskManagers, dataflow graph, state backends, sources/sinks, windows, watermarks, ch…
Read article →Flink Async I/O architecture
Deep-dive on Flink Async I/O: overlapping many in-flight external lookups through a bounded-capacity buffer, the AsyncFunction and ResultFuture contra…
Read article →Flink savepoints architecture
Deep-dive on Apache Flink savepoints: user-triggered canonical-format state snapshots keyed by stable operator UIDs, exactly-once continuity across up…
Read article →Flink state architecture
Deep-dive on Flink state: keyed and operator state, backends, checkpoints, savepoints, barrier alignment, restore, rescale, TTL.
Read article →Apache Hudi architecture
Deep-dive on Apache Hudi: COW vs MOR, timeline, record index, base + log, compaction, clustering, cleaning, incremental read.
Read article →Iceberg architecture
Deep-dive on Apache Iceberg: data files, manifests, metadata, snapshots, partition specs, hidden partitioning, time travel, compaction.
Read article →Apache Kafka Streaming Architecture in Depth
A 2500-word walkthrough of Apache Kafka: producers, brokers with KRaft, topics + partitions, replication, schema registry, connect, stream processing,…
Read article →Kafka ISR replication architecture - leaders, followers, and the in-sync replica set
Deep-dive on Kafka's replication protocol: leader and follower fetch, the in-sync replica set, high watermark and commit sema…
Read article →Kafka ordering architecture
Deep-dive on Kafka ordering: partitioning, keys, idempotent producer, transactions, isolation, cooperative rebalance, and repartitioning.
Read article →Kafka client quotas architecture
Deep-dive on Kafka quotas: user/client-id entity precedence, produce/fetch/request/mutation quota types, the sliding-window rate sampler, the throttle…
Read article →Kafka Streams
Deep-dive on Kafka Streams: processor topologies over KStream/KTable, local RocksDB state stores backed by changelog topics, partitioning and stream t…
Read article →Kafka log compaction architecture
Deep-dive on Kafka log compaction: clean vs dirty segments, the cleaner thread and its offset map, tombstones and delete.retention.ms, dirty-ratio and…
Read article →Kafka MirrorMaker 2 architecture
Deep-dive on Kafka MirrorMaker 2 (MM2): cross-cluster replication built on Kafka Connect that copies records, topic configs, ACLs, and consumer offset…
Read article →Apache Paimon architecture
Deep-dive on Apache Paimon: LSM inside, streaming ingest with Flink, batch reads, merge engines, change-log stream, snapshots.
Read article →Apache Pulsar Architecture in Depth
A 2500-word walkthrough of Pulsar: brokers, BookKeeper bookies, ZooKeeper, topics, subscriptions, geo-replication, tiered storage, Functions.
Read article →Redis Streams Architecture in Depth
A 2500-word walkthrough of Redis Streams: XADD, entry ID, consumer groups, XREADGROUP, XACK, XPENDING, XCLAIM, MAXLEN, persistence + cluster.
Read article →Schema registry architecture
Deep-dive on the streaming schema registry: subjects and versions, the schema-id wire format, backward/forward/full compatibility modes, reader vs wri…
Read article →Stateful stream joins -- joining unbounded streams
Deep-dive on stateful stream joins: the unbounded-join problem, keeping both sides as state (buffered by key), window/interval joins bounding the stat…
Read article →Kafka tiered storage architecture
Deep-dive on Kafka tiered storage: the RemoteLogManager and remote index, segment rolling and upload, local vs remote retention, tail vs backfill read…
Read article →Stream windowing -- bounding the unbounded
Deep-dive on stream windowing: the unbounded-stream aggregation problem, tumbling/sliding/session windows, event vs processing time, watermarks for ev…
Read article →