Change Data Capture in 2026, in depth: choosing a topology, surviving primary failover with Postgres 17 failover slots, schema evolution and landing changes in the lakehouse

Four topologies

Every CDC deployment reads the same source log. They differ in where the reader runs and where the changes go first. Figure 1 shows the four common shapes.

The question that decides most cases is how many consumers you have. A changelog in Kafka is a durable, replayable fan-out point: a new consumer can start from the earliest retained offset without touching the database. Without that, every new consumer either opens its own replication slot, adding load and another retained-WAL risk, or reads from the lakehouse table and accepts its latency. Choose Kafka when you expect three or more consumers or need to replay. Choose a direct pipeline when the lakehouse is the product and other consumers can read the table.

The position is the pipeline

Every CDC reader holds a position in the source log: a confirmed LSN on a Postgres slot, a GTID set in MySQL, an SCN in Oracle. Two invariants follow. The source keeps log data until the reader confirms it, so a stalled reader fills the source's disk. And the reader can only resume from a position the source still has, so anything that discards or resets that position forces a fresh snapshot.

In Postgres, the position lives in a replication slot on the primary. Before version 17, that slot existed only on the primary. After a failover to a physical standby, the new primary had no slot. The connector either failed, or it was configured to create a new slot at the current WAL position and silently skipped every change committed between the last confirmed LSN and the promotion. Teams handled this with a re-snapshot after every failover, with extensions, or with managed-service features. Postgres 17 makes failover-safe slots a core feature.

Postgres 17 failover slots

A failover slot is a logical slot on the primary that a standby keeps a synchronised copy of. Three pieces of configuration make it work:

-- On the primary: create the CDC slot with failover = true (5th argument, Postgres 17+)
SELECT * FROM pg_create_logical_replication_slot('cdc_orders', 'pgoutput', false, false, true);

-- On the primary: never let a logical consumer confirm WAL the standby has not received
ALTER SYSTEM SET synchronized_standby_slots = 'standby1_phys';   -- the standby's physical slot
SELECT pg_reload_conf();

# On the standby (postgresql.conf)
wal_level = logical
hot_standby_feedback = on
sync_replication_slots = on
primary_slot_name = 'standby1_phys'
primary_conninfo = 'host=pg-primary user=replicator dbname=orders'   # dbname is required

-- Check on both nodes
SELECT slot_name, failover, synced, confirmed_flush_lsn FROM pg_replication_slots;

The synchronized_standby_slots setting is the subtle part. Without it, the CDC consumer could confirm an LSN that the standby has not yet received; after promotion, the new primary's slot would point to a position the consumer had already passed. With it, the logical slot waits for the physical standby, so a promoted standby is never behind the consumer. The cost is that CDC latency now includes standby replication lag, and a dead standby stalls CDC until you remove it from the list.

A synced slot on the standby cannot be consumed while it is a standby. It becomes usable only after promotion, and the synced column shows whether synchronisation is working. Connector support also varies. Debezium tracked automatic creation of failover slots as DBZ-8412; unless your connector version documents an option for it, create the slot yourself as above and point the connector at it by name. Test promotion in staging. Confirm the connector reconnects to the new primary through a DNS name or proxy, finds the slot, and resumes without a gap or a snapshot.

MySQL has a simpler story because the position is a GTID set that every replica shares. A connector that stores GTIDs can resume against any replica that has the same transactions, provided binlog retention on the new primary covers the gap. Check binlog_expire_logs_seconds on every node a connector might fail over to, not only the current primary.

Schema evolution end to end

Schema changes break more CDC pipelines than outages do. A single ALTER TABLE passes through three schemas: the source table, the event schema in the registry, and the sink table. Each has its own compatibility rules.

• Additive changes (a new nullable column) are safe everywhere if the registry uses backward compatibility and the sink allows adding columns. Flink CDC pipelines can propagate them to Paimon, Iceberg and several OLAP sinks automatically; Debezium emits the new field, and the sink connector or job must add the column.
• Renames look like a drop plus an add to almost every tool. Downstream, the old column stops receiving values and a new one appears. Plan renames as add, backfill, switch readers, then drop.
• Type changes that widen (int to bigint) are usually fine in Iceberg, which supports promoting int to long and float to double. Narrowing or changing type families requires a new column.
• Dropped columns break consumers that require them. Use the registry's compatibility check as a gate in the migration pipeline, not just at runtime.

The practical rule is to make schema migrations a coordinated change: the migration tool checks registry compatibility before running DDL, and sink-table evolution is either automatic and tested or done before the source change ships.

Landing changes in a lakehouse

Lakehouse tables store immutable files, so an update is never in place. It is a delete of the old row plus an insert of the new one. Table formats differ in when they pay for that.

Iceberg format v2 lets a writer record deletes as equality deletes, which say "remove any row with this key". They are cheap to write because the writer does not need to find the old row. They are expensive to read: every query must match data files against accumulated delete files until compaction rewrites them. A CDC stream that updates hot keys every few seconds can build up delete files quickly, so regular compaction is required. Writing position deletes instead requires the writer to know where the old row lives, which costs more at write time but less at read time. Format v3 adds deletion vectors, which make row-level deletes cheaper to read; check whether your engines support v3 before relying on them. Hive and Iceberg tables covers the commit path and row-level DML in more detail.

Paimon was designed for this workload. A primary-key table is an LSM tree per bucket, so updates merge during compaction and readers see the latest value per key through the merge engine; the Paimon deep dive explains buckets, merge engines and changelog producers. The Flink CDC Paimon sink is documented as at-least-once and requires primary-keyed source tables, which is safe because a replayed upsert on the same key gives the same result.

Either way, commit frequency is the main tuning knob. Each Flink checkpoint becomes a table commit. One-minute checkpoints give fresh data but many small files and frequent compaction; five to ten minutes is a common starting point for analytics.

Worked example: orders into Paimon, without losing a failover

An e-commerce team runs MySQL for orders and Postgres 17 for payments. Analysts want both in the lakehouse within ten minutes. Support wants a search index of orders within seconds.

There are two consumers of orders and one of payments, with no replay requirement beyond what the table provides. The team chooses a Flink CDC pipeline for the lakehouse and an embedded Debezium engine inside the search service. That means two readers on the orders binlog, which MySQL handles easily. The orders pipeline is a YAML file submitted with Flink CDC's command-line tool:

source:
  type: mysql
  hostname: orders-db.internal
  port: 3306
  username: cdc_reader
  password: ${CDC_PASSWORD}
  tables: shop.orders, shop.order_items
  server-id: 5401-5404          # one id per parallel reader; must not clash with real replicas

sink:
  type: paimon
  catalog.properties.metastore: filesystem
  catalog.properties.warehouse: s3://lake/warehouse

pipeline:
  name: orders-to-paimon
  parallelism: 4

The first run takes a parallel snapshot of both tables, then switches to the binlog. Checkpoints every five minutes give the analysts their ten-minute target. For payments, the team creates a failover slot as shown earlier, points the Postgres connector at it, and sets synchronized_standby_slots. During a planned failover drill, they promote the standby, flip the DNS name, and confirm the pipeline resumed from the synced slot. A count of payments per minute in the lake matches the source for the failover window, with no gap and no duplicate rows, because Paimon's primary key absorbs the replayed upserts.

The search service's embedded engine stores its offsets in the service's own database, in the same transaction as the index update metadata. A crash replays a few events, and because index writes are keyed upserts carrying the source position, replays are harmless:

def apply(event):
    key, pos = event.key["id"], event.source_position     # binlog file+pos or GTID-derived ordinal
    if pos <= index.last_applied(key):                     # replayed or reordered event: skip
        return
    if event.op == "d":
        index.delete(key, pos)
    else:
        index.upsert(key, render(event.after), pos)

Failure modes

• Retained WAL fills the disk. A stopped connector, or an idle table on a busy database, keeps a slot's position from advancing. Alert on retained WAL per slot and use heartbeats. On Postgres 13 and later, max_slot_wal_keep_size caps retention, at the cost of invalidating the slot when the cap is hit.
• Silent gap after failover. The connector recreates a slot at the current position on the new primary. Use failover slots, and configure the connector to fail rather than create a missing slot.
• CDC stalls because a standby is down. This is the price of synchronized_standby_slots. Monitor it and have a runbook for removing a dead standby from the list.
• Delete-file build-up. Iceberg query latency climbs steadily because equality deletes accumulate. Schedule compaction and alert on delete-file counts.
• Snapshot and stream overlap. Rows changed during the initial snapshot appear twice. Keyed upsert sinks absorb this; append-only sinks do not.
• Out-of-order applies. Parallel consumers apply two updates to one key in the wrong order. Partition by primary key and guard writes with the source position, as in the example.
• Large transactions. A bulk update of fifty million rows becomes fifty million events at once. Throttle bulk jobs, or exclude the table and reload it.

Trade-offs

Log-based CDC adds almost no load to the source and captures every change, including deletes, but it couples you to the database's replication internals and to its schema. The transactional outbox pattern, covered in the outbox article, publishes events the application designs on purpose. It decouples consumers from table layout, at the cost of application code. Many teams use both: outbox events for domain integration between services, and table CDC for analytics and caches.

Kafka in the middle costs operations but buys replay and fan-out. A direct pipeline costs flexibility but removes a system. Failover slots cost some latency, but they remove a whole class of silent data loss, which makes them worth it for anything feeding finance or compliance.

What to do next

1. Count your consumers for each source and pick a topology using the table above. Write the reason down.
2. For each Postgres source on version 17 or later, convert CDC slots to failover slots, set synchronized_standby_slots, and run a promotion drill that checks for gaps.
3. For MySQL, confirm GTIDs are on and binlog retention on every failover candidate covers your worst connector outage.
4. Add alerts for retained WAL or binlog lag per connector, connector task state and end-to-end freshness at the sink.
5. Put a registry compatibility check into your migration pipeline, and practise a rename using add, backfill, switch and drop.
6. Choose sink keys and make every apply idempotent by key and source position.
7. Schedule lakehouse compaction and set a checkpoint interval based on your freshness target, not the lowest number possible.