Why architecture matters here
The architecture matters because cross-region resilience and data locality are not optional at scale, and Kafka's intra-cluster replication cannot deliver them. Kafka's own replication keeps partition replicas in sync within a cluster and depends on the tight, reliable network of a single data center; stretch it across regions and the higher, jittery latency wrecks throughput and stability. MM2 solves a genuinely different problem — copying between two independent clusters, each with its own controllers and its own offset space — which is the only sane way to get a Kafka footprint in two places.
It matters because replicating records is the easy half; replicating position is the hard half, and it is what makes failover real. A disaster-recovery plan that copies messages but cannot tell a consumer where to resume on the replica forces every application to reprocess from the beginning or guess, which for a high-volume stream is catastrophic. MM2's offset translation — mapping a consumer's committed position in the source cluster to the equivalent position in the target — is the feature that turns 'we have a copy of the data' into 'we can actually run our applications over there.'
It matters because the remote-topic naming scheme is not cosmetic; it is what makes multi-cluster topologies safe. By default MM2 replicates a source topic into a target topic prefixed with the source cluster's alias — orders on cluster A becomes A.orders on cluster B. That prefix encodes provenance directly in the name, so when you replicate in both directions you can tell which cluster originated each record and refuse to copy it back, which is exactly how MM2 prevents the infinite replication loops that would otherwise form in an active-active setup.
It matters because the recovery-point objective is a business decision that the architecture makes visible rather than hides. Because replication is asynchronous, there is always a window of events that exist on the source but have not yet reached the target, and the size of that window — measured directly by the heartbeat lag — is exactly how much data a hard failover can lose. Making that number observable turns an abstract disaster-recovery promise into a quantity you can watch, alert on, and hold to a budget: if the plan says 'lose at most thirty seconds,' you provision the mirror so lag stays under thirty seconds, and you know the instant it does not. A replication design that cannot surface its own recovery point is one you cannot operate against a service-level target.
Finally, it matters because MM2 inherits both the strengths and the operational weight of Kafka Connect. Running on Connect gives MM2 a distributed, fault-tolerant, horizontally scalable execution substrate for free — connectors are just tasks spread across a worker cluster, and Connect handles rebalancing and restart. But it also means MM2 is a Connect cluster you must size, monitor, and tune: too few workers and replication lag climbs; misconfigured, and offsets drift. Understanding that MM2 is Kafka Connect, not a magic sidecar, is the key to operating it, because everything you know about scaling and debugging Connect applies directly.
The architecture: every piece explained
The two clusters anchor the picture: a source (the primary, in region A) and a target (the replica, in region B), each a fully independent Kafka cluster with its own brokers, controllers, and — importantly — its own partition offset space. Nothing about the two offset spaces lines up, which is why simple offset copying is impossible and translation is required.
The MirrorSourceConnector is the workhorse. It runs as a Connect connector whose tasks consume records from the source cluster's topics and produce them into the target, where they land in remote topics named with the source alias as a prefix — A.orders for the source's orders. It also replicates topic configurations and can replicate ACLs, so the target's copy is a faithful mirror and not just a bare stream of bytes. The prefix is the provenance marker that keeps bidirectional replication from looping.
The MirrorCheckpointConnector handles the hard problem of offsets. Because the source and target number their records differently, a consumer's committed offset on the source is meaningless on the target. As MM2 copies records, it records the correspondence between source and target offsets in an internal offset-sync topic; the checkpoint connector periodically reads consumer-group commits on the source, uses those syncs to translate each to the equivalent target offset, and writes checkpoints into the target so a failed-over consumer group can look up where to resume.
The MirrorHeartbeatConnector provides observability. It emits heartbeat records into a heartbeat topic on a schedule; by watching how long each heartbeat takes to appear on the target, you get a direct measurement of end-to-end replication lag and a liveness signal that the pipeline is flowing. Heartbeats are cheap and continuous, which makes them the canonical way to alert on a stalled mirror.
Those pieces compose into topologies. In active-passive disaster recovery, region A serves all traffic and MM2 keeps region B warm; on a disaster you fail consumers over to B using the translated offsets. In active-active, both regions serve traffic and each replicates to the other, with the alias prefix ensuring a record originated on A and copied to B is never copied back to A. The ops strip states the invariant that colors all of it: replication is asynchronous, so the recovery point is always greater than zero — watch the lag, size the Connect workers, avoid cycles, and translate offsets on failover.
End-to-end flow
Trace a record through active-passive DR. An application produces an orders event to the source cluster in region A, where it is committed to a partition and assigned an offset in A's offset space. A consumer application reads it and commits its own progress against A. So far this is ordinary single-cluster Kafka.
MM2's MirrorSourceConnector, running on its Connect cluster, is consuming that same orders topic. It reads the new record and produces it into region B's cluster as A.orders, where B assigns it a fresh offset in B's own space. As it does so, MM2 writes an entry into the offset-sync topic recording that source offset X for this partition corresponds to target offset Y. This mapping is the Rosetta Stone that later makes failover possible.
Meanwhile the MirrorCheckpointConnector periodically reads the source consumer group's committed offsets — where the region-A consumer has gotten to — and, using the offset syncs, translates each committed source offset into the equivalent target offset. It writes these translated positions as checkpoints into region B. And the heartbeat connector keeps dropping heartbeats that let operators watch, second by second, how far B trails A.
Now region A suffers a disaster. Operators trigger failover: the consumer applications are pointed at region B. Because MM2 wrote translated checkpoints, each consumer group can look up its last known good position in B's coordinates and resume from there rather than reprocessing the entire topic. Resumption is approximate, not exact — the translation is periodic and replication is asynchronous — so a few of the most recent events may be reprocessed or, if they never crossed before the disaster, lost. That gap is the recovery-point objective, and it is intrinsic to asynchronous replication.
The active-active case adds the loop-prevention flow. Suppose both A and B are live and each mirrors to the other. A record born on A is copied to B as A.orders. B's MirrorSourceConnector, replicating back to A, sees A.orders, recognizes from the alias prefix that this record originated on A, and refuses to copy it back — otherwise the record would ping-pong forever, each hop adding another prefix. The naming convention, not a separate bookkeeping system, is what breaks the cycle, which is why the prefix scheme is load-bearing rather than decorative.