Why architecture matters here
The first reason backup architecture matters is that the failures it guards against are logical, not physical. HDFS keeps three replicas of every block, so a dead disk is a non-event. But a replica is a faithful copy of whatever you wrote — including the mistake. If an application deletes a row or a migration rewrites a column family with garbage, all three replicas dutifully reflect the damage instantly. Only an independent copy taken before the mistake can undo it, and that is what a backup is: not redundancy against hardware, but a time machine against human and software error.
The second reason is that a snapshot alone is not durable in the way a backup must be. An HBase snapshot is cheap because it does not copy data — it captures references to the current HFiles and relies on the fact that compaction will archive rather than delete files a snapshot still needs. That makes snapshots excellent for fast, in-cluster rollback, but it also means a snapshot lives inside the same cluster and the same HDFS it is protecting. Lose the cluster, the HDFS, or the datacenter and you lose the snapshot with it. A backup exports the data to a separate destination precisely so it survives the loss of the source.
The third reason is cost asymmetry. A full copy of a large table is enormous and slow to produce; taking one every hour is infeasible. But the rate of change is usually a small fraction of the total size, and that change is already recorded in the WAL. Incremental backups exploit this: after one expensive base image, each subsequent backup copies only the edits since the last one, so a rich recovery granularity (say, hourly) becomes affordable. The architecture's whole shape — occasional full, frequent incremental — is a direct response to this asymmetry.
The fourth reason is that recovery objectives are business decisions that the backup schedule must encode. RPO (recovery point objective) is how much data you can afford to lose — it is bounded by how often you take incrementals. RTO (recovery time objective) is how long a restore may take — it is bounded by the size of the base image plus the number of incrementals that must be replayed. A long chain of incrementals gives a tight RPO but a slow RTO; periodic new full backups reset the chain and shorten restores. The architecture matters because these knobs are how you turn an abstract 'we have backups' into a concrete, measured guarantee.
Finally, backup architecture matters because an unverified backup is worse than no backup — it is a false belief that induces risky behavior. The only proof that a backup is good is a successful restore, and the only way to know your RTO is to have measured it on a real recovery. A system designed around backup and restore as a single tested loop, with the metadata to make restore deterministic, is what converts stored bytes into an actual recovery capability.
The architecture: every piece explained
The backup client is the entry point: it drives a backup session, either full or incremental, over one or more tables. A session is atomic in the sense that it produces one coherent image with a recorded boundary; if it fails partway, it is rolled back rather than leaving a half-written image that restore might trust. The client coordinates the underlying steps — snapshotting, WAL roll, distributed copy — and writes the resulting metadata on success.
A full backup works by taking a snapshot of each target table (freezing a consistent set of HFile references) and then exporting those HFiles to the backup destination — an HDFS path or object-store bucket, ideally in a different failure domain from the source cluster. The exported image is self-contained: it can be restored with no dependency on any other backup. This is the anchor of a backup chain and the thing every incremental is measured against.
An incremental backup captures the write-ahead-log edits accumulated since the previous backup's WAL roll point. When an incremental session starts, the system rolls the WALs — closing the current log segments so there is a clean boundary — and copies the segments containing edits since the last recorded roll marker to the destination. Because the WAL is a complete, ordered record of every mutation, replaying these edits reproduces exactly the changes that occurred in the interval. An incremental is therefore small and fast but only meaningful in the context of the base and any earlier incrementals.
The backup system table (commonly backup:system) is the ledger that makes the chain coherent. For every session it records the backup ID, type, the tables covered, the start and end WAL roll markers that bound the edits, the destination path, and the completion status. Restore reads this table to determine, for a target recovery point, which full image to restore first and which incrementals to apply and in which order. This metadata is the difference between a set of files and a restorable system; protecting it is as important as protecting the data images themselves.
Restore is the mirror operation. To recover a table to a chosen point, it lays down the relevant full image to reconstruct the base state, then replays the incrementals in recorded order up to the target, applying WAL edits on top of the base. Restore can target the original table name or a new name (useful for validating a backup, or for extracting a subset of history without disturbing production). A merge capability lets operators compact several consecutive incrementals into one, shortening the replay chain and improving RTO without discarding recoverable points. The diagram shows how full and incremental sessions, the system table, and restore fit together.
End-to-end flow
Walk a concrete cycle. On day zero an operator runs a full backup of the orders table to an object-store bucket in a second region. The backup client snapshots orders, exports its HFiles to the bucket, and writes a session row to backup:system recording the backup ID, the table, the WAL roll marker at the moment of the snapshot, and the destination path. This base image is now independently restorable.
Every hour thereafter an incremental backup runs. Each session rolls the WALs to establish a clean boundary, copies the WAL segments containing edits since the previous session's marker to the bucket, and records a new row in the system table with the new start and end markers linking it to its predecessor. After a day there is one full image plus twenty-three hourly deltas, a chain that can reconstruct the table at any hourly boundary since the base.
Now a bad deployment at 14:30 corrupts a column family. The team decides to restore orders to its 14:00 state. An operator issues a restore against backup ID chain up to the 14:00 incremental, targeting a staging table name to validate before cutting over. Restore reads backup:system, identifies the day-zero full image as the base, and computes the ordered list of incrementals from the base up to and including the 14:00 delta.
Restore then lays down the full image to reconstruct the base state of orders, and replays each incremental's WAL edits in order on top of it, stopping at the 14:00 boundary. The corrupting edits from 14:30 are never in the replayed set, so the recovered staging table reflects clean 14:00 state. The team verifies row counts and a few sample keys against expectations, confirms the data is correct, and then swaps the staging table into production — or, if they had restored directly over orders, they would simply resume traffic.
Two operational subtleties shape this flow. First, the RPO here is one hour — anything written between 14:00 and the corruption is lost on restore, because that is the granularity of the incremental schedule; tightening RPO means more frequent incrementals. Second, the RTO is dominated by the size of the base image plus twenty-three deltas to replay; if that chain had grown to hundreds of incrementals over weeks, restore would be slow, which is why periodic new full backups (or merges) reset the chain length. The flow is deterministic precisely because the system table records exactly which images compose the target point.