TRACE · INSPECT · CHALLENGE

Tombstones: repair, grace and resurrection

Follow one deleted value across three replicas and inspect exactly when its marker disappears.

60-second event inspection

Your experiment

Use the default scenario: B reconnects at 4, repair propagates the delete at 5, and compaction at 9 removes markers older than grace 6. Then reconnect B at 11: repair at 5 misses it, A/C purge, and final anti-entropy at 13 resurrects its stale value. Increase grace to 8: the markers survive the same compaction and suppress the stale value.

Play inspects 14 modeled ticks over 60 seconds at 1×. This is teaching time, not database runtime. Pause, scrub or step to the next event. Restart keeps your inputs; changing a parameter recalculates without autoplay.

What the model means

Repair copies timestamped mutations; it is not garbage collection. A local purge in this bounded model requires a tombstone strictly older than grace and inclusion of its older local data. Those local conditions cannot prove an offline replica received the delete. Return is processed before repair, then compaction at a shared tick. Final anti-entropy always attempts to reconcile reachable replicas at tick 13.

State at this event

Exact event and state inspector

Scope and references

Single key, distinct timestamps, three fixed replicas. No hints, read repair, TTL, overlapping ranges, incremental-repair metadata or only_purge_repaired_tombstones setting. The older-data checkbox represents compaction coverage, not a Cassandra command. Ticks are teaching units, not recommended production grace settings.

Apache Cassandra: tombstones · Official AICassindra articles

Deletion is a versioned fact

Three replicas begin with value 100 at timestamp one. Replica B is offline while a deletion at timestamp two reaches A and C. A tombstone is not merely an empty cell: it carries the ordering information needed to reject an older value. Watch the replica table and decision notes as B returns, repair compares versions and compaction evaluates the marker. The final anti-entropy step reconciles online replicas, making an unsafe early purge visibly different from a retained deletion.

Control the ordering

B reconnects at tick chooses when the stale replica becomes available; Keep B offline overrides reconnection. Run first repair enables a reconciliation at First repair tick. Compaction tick schedules one local purge decision. Grace controls marker retention in model ticks: age must be strictly greater than grace, not equal, before it is eligible. Compaction includes older local data controls whether older values in that local selection are covered. It cannot establish that an offline replica has received the deletion. A reconnect and repair at the same tick occur before compaction.

Create and prevent resurrection

Start with repair tick five, compaction tick nine and grace six. Reconnect B at tick eleven: repair misses it, the marker is old enough to purge at nine, and final reconciliation can reintroduce B's stale value. Keep the same order but increase grace to eight. At compaction the deletion is seven ticks old, so it remains and later suppresses the old value. Alternatively reconnect B before the repair and inspect its tombstone before permitting the purge. Compare actual replica versions, not only the final warning.

Inspect the teaching limits

Play reveals fourteen modeled ticks over sixty inspection seconds. Pause freezes the state; Next event, Restart, Speed and the seek control let you inspect and replay the ordering. Changing a policy starts a fresh scenario. This bounded model has one deletion, simplified local compaction and deterministic reconciliation. It does not implement Cassandra repair coverage metadata, consistency levels, clock skew, transient replicas, TTL expiration or every tombstone kind. The final reconciliation always occurs at tick thirteen for online replicas. A successful toy run therefore demonstrates one causal ordering, not a production-safe grace setting.

Views are my own and do not represent any employer, past or present.

Search