Why it matters

Any production data platform needs a way to answer three operational questions: what did this file look like yesterday, can we roll back a bad ETL job that overwrote good data, and how do we take a consistent backup of a live directory. Traditional filesystems require either offline copies or expensive snapshot mechanisms like LVM. HDFS handles all three cheaply through the snapshot feature.

Regulatory workflows also benefit. Compliance rules often require immutable retention of specific data for defined periods. Snapshots plus a policy that no operator can delete them until an expiry date is a simple way to satisfy those rules without moving data to an external archive.

Advertisement

The architecture

A snapshot is a named read-only handle attached to a directory. Once enabled on a directory, snapshots can be created with hdfs dfs -createSnapshot at any time. The snapshot captures the exact block map of every file in the directory tree at the moment of creation. Existing blocks are pinned so they cannot be garbage collected even if the live file is deleted or overwritten.

When new writes modify a file inside a snapshotted directory, HDFS uses copy-on-write semantics. The live version of the file gets new blocks. The snapshot version still points to the original blocks. This means no data is copied at snapshot creation time; the storage cost is zero at that moment and grows only if the live data diverges from the snapshot.

Live directory /prod/user_events (writeable)Snapshot: 2026-07-01read-only pointerSnapshot: 2026-07-02read-only pointerSnapshot: 2026-07-05read-only pointercopy-on-writecopy-on-writecopy-on-writeUnderlying blocks unchanged; snapshots pin old versions until deletedBlocks kept alive by snapshots (not physically duplicated)New writes create new blocks + tombstones for snapshot lookupsDeleted snapshots free their pinned blocks
Copy-on-write snapshots: instant creation, storage grows only when live data changes.
Advertisement

How it works end to end

Creating a snapshot requires the directory to be marked as snapshottable using hdfs dfsadmin -allowSnapshot. This flag tells the NameNode to track modifications so that snapshots can be created. Without this flag, the NameNode refuses snapshot operations for the directory.

Reading from a snapshot is the same as reading a regular file. The snapshot appears as a hidden subdirectory named .snapshot inside the snapshottable directory, and every snapshot appears as a subdirectory of .snapshot. Files inside are addressed exactly like regular HDFS paths, and the client reads them by contacting the NameNode for the block map (which returns pinned blocks) and streaming from DataNodes.

Writes to the live directory that touch files existing in a snapshot cause the NameNode to allocate new blocks and update the live file to point to them. The snapshot continues to reference the old blocks. Blocks referenced only by deleted snapshots are eventually garbage collected during background scanning. This is why storage cost grows over time as snapshots diverge from the live tree.