Why it matters
Decommissioning is a routine operation on any long-lived cluster. Hardware refreshes, capacity right-sizing, and moving workloads across environments all require removing nodes cleanly. Doing it wrong loses data; doing it right is invisible.
Decommission is also load-intensive: re-replicating a full DataNode's worth of data (typically tens of terabytes) generates massive network and disk traffic. Scheduling and pacing matter for cluster stability during the operation.
The architecture
The decommission mechanism uses an excludes file: a text file on the NameNode host listing hostnames to be decommissioned. Adding a hostname and then running hdfs dfsadmin -refreshNodes tells the NameNode to start decommissioning. The NameNode marks the target node's state as DECOMMISSIONING and stops writing new blocks to it. Existing blocks on the node need to be re-replicated to other nodes before the state can transition to DECOMMISSIONED.
Re-replication schedules one block copy at a time from the decommissioning node to a healthy peer, respecting rack placement policy. The rate is capped by replication bandwidth settings, similar to the balancer.
How it works end to end
Once you add a hostname to excludes and call refreshNodes, the NameNode's decommission monitor picks up the change. It scans the target node's blocks, determines which are under-replicated after excluding the target, and schedules replication of each. The target node continues serving reads while decommissioning; only writes to it are stopped.
Progress is visible in the NameNode web UI. The DataNode's status shows DECOMMISSIONING with a count of blocks remaining to replicate. When the count reaches zero, status transitions to DECOMMISSIONED and the node can be safely powered off.
YARN NodeManager decommissioning uses a similar excludes file. Running containers on the decommissioning node are drained: no new containers are scheduled, running containers finish naturally or are killed after a grace period. HBase uses graceful stop scripts that migrate regions before shutting down RegionServers.