Why architecture matters here
EC is a real cost lever but not free. Reads must fetch cells from multiple DataNodes — more network. Reconstruction requires reading (K + M - 1) cells to rebuild one — expensive under failure. Small files don't fit the stripe and lose EC benefit.
The architecture matters because policy per directory lets you use EC only where it helps (cold data, large files) while keeping replication for hot small files. Capacity plans must include reconstruction burst.
With the pieces in mind, you can plan EC deployment that hits savings targets without regressing SLOs.
The architecture: every piece explained
The top strip is the write path. Client write passes large data. EC policy — RS-6-3, RS-3-2, XOR — decides K data cells and M parity cells. Block group is the (K + M) tuple; each cell is written to a distinct DataNode. DataNode layout spreads cells across racks so a rack failure loses at most one cell.
The middle row is the read + repair. Read path fetches all K data cells concurrently; if some are unavailable, fetches parity cells and reconstructs. Reconstruction triggers on cell loss — reads K live cells and computes the missing one; expensive but durable. Migration tool converts replicated files to EC (and back). Small file impact: files smaller than the stripe size (typically 1 MB × K) waste space in padding.
The lower rows are ops. Policy per directory applies EC selectively — cold data goes to EC, hot small files stay replicated. Metrics track reconstruction operations and throughput. Ops plans capacity for reconstruction bursts and documents fallback to replication for hot data.
End-to-end flow
End-to-end: A cluster stores 500 PB. Archive directories are set to RS-6-3 (50% overhead). Hot analytics tables stay 3x replicated. Storage drops from 500 PB × 3 = 1.5 EB to (archive × 1.5) + (hot × 3), saving petabytes. A DataNode fails; NameNode identifies EC block groups needing reconstruction; scheduler dispatches at throttled rate. Reads on hot data are unaffected; archive reads see a small latency bump during reconstruction. Metrics dashboards confirm reconstruction completes within SLA. Capacity plan included the 20% reconstruction bandwidth headroom so hot workloads weren't affected.