Why architecture matters here
Ozone matters because HDFS hits the NameNode wall around hundreds of millions of files. Ozone splits metadata, so it scales further. But the new architecture has new failure modes: Ratis quorums, container health, S3 gateway compatibility.
The architecture matters because Ozone becomes an object store for HDFS-scale ops teams — S3-compatible with predictable behavior on-prem.
The architecture: every piece explained
The top strip is the request path. Client uses S3 API, ofs://, or native RPC. Ozone Manager holds volumes/buckets/keys namespace. SCM assigns keys to containers + blocks. Storage container is the unit of replication (~5 GB) instead of individual blocks.
The middle row is durability + compatibility. Ratis (Raft) replicates OM + SCM state. DataNodes run container replicas. Replication is 3x or erasure-coded. S3 gateway offers standard bucket + object API.
The lower rows are structure + ops. Volumes + buckets + keys mirror S3 hierarchy. Metrics per container health and IO. Ops covers upgrade, capacity, and monitoring.
End-to-end flow
End-to-end: an app writes an object via S3 gateway. OM allocates a key in the bucket. SCM assigns to a container replicated 3x on DataNodes. Client streams data; DataNodes ack. Ratis records OM + SCM changes durably. Later a container fails; SCM triggers re-replication. Cluster serves billions of keys — beyond HDFS NN limits.