Why architecture matters here
Multi-tenancy on YARN fails on placement, not on compute. Two workloads want the same GPU node. A gang-scheduled Spark app waits behind a long MapReduce job. A production queue is starved by a rogue dev job. Every incident traces back to a placement decision.
The architecture matters because the pieces interact. Labels alone are not enough if queues do not enforce them. Preemption alone is not enough if gang scheduling is not honored. Reservations alone create backpressure if applications ignore them.
With the full pipeline in mind, you can design queue hierarchies, label strategies, and preemption policies that hold under load.
The architecture: every piece explained
The top strip is the request path. App Master req declares resource asks — cores, memory, node preferences. Placement policy uses labels and tags to narrow candidate nodes. Constraints add affinity (place two containers on the same node) or anti-affinity (spread across racks). Gang scheduling holds until N containers can start together, avoiding partial allocation that stalls the app.
The middle row is the substrate. Node labels tag nodes with capabilities — GPU, SSD, prod, dev. Queues are hierarchical with min/max capacity per branch; each queue can allow specific labels. Preemption reclaims containers from over-quota queues to satisfy under-quota SLA; policies decide which containers are safe to kill. Reservation books capacity for future use so a large job can start on schedule.
The lower rows are the control plane. Resource Manager makes the scheduling decisions using CapacityScheduler or FairScheduler. Node Managers receive assignments and run container lifecycles. Metrics track fairness (Jain's index), queue utilization, preemption count, and placement latency (time from request to container start).
End-to-end flow
End-to-end: a ML training job submits to the ml queue asking for 32 GPU containers with anti-affinity across racks. Placement policy filters to nodes with label GPU. Constraints spread the 32 across 8 racks (4 per rack). Gang scheduling waits until 32 are simultaneously available. The ml queue is under-utilized, but the etl queue over-quota is holding GPU containers. Preemption policy identifies replaceable etl containers and reclaims them. Within 45 seconds, all 32 ml containers launch together. Metrics show queue fairness restored, preemption count acceptable, placement latency within SLO. The ML job completes without partial-start failure.