GPU Scheduling With Slurm — HPC Job Scheduler

Why it matters

Many ML researchers use Slurm-managed clusters. Understanding shapes multi-node training on academic infrastructure.

The architecture

GRES: gres:gpu:N request in job script.

salloc, srun, sbatch: interactive, run, batch job submission.

How it works end to end

Partitions: named queues with policies. gpu partition for GPU nodes.

Multi-node: sbatch --nodes=N --ntasks-per-node=M. srun launches processes across nodes.

MPI + NCCL: standard for multi-node ML on Slurm.

Common patterns: pyxis + enroot for containers on Slurm.