Why architecture matters here
Scheduler architecture matters because it directly affects tail latency, fairness, and throughput. A scheduler that isn't NUMA-aware ping-pongs threads and destroys cache; a scheduler that lacks priority discipline lets background jobs steal cycles from interactive workloads. Tuning it is often the difference between p99 5ms and 50ms.
Cost impact is real. Right scheduling policy + affinity can free 20-30% of throughput on cache-sensitive workloads.
Reliability under load is where the scheduler shines when tuned right and hurts when not. Real-time classes for critical work, cgroups for isolation, cpusets for affinity keep behavior predictable.
The architecture: every layer explained
Walk the diagram top to bottom.
Runnable Tasks. Threads and processes ready to run. Each has state, priority, and history.
Scheduler. The kernel component that picks next task per CPU. Runs on every timer tick + on wakeups.
CPUs. Physical cores + hyperthreads. Scheduling accounts for both.
CFS (Linux default). Completely Fair Scheduler tracks virtual runtime per task; the task with the lowest vruntime runs next. Red-black tree keeps sorted access.
Priority + Niceness. nice values (-20 to +19) skew vruntime accumulation. Priorities via classes (real-time, batch, idle).
Work Stealing. User-space (Go runtime, Java ForkJoin, Rust Tokio) implements its own scheduling on top of kernel threads. When a worker idles, it steals from busy workers' queues.
Preemption. Kernel preempts a running task when timer fires or higher-priority task wakes.
Real-time. SCHED_FIFO (run until yield/higher priority) or SCHED_RR (round-robin at same priority). Use carefully; can starve normal work.
cgroups + Kubernetes. Container isolation. CPU limits, requests. Kubernetes maps to cgroups.
NUMA + affinity. Threads run near their memory. sched_setaffinity() pins to specific CPUs.
End-to-end scheduling event flow
Trace a scheduling event. Application has 100 threads running on 8 CPUs.
Timer tick fires. Kernel enters scheduler on CPU 0. Current task's vruntime is updated (add time consumed × 1024 / weight). Kernel checks: is there a task with lower vruntime?
CFS looks at leftmost node of red-black tree for CPU 0's run queue. Selects. Context switch: save current registers, load new task's registers, switch TLB if needed.
Higher-priority thread (say, real-time) wakes elsewhere. Kernel preempts even mid-quantum. Real-time thread runs.
Meanwhile Go program has 1000 goroutines but only 8 kernel threads. Go's user-space scheduler distributes goroutines across those threads with work stealing. Idle worker steals from busy worker's queue.
CPU 4 is idle. Its scheduler checks for tasks. Local run queue empty. Load balancer moves a task from CPU 2 (loaded) to CPU 4. Cache warm-up cost paid; balance restored.
NUMA-aware scheduling: task with memory on NUMA node 0 preferred on CPUs 0-3 (co-located). Moving it to CPUs 4-7 (other NUMA node) would incur remote memory access.