CPU Limits and CFS Throttling: Why a Pod at 40 Percent of Its Limit Still Stalls, and How to Measure and Fix It

The symptom

A service runs in a container with a CPU limit of one core. Dashboards show it using 0.4 cores on average. Yet its p99 latency is four times what it was on a virtual machine, and the extra time appears in no profiler: the code is not slow, it is not running. The cause is usually the CPU limit itself, enforced in slices of time that are much shorter than the averaging window of any dashboard.

From a limit to a quota

Kubernetes passes requests and limits to the container runtime, which configures cgroups; the documentation describes a CPU limit as a hard ceiling the kernel enforces by throttling, and a CPU request as a weighting between containers competing for CPU. On cgroup v2 the limit becomes cpu.max, written as a quota and a period in microseconds. A limit of 1,000 millicores with the default 100 ms period becomes 100000 100000: 100 ms of CPU time per 100 ms of wall time.

The kernel’s CFS bandwidth control documentation explains the rest. Within each period the group may consume up to its quota; once the quota is spent, every runnable thread of the group waits until the next period refills it. The quota is CPU time summed across all cores. A container with eight runnable threads on an eight-core node spends 100 ms of quota in 12.5 ms of wall time, then sits idle for 87.5 ms although the node has free cores.

Worked example

A request needs 200 ms of CPU and the service spreads it over 8 threads, as parallel garbage collection, a fork-join pool or a parallel stream would. Requests arrive every 500 ms. On an idle node with no limit a request takes 25 ms. With a 1,000-millicore limit and a 100 ms period, the request spends the whole quota in 12.5 ms, waits for the next period, and finishes 12.5 ms into it: about 113 ms. The figures below come from the interactive lab’s model over two seconds.

The container never uses more than 40 percent of its limit on average, yet half of its active periods are throttled and every request is four and a half times slower than it needs to be. One thread avoids throttling and is still slower, because the work is serial. A shorter period makes it worse here: the burst is longer than the period, so it is cut into many throttled slices. And when the request lands relative to the period boundary changes the latency, which is one reason throttling shows up as jitter rather than a constant offset.

Measuring it

Read the cgroup’s cpu.stat. With the controller enabled it reports nr_periods, the number of enforcement periods that elapsed, nr_throttled, the number of times the group was throttled, and throttled_usec, the total time it was throttled. The ratio nr_throttled / nr_periods over a window is the share of periods in which the container hit its quota. cAdvisor exports the same counters as Prometheus metrics named container_cpu_cfs_periods_total, container_cpu_cfs_throttled_periods_total and container_cpu_cfs_throttled_seconds_total, which is where most Prometheus setups will find them.

Shell and PromQL; adjust label names to your metrics pipeline.

# inside the container (cgroup v2)
cat /sys/fs/cgroup/cpu.max      # e.g. "100000 100000": quota and period in microseconds
cat /sys/fs/cgroup/cpu.stat     # nr_periods, nr_throttled, throttled_usec

# PromQL: share of periods throttled over five minutes, per container
sum by (namespace, pod, container) (rate(container_cpu_cfs_throttled_periods_total[5m]))
  / sum by (namespace, pod, container) (rate(container_cpu_cfs_periods_total[5m]))

Average CPU, by contrast, is usually a rate over a minute or more of container_cpu_usage_seconds_total, which smooths a 12 ms burst and an 88 ms pause into a single unremarkable number. Correlate the ratio with latency, not with average CPU. A container that is throttled in a third of its periods while using 40 percent of its limit is a container whose limit is shaped wrong for its bursts.

Choosing a fix

In the lab, a 100 ms burst allowance brings the median back to 25 ms while the first request, which has nothing saved yet, still takes 113 ms. For runtimes that size thread pools from the number of CPUs they see, set the pool size from the limit explicitly; the JVM, for example, reads cgroup limits and sizes its defaults from them, as the guide to Java in containers describes.

Failure modes

• Judging CPU limits by average utilisation, which hides throttling that happens within 100 ms.
• Setting limits equal to requests on every container by policy, which turns every burst into throttling.
• Thread pools sized from the node’s core count inside a container limited to one or two cores.
• Shortening the period to smooth latency without checking burst length; a burst longer than the period gets worse.
• Removing limits without capacity planning by requests, so one noisy container slows its neighbours.

Verification

• Before changing anything, record the throttled-period ratio and p99 latency for a day.
• Change one container class at a time and compare both numbers at the same traffic.
• Load-test with a burst shape that matches production (arrivals per second and CPU per request), not a steady trickle.
• Alert on a sustained throttled-period ratio above a threshold you chose from that baseline.

Practice and related reading

Hands-on: CPU limits and CFS throttling: quota, periods and bursts, an interactive lab on synthetic data.

Further reading:

• Java in containers
• Kubernetes deployment for ADK Java
• Thread schedulers

Sources and scope

Checked against these sources on 22 October 2026. Examples and figures are illustrative, not measurements.

• Kubernetes documentation: Resource Management for Pods and Containers
• Linux kernel documentation: CFS Bandwidth Control (quota, period, burst, cpu.stat)
• Linux kernel documentation: Control Group v2 (cpu.max, cpu.max.burst, cpu.stat)
• cAdvisor documentation: Prometheus metrics (container_cpu_cfs_periods_total, container_cpu_cfs_throttled_periods_total)