Off-Heap Memory Growth in a JVM: Native Memory Tracking, Direct Buffers, Metaspace, Thread Stacks and the Container Kill That Heap Graphs Do Not Explain

Core engineering · Practical technical guide · 19 October 2026

A diagnosis method for a Java process whose resident memory climbs while the heap stays flat: enable Native Memory Tracking, baseline and diff it, read the categories that matter, measure what NMT cannot see, and set the limits that turn a silent container kill into an exception you can catch.

The kill that the heap graph cannot explain

The symptom is familiar to anyone who runs Java in containers. The heap graph is a healthy sawtooth well under the maximum, garbage collection logs show nothing unusual, and the pod is killed with exit code 137 because its resident set crossed the memory limit. The heap is not the process. Resident memory also holds class metadata, compiled code, thread stacks, garbage collector data structures, direct buffers, and whatever native libraries allocate through malloc or their own allocators. Each of those can grow, and only the heap has a graph that most teams watch.

The JVM memory regions and why resident size exceeds the heap maximum are covered in the JVM memory model guide. This article is the procedure for the day the kill happens: how to find which region grew, with the tools the JDK ships.

Native Memory Tracking: the instrument

Native Memory Tracking, NMT, is a HotSpot feature that attributes the memory the JVM itself allocates to categories: Java Heap, Class, Thread, Code, GC, Compiler, Internal, Symbol, Arena Chunk, Metaspace and others. It is off by default and is enabled at start with a command line flag, at summary or detail level; detail also records call sites. The documentation puts the overhead at 5 to 10 percent and notes that NMT adds two machine words to every malloc as a header, so it is a flag for the instance you are diagnosing, not necessarily for the whole fleet. NMT cannot be started by jcmd on a running process; it can only be stopped.

The data is read with jcmd. A summary prints reserved and committed memory per category; a baseline snapshots the current state; a later summary.diff or detail.diff prints the change since the baseline, which is the only view that answers which region is growing.

Commands from the JDK 25 Native Memory Tracking and jcmd documentation.

# start the JVM with tracking (summary is usually enough to find the category)
java -XX:NativeMemoryTracking=summary -XX:MaxRAMPercentage=50 -jar service.jar

# after warm-up, take a baseline; later, diff against it
jcmd <pid> VM.native_memory baseline
jcmd <pid> VM.native_memory summary.diff scale=MB

# escalate to call sites only once the category is known
java -XX:NativeMemoryTracking=detail ...
jcmd <pid> VM.native_memory detail.diff scale=MB

# print NMT at exit, for a process that dies before you can attach
java -XX:NativeMemoryTracking=summary -XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics ...

Read the diff by committed bytes, not reserved. Reserved is address space the JVM has claimed, and the Java heap alone reserves its full maximum at start; committed is what is backed by memory and what the container limit counts. A category whose committed value grows steadily between diffs is the suspect.

Reading the categories that grow in practice

Category in the diff	What usually grows it	Limit or lever
Thread	More platform threads; each reserves a stack and commits what it touches	-Xss or -XX:ThreadStackSize; a bounded pool; virtual threads for request work
Class and Metaspace	Class loader leaks: redeploys, dynamic proxies, scripting engines that define classes per call	-XX:MaxMetaspaceSize turns growth into an OutOfMemoryError you can see
Code	JIT output; a large application compiles a lot of code	-XX:ReservedCodeCacheSize; rarely a leak
GC	Collector data structures sized from the heap; remembered sets under G1	Collector choice; grows with heap, not with time
Other	Direct byte buffers allocated through Unsafe, which java.nio direct buffers use	-XX:MaxDirectMemorySize; release buffers deterministically
Internal, Arena Chunk, Symbol	JVMTI agents, interned strings, parser arenas	Agent audit; usually stable
Not in NMT at all	JNI libraries, third-party allocators, malloc fragmentation	Compare RSS with the NMT total; see below

The direct buffer case deserves a note. Buffers allocated with ByteBuffer.allocateDirect are freed when the buffer object is collected, so their memory is reclaimed only as fast as the garbage collector runs. A service with a large, calm heap collects rarely, and direct memory accumulates between collections even without a leak. The maximum is set by the MaxDirectMemorySize flag; when it is not set the limit defaults to the maximum heap size, which means a 1 GiB heap permits another gigabyte of direct memory that no heap graph shows. Libraries that pool direct buffers, such as network frameworks, manage their own pools above this and expose their own counters.

Measuring what NMT cannot see

NMT tracks the JVM's own allocations. It does not see memory allocated by native libraries through JNI or by their own allocators, and it does not see fragmentation inside the C allocator. The method is subtraction: read the process resident set from the operating system, read the NMT total committed, and treat the difference as native-and-allocator memory. On Linux the jcmd command System.map prints the process memory mappings annotated with what the JVM knows about each range, which separates JVM regions from anonymous mappings owned by someone else.

One cause of the difference is glibc's malloc arenas. To reduce lock contention glibc gives threads their own arenas, up to a limit computed as a multiple of the CPU count, and each arena keeps freed memory for reuse. A process with many threads that allocate natively, a common pattern around compression and TLS libraries, can hold hundreds of megabytes in arena free lists. The mallopt manual documents M_ARENA_MAX as a hard limit on the number of arenas and the MALLOC_ARENA_MAX environment variable as the way to set it without code; reducing it trades some allocator contention for a smaller, more predictable resident set. Alternatively, a different allocator such as jemalloc can be preloaded, with its own profiling.

Illustrative bash, checked for syntax; adapt names and storage to your system.

# resident set from the kernel, in kB
grep VmRSS /proc/<pid>/status

# NMT total, committed, for the same moment
jcmd <pid> VM.native_memory summary scale=KB | grep -m1 Total

# JVM-annotated memory map (Linux); anonymous ranges without a JVM owner are native
jcmd <pid> System.map

# cap glibc arenas for the next start; measure before and after
MALLOC_ARENA_MAX=2 java -XX:NativeMemoryTracking=summary -jar service.jar

Inside the JVM, the java.nio buffer pools are exposed as management beans, so direct and mapped buffer counts and bytes can be read without NMT and graphed next to the heap. That graph, not the heap graph, is the one that would have predicted the kill in the direct buffer case.

Illustrative Java; export these values to your metrics system.

import java.lang.management.BufferPoolMXBean;
import java.lang.management.ManagementFactory;

for (BufferPoolMXBean pool : ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class)) {
    // names: "direct" and "mapped"
    System.out.printf("%s count=%d used=%d MiB capacity=%d MiB%n", pool.getName(),
        pool.getCount(), pool.getMemoryUsed() >> 20, pool.getTotalCapacity() >> 20);
}

Worked example: a gateway killed every nine hours

In this illustrative case a gateway runs in a container with a 2 GiB limit and MaxRAMPercentage set to 50, so the heap maximum is 1 GiB. Resident memory starts at 1.3 GiB and climbs to the limit in about nine hours; the heap graph never passes 700 MiB. The team enables NMT at summary level on one replica, baselines after ten minutes and diffs hourly. Thread, Class and Code are flat. Other grows by roughly 60 MiB per hour. That category points at direct buffers, and the buffer pool bean confirms it: the direct pool count grows without bound while used bytes track it.

The detail diff on the next replica shows the call sites: a response-streaming path allocates a direct buffer per request and relies on the collector to free it, and the heap is large enough that collections happen rarely. Two fixes are made. The code releases the buffer explicitly at the end of the request through the framework's reference-counted pool, which stops the growth. And MaxDirectMemorySize is set to 256 MiB, so that a future regression fails with an OutOfMemoryError that names direct buffer memory, in the logs, within minutes, instead of a 137 exit code nine hours later with no message. The remaining gap between resident memory and the NMT total, around 150 MiB, is attributed to malloc arenas by the memory map, and MALLOC_ARENA_MAX set to 2 removes most of it.

Failure modes worth testing

Watching only the heap. Add resident set, NMT total committed, direct buffer bytes and thread count to the dashboard that already has the heap.
Container limit equals heap plus a guess. Size the limit from a measured resident set under load, with headroom; MaxRAMPercentage of 75 leaves little room for everything else.
Unbounded direct memory. With no MaxDirectMemorySize the default is the heap maximum. Set it below what the container can afford.
Unbounded metaspace. MaxMetaspaceSize is unlimited by default; a class loader leak grows until the kill. A limit turns it into an exception with a stack trace.
Threads per request. Each platform thread commits stack pages as it uses them; a pool that grows under load grows resident memory with it. Bound the pool or move to virtual threads for blocking request work.
NMT on the whole fleet without measuring the cost. The overhead is real; enable it on a canary.
Reading reserved instead of committed. Reserved is address space; the kill is on committed resident pages.

What to do next

Add resident set size, NMT total committed and direct buffer pool bytes to the service dashboard beside the heap.
Enable NMT at summary level on one replica for a day; baseline after warm-up and diff hourly.
Set MaxDirectMemorySize and MaxMetaspaceSize to values the container can afford, so the next growth fails loudly.
Compare resident set with the NMT total; if the gap grows, try MALLOC_ARENA_MAX and look at the JVM-annotated memory map.
Re-derive the container limit from the measured resident set under load, and write the derivation next to the limit.

Sources and scope

Checked against these sources on 19 October 2026. Examples and figures are illustrative, not measurements.

Enable NMT on a canary, baseline and diff by committed bytes, compare resident set with the NMT total, and set MaxDirectMemorySize and MaxMetaspaceSize so growth fails loudly.