Tomcat MaxMetaspaceSize unset: the silent OS OOM-kill with no Java error

The Tomcat process is gone. catalina.out ends mid-line or shows a normal shutdown request that never executed. There is no OutOfMemoryError, no HeapDumpOnOutOfMemoryError file, and no JFR recording. The systemd unit reports code=killed, status=9/KILL or simply a vanished PID. Your heap dashboard looked healthy right up to the moment the process disappeared.

This is the silent Metaspace kill. The JVM never throws a Java-level error because the kernel terminates it first. When -XX:MaxMetaspaceSize is left at its default (effectively unlimited), class metadata lives in native memory that is not bounded by -Xmx, not covered by heap alerts, and not reclaimed when the heap looks fine. Metaspace grows until process RSS hits the OS limit or the container cgroup limit, and the Linux OOM-killer ends the JVM with SIGKILL.

The fix is mechanical: set -XX:MaxMetaspaceSize. That flag converts an invisible, uncatchable kernel kill into a catchable java.lang.OutOfMemoryError: Metaspace that the JVM logs, that your alerting can see, and that -XX:+HeapDumpOnOutOfMemoryError can act on.

What this means

Metaspace is the native-memory region where the JVM stores class metadata: class definitions, method metadata, constant pools, and annotations. It replaced PermGen in JDK 8. Two properties matter for this incident:

1. Metaspace is not heap. It is allocated from native memory outside the -Xmx budget. Heap utilization metrics (HeapMemoryUsage, the Manager Status XML <jvm><memory> block) do not include it. A heap dashboard can be flat at 40% while Metaspace is climbing to gigabytes.

2. The default cap is unlimited. Oracle documents this directly: “The amount of native memory that can be used for class metadata is by default unlimited.” The internal value is the maximum representable integer.

Without -XX:MaxMetaspaceSize, there is no JVM-level signal that fires before the kernel does. The OS OOM-killer is the only backstop. It sends SIGKILL, which the JVM cannot intercept, cannot log, and cannot dump on. HeapDumpOnOutOfMemoryError is useless here because no OutOfMemoryError is ever thrown.

flowchart TD
  A[Classloader leak or
large class count] --> B[Metaspace grows
in native memory]
  B --> C{MaxMetaspaceSize set?}
  C -- No --> D[No JVM bound,
no Java error possible]
  D --> E[RSS hits OS or
cgroup limit]
  E --> F[Linux OOM-killer
sends SIGKILL]
  F --> G[Process gone.
No heap dump,
no catalina log]
  C -- Yes --> H[JVM throws
OutOfMemoryError: Metaspace]
  H --> I[Logged, alertable,
heap dump possible]

Common causes

Quick checks

These are read-only. Run them before touching JVM flags. If you have multiple JVMs on the host, replace the pgrep subshells with the specific PID.

# Check whether the OOM-killer fired. Look for the Tomcat PID.
# On restricted kernels (kernel.dmesg_restrict=1), this needs root or CAP_SYSLOG.
dmesg -T | grep -iE "oom|killed process"

# Confirm the Tomcat process is actually gone (standalone).
pgrep -f 'catalina.startup.Bootstrap' || echo "DOWN"

# For Spring Boot embedded, substitute the app jar.
pgrep -f 'myapp.jar' || echo "DOWN"

# Effective MaxMetaspaceSize on the running JVM. No explicit value means
# the default (unlimited) is in effect.
jcmd $(pgrep -f 'catalina.startup.Bootstrap') VM.flags | grep -i MaxMetaspaceSize

# To see the raw default for the java binary on PATH:
java -XX:+PrintFlagsFinal -version 2>&1 | grep -i MaxMetaspaceSize

# Current Metaspace usage from a live JVM.
jcmd $(pgrep -f 'catalina.startup.Bootstrap') VM.metaspace

# jstat -gc columns: MC = Metaspace Capacity (KB), MU = Metaspace Used (KB).
# Samples every 1s until interrupted (Ctrl+C).
jstat -gc $(pgrep -f 'catalina.startup.Bootstrap') 1000

# RSS is what the kernel scored before killing. VmPeak shows the high-water mark.
grep -iE 'VmRSS|VmPeak|VmSize' /proc/$(pgrep -f 'catalina.startup.Bootstrap')/status

# LoadedClassCount trend via JMX (requires jmxterm, jconsole, or equivalent).
# jmxterm has no -e execute flag (-e is exit-on-failure); pipe the command on stdin.
echo "get -b java.lang:type=ClassLoading LoadedClassCount UnloadedClassCount" | \
  java -jar jmxterm.jar -l localhost:9090 -n -v silent

The single most important check is dmesg. If it shows Out of memory: Killed process <pid> (java), the JVM was externally killed. Stop looking for a Java-level error.

How to diagnose it

1. Confirm an external kill. Run dmesg -T | grep -iE "oom|killed process". A line naming the java PID is definitive. In Kubernetes, check kubectl describe pod for Last State: Terminated, Reason: OOMKilled, Exit Code: 137. Without this evidence, you may be chasing a different failure.

2. Verify MaxMetaspaceSize is unset on the running JVM. Run jcmd <pid> VM.flags | grep MaxMetaspaceSize. No explicit value means the default is in effect. Cross-check the launch command: ps -ef | grep -- '-XX:MaxMetaspaceSize'.

3. Establish a Metaspace baseline. Once the JVM is back up, sample Metaspace over time. With jcmd VM.metaspace, look at the “used” line. With jstat -gc, watch the MC and MU columns. The baseline after warmup is what you will size against.

4. Correlate with redeploys. If you hot-deploy, sample Metaspace immediately before and after each redeploy cycle. A clean redeploy returns close to the previous baseline. A leaky redeploy adds roughly the same delta every time.

5. Check for classloader-leak warnings. Tomcat logs these on undeploy. Look in catalina.out and localhost.<date>.log for messages about threads, ThreadLocals, or JDBC drivers not being cleaned up. These identify the specific retention path.

6. Rule out a plain heap OOM. Check GC logs (if -Xlog:gc* is enabled) and heap metrics leading up to the kill. If heap was flat and RSS was climbing, native memory is the culprit. Native Memory Tracking (-XX:NativeMemoryTracking=summary, then jcmd VM.native_memory summary) separates Metaspace from thread stacks, direct buffers, and JNI. NMT requires a JVM restart to enable.

Metrics and signals to monitor

Fixes

Set a MaxMetaspaceSize bound

This is the mandatory change. Add -XX:MaxMetaspaceSize=<value> to JAVA_OPTS (or your container’s equivalent). Sizing methodology:

• Measure steady-state Metaspace with jcmd VM.metaspace after warmup under realistic load.
• If you hot-deploy, measure the per-redeploy delta and multiply by the number of redeploys you want to survive between restarts.
• Add headroom. A common starting point is roughly 2x the observed steady-state value, adjusted upward if you have a known classloader leak you have not yet fixed.
• Re-evaluate after application changes that add dependencies, frameworks, or dynamic class generation.

The bound does not fix a leak. It converts an invisible kill into a visible, alertable error. Once set, exceeding it throws java.lang.OutOfMemoryError: Metaspace, which Tomcat logs and which -XX:+HeapDumpOnOutOfMemoryError can capture. A heap dump is less useful for Metaspace leaks than for heap leaks because the retained classes live in native memory, not in the Java object graph.

Stop hot-redeploying in production

Classloader leaks on redeploy are the number-one cause of unbounded Metaspace growth in Tomcat. Each redeploy that fails to fully release the previous WebappClassLoader retains all of its loaded classes in Metaspace permanently. Tomcat’s JreMemoryLeakPreventionListener mitigates some common cases but cannot prevent application-level retention.

The reliable fix is a full JVM restart per deployment rather than in-place redeploy. Immutable container deployments naturally avoid this pattern. If you must hot-deploy, treat the leak as inevitable and size MaxMetaspaceSize and your restart cadence accordingly.

Fix the leak itself

The usual suspects:

• JDBC drivers registered with DriverManager and never deregistered in a ServletContextListener.
• ThreadLocal instances holding references to application classes, with threads that outlive the webapp.
• java.util.Timer or ScheduledExecutorService threads started by the application and never cancelled on undeploy.
• Logging framework appenders or Logger objects retaining classloader references.
• JMX MBeans registered globally and not unregistered on undeploy.
• Static fields in shared libraries referencing application classes.

Tomcat logs most of these during undeploy. Read the warnings. They name the offending thread or resource.

Account for the container limit

Inside a container, the kernel does not care whether the memory consumer is heap, Metaspace, thread stacks, or direct buffers. The cgroup memory limit is total. Set MaxMetaspaceSize such that -Xmx plus expected Metaspace plus thread stack memory (-Xss times thread count) plus native overhead stays comfortably below the container limit. Otherwise the cgroup OOM-killer fires before any JVM cap does, and you are back to a silent kill with exit code 137.

Prevention

• Always set -XX:MaxMetaspaceSize in production. Treat it like -Xmx: a non-optional bound.
• Monitor Metaspace as a first-class signal. Track java.lang:type=MemoryPool,name=Metaspace usage and trend it. It is not covered by heap metrics.
• Track LoadedClassCount. Monotonic growth after undeploy is the earliest classloader-leak signal.
• Alert on the OOM-killer, not just on Java errors. Feed kernel OOM events into your alerting. A java process killed by the OOM-killer is a production event even if no Java error was thrown.
• Prefer full restarts over hot redeploys. If hot redeploy is required, schedule periodic full restarts before Metaspace pressure becomes critical.
• Size for the container, not just the JVM. The sum of heap, Metaspace, thread stacks, and native memory must fit inside the cgroup limit with headroom.

How Netdata helps

• Per-second JVM memory-pool metrics, including the Metaspace and Compressed Class Space pools, show Metaspace growth as it happens rather than after the process is gone. A monotonic climb after each redeploy is visible without ad-hoc jcmd sampling.
• Correlation between Metaspace, LoadedClassCount, and deploy events turns a silent staircase into an explainable pattern. Overlay class-loading counters on Metaspace usage and redeploy markers in a single chart.
• RSS tracking alongside heap exposes the gap between what the JVM thinks it is using and what the kernel sees. RSS rising while heap is flat is the leading indicator of a native-memory kill.
• Kernel OOM event collection means a SIGKILL of the java process shows up in your monitoring, not just in dmesg on the host. The alert is tied to the specific PID and timestamp.

Related guides

• Tomcat java.net.BindException: Address already in use: the connector never starts
• Tomcat accepts connections but never responds: the TCP-connect trap
• Tomcat frequent Full GC: pause time, G1, and the 5% overhead rule
• Tomcat GC death spiral: full GCs dominating and throughput collapsing
• Tomcat heap dump before restart: capturing evidence with jmap and jstack
• Tomcat heap usage: watch the post-GC baseline, not the sawtooth peak
• How Tomcat actually works in production: a mental model for operators
• Tomcat HTTP Status 503 Service Unavailable: the connector is out of threads
• Tomcat process not running: crashes, OOM-kills, and failed restarts
• Tomcat maxThreads and minSpareThreads: sizing the executor correctly
• Tomcat monitoring checklist: the signals every production instance needs
• Tomcat monitoring maturity model: from survival to expert