How Tomcat actually works in production: a mental model for operators

Tomcat looks like a single process serving HTTP, but inside it is a nested container hierarchy with a three-tier buffering model in front of your servlet code. Most production incidents are not bugs in Tomcat; they are mismatches between what the operator thought was happening and what the connector and thread pool were doing. The thread pool exhausts while CPU sits at 5%. The JVM is alive but every request hangs. Sessions fill the heap while request rate looks normal. Clients see connection timeouts while Tomcat logs nothing.

This is the mental model that makes those symptoms legible: the Server -> Service -> Connector -> Engine -> Host -> Context -> Servlet nesting, the NIO connector’s separation of connection multiplexing from request processing, the executor as the real capacity model, the per-webapp classloader, and how StandardManager pins sessions on the heap.

What it is and why it matters

Tomcat is a Java servlet container. It accepts TCP connections, maps them to application code (servlets, filters, JSPs), and returns HTTP responses. The nested hierarchy is:

Server -> Service -> Connector -> Engine -> Host -> Context -> Servlet

• A Server is the entire Tomcat process. One JVM, one Server.
• A Service groups one or more Connectors with one Engine. Most deployments have one Service.
• A Connector binds a port and implements an I/O model. Default ports are 8080 (HTTP), 8443 (HTTPS), and 8009 (AJP).
• An Engine receives requests from Connectors and routes them to Hosts.
• A Host is a virtual host. It maps hostnames to web applications.
• A Context is a single web application. Each Context gets its own classloader and session manager.
• A Servlet is the application code that produces a response.

Operators rarely manipulate the top of this hierarchy. The two layers that decide whether your service stays up are the Connector (I/O model, connection limits) and the Context (classloader, sessions, application state). Everything above Engine is configuration you set once and forget.

Every Tomcat incident is a story about one layer hitting a limit. Thread exhaustion is the Executor. Connection drops are the Connector’s poller plus the OS accept queue. Metaspace OOM is the Context’s classloader. Heap OOM is usually the Context’s session manager or the application’s own state. If you do not know which layer owns which resource, you debug blindly.

How it works

The NIO connector: three populations, not one

The default connector is NIO (non-blocking I/O built on java.nio.Selector). NIO separates three populations that older BIO Tomcat conflated: accepted connections, sockets with data ready, and requests being processed.

flowchart LR
  SYN["TCP SYN from client"] --> AQ["OS accept queue
acceptCount default 100"]
  AQ --> AC["Acceptor thread
accept(), hand to poller"]
  AC --> PL["Poller threads
maxConnections default 8192"]
  PL --> WP["Worker pool
maxThreads default 200"]
  WP --> SRV["Servlet
thread held for full request"]

• Acceptor: a single thread per connector calling accept() on the server socket. It hands the socket to the poller. If the acceptor is blocked by a GC pause or because maxConnections is reached, no new connections are received.
• Poller: a small number of threads using selector multiplexing to watch registered sockets for read or write readiness. Capacity is bounded by maxConnections (default 8192 for NIO). An idle keepalive connection occupies a poller slot and a file descriptor, but not a thread.
• Worker pool: the Executor. This is where request processing happens. Default minSpareThreads=10, maxThreads=200. Each active request occupies one thread for the entire request lifecycle, unless the application uses async servlets.

This separation is why Tomcat can hold thousands of idle keepalive connections without dying. Under BIO, every keepalive connection held a thread. Under NIO, an idle connection is cheap. The expensive resource is the worker thread.

The executor is the capacity model

The thread pool is Tomcat’s heart. Every active request consumes one thread. When the pool is full, new requests queue. When the poller is full, the acceptor stops calling accept(). When the OS accept queue behind the connector is also full, the kernel drops new connections.

The defaults, maxThreads=200, minSpareThreads=10, acceptCount=100, and maxConnections=8192 (NIO), imply a specific failure order:

1. Worker threads fill (maxThreads reached). Requests wait for a free thread.
2. Connections still arrive. The poller holds them up to maxConnections.
3. Once maxConnections is reached, the acceptor stops calling accept(). New SYNs pile up in the OS accept queue, bounded by acceptCount.
4. When acceptCount fills, the kernel drops or resets new connections. On Linux with default settings (tcp_abort_on_overflow=0), the ACK is silently dropped and clients see timeouts. With tcp_abort_on_overflow=1, the kernel sends RST and clients see connection refused.

Operators who do not know this order interpret the symptoms backwards. They see “Tomcat is up, port is open, but clients time out” and assume Tomcat crashed. It did not. The accept queue is full. Tomcat has no JMX counter for accept queue depth. The only signal is OS-level ss -tnl Recv-Q.

A second consequence: maxThreads is not a performance knob. It is a statement about how many concurrent requests your backends can sustain. Setting maxThreads=2000 when the JDBC pool is sized at 20 just moves the queue from Tomcat to the database.

Sessions: StandardManager pins them on the heap

The default session manager is StandardManager. Sessions live in a ConcurrentHashMap on the JVM heap. Default session timeout is 30 minutes. There is no eviction based on memory pressure. Sessions stay until they expire or are invalidated.

This makes sessions a direct heap consumer with no backpressure. A burst of bot traffic that creates a session per request, combined with a long timeout, fills old gen. The application code looks fine. The heap graph shows old gen climbing after GC. The thread pool is healthy. The cause is the session manager accumulating stateful objects nobody asked it to hold.

PersistentManager (serialize to disk or database) and the cluster managers (DeltaManager, BackupManager) exist but are not the default. Most production deployments either accept StandardManager’s heap behavior or move sessions to an external store like Redis.

Per-webapp classloader: isolation with a leak trap

Each Context gets its own WebappClassLoader. This is what lets you deploy application A and application B in the same Tomcat without their libraries colliding. It is also the source of the most notorious Tomcat operational issue.

When a webapp is undeployed, its classloader and all the classes it loaded should be garbage collected. They are not collected if anything still references them: a ThreadLocal, a JDBC driver registered with the global DriverManager, a thread the application started and never stopped, a static field in a shared library, a log appender. The classloader and every class it loaded are pinned in Metaspace.

Each hot redeploy that leaks adds another copy of the application’s classes to Metaspace. After enough redeploys, Metaspace is exhausted. If MaxMetaspaceSize is not set, native memory grows until the kernel OOM killer kills the process (in a container, against the cgroup limit) with no JVM-level OutOfMemoryError: Metaspace preceding it.

In containerized deployments that never hot redeploy, this is a non-issue. In environments that do hot redeploy, it is the single most reliable way to slowly kill a Tomcat.

Valves and the request pipeline

Valves are pipeline components at the container level, analogous to filters but outside the application. AccessLogValve, RemoteAddrValve, and StuckThreadDetectionValve are the operationally relevant ones. Valves execute synchronously in the request thread, which means a slow valve is a slow request and a stuck valve is a stuck thread. StuckThreadDetectionValve is not configured by default. Many teams discover this during an incident when they need stuck thread counts and the MBean does not exist.

Where it shows up in production

The mental model maps directly onto the failure archetypes you see in incidents.

The pattern: most Tomcat outages are not Tomcat bugs. They are bounded resources hitting limits while adjacent resources look healthy. CPU low and threads at max means I/O wait, not overload. Heap fine and RSS climbing means off-heap growth, not a leak in your application code. Process alive and port open tells you nothing about whether requests are being served.

Tradeoffs and when this matters

This mental model matters most when:

• You are sizing a deployment. maxThreads is not a knob you tune for throughput in isolation; it is a statement about how many concurrent requests your backends can sustain before they degrade. A 200-thread pool in front of a 20-connection database pool just relocates the queue.
• You are interpreting a “Tomcat is slow” page. Without knowing whether the slowness is in the worker pool, the poller, the accept queue, or the servlet, you cannot pick a fix. Thread dumps and accept queue depth tell you which layer is saturated.
• You are choosing a redeploy strategy. Hot redeploy trades convenience for classloader leak risk. Immutable containers trade startup time for leak-free Metaspace.
• You are behind a reverse proxy. Connection metrics reflect the proxy’s keepalive behavior, not end-user patterns. A single nginx can hold dozens of keepalive connections to Tomcat, which is normal and not a connection leak. Coordinate proxy keepalive timeout with Tomcat’s connectionTimeout (code default 60000ms; the stock server.xml ships 20000ms) to avoid reset storms.

It matters least when:

• You are running embedded Tomcat (Spring Boot) with no Manager app, no server.xml, and no hot redeploy. The failure modes are the same, but several configuration surfaces disappear.
• You are running a stateless service with no sessions and no database pool. The session and JDBC layers drop out of the failure surface.

Signals to watch in production

How Netdata helps

• Per-second collection of currentThreadsBusy, currentThreadCount, and maxThreads from the Catalina:type=ThreadPool MBeans lets you see the executor saturate in real time, before a five-minute aggregation window hides the cliff edge.
• Correlating thread pool saturation with request processing time and GC pause time in the same UI separates backend-driven thread exhaustion from GC-driven latency. CPU low with threads at max points one direction; CPU high with Full GCs points another.
• JVM memory pool charts break out Eden, Survivor, Old Gen, and Metaspace separately. The post-GC old gen baseline and the Metaspace staircase across redeploys are visible without manual jstat sessions.
• Anomaly detection on session and connection counts surfaces slow accumulation that static thresholds miss: sessions creeping up from bot traffic, connections creeping up from a misconfigured proxy keepalive pool.
• OS-level metrics alongside JVM metrics let you read accept queue depth, file descriptor usage, and process RSS in the same view as the Tomcat signals, which is where the real diagnosis happens.

Related guides

• Tomcat monitoring checklist: the signals every production instance needs
• Tomcat monitoring maturity model: from survival to expert
• Tomcat process not running: crashes, OOM-kills, and failed restarts
• Tomcat accepts connections but never responds: the TCP-connect trap
• Tomcat java.net.BindException: Address already in use: the connector never starts
• Tomcat thread pool exhaustion: currentThreadsBusy at maxThreads and requests hanging
• Tomcat HTTP Status 503 Service Unavailable: the connector is out of threads
• Tomcat maxThreads and minSpareThreads: sizing the executor correctly
• Tomcat threads busy but CPU idle: telling a blocked backend from a GC spiral
• Tomcat virtual threads (JDK 21+): why currentThreadsBusy reports -1
• Tomcat java.lang.OutOfMemoryError: Java heap space: the heap is genuinely full
• Tomcat OutOfMemoryError: GC overhead limit exceeded: GC running but freeing nothing