Tomcat maxConnections saturation: the NIO poller stops accepting

Clients report connection timeouts or “connection refused” errors. The Tomcat JVM is running, the HTTP port is bound, GC looks normal, and the worker thread pool may be mostly idle. What you are looking at is maxConnections saturation: the NIO poller has filled to its configured ceiling and the acceptor has stopped registering new sockets.

The default maxConnections for NIO is 8192 (since Tomcat 9.0.30; it was 10000 for NIO on older 8.5.x and early 9.x releases). When connectionCount approaches that number, Tomcat stops accepting new connections until existing ones close. New SYNs pile into the OS TCP backlog, bounded by acceptCount (default 100). When that queue fills, the kernel stops accepting: on the default Linux setting (net.ipv4.tcp_abort_on_overflow=0) new SYNs are silently dropped and clients time out; with tcp_abort_on_overflow=1 the kernel sends RST and clients see a hard refusal.

This looks nothing like a “Tomcat is slow” incident. CPU is low, heap is fine, the thread pool is healthy, and the process is alive. Monitoring that checks only process health, thread pool, and GC will miss it entirely.

What this means

With NIO (the default connector since Tomcat 8.5), a single acceptor thread per connector calls accept() on the server socket and hands each accepted socket to the poller. The poller uses a java.nio.Selector to multiplex thousands of connections across one or two threads. Idle keepalive connections sit in the poller consuming a socket, a small buffer, and one file descriptor, but no worker thread. This is what lets NIO hold thousands of open connections with only 200 request-processing threads.

The poller’s capacity is bounded by maxConnections. When connectionCount reaches maxConnections, the acceptor stops handing sockets to the poller. Connections then accumulate in the OS accept queue (the listen backlog, sized by acceptCount, default 100). That queue is the last buffer. When it fills, the kernel rejects new SYNs with RST or silently drops them.

This is a separate limit from the worker thread pool (maxThreads, default 200). The two saturate independently. You can have a saturated poller with idle threads (lots of idle keepalive connections holding poller slots), or saturated threads with a mostly-empty poller (heavy request load, few persistent connections). The diagnostic path for each is different.

flowchart LR
  Client["Client TCP SYN"]
  Accept["Acceptor thread
accept"]
  Poller["NIO Poller
maxConnections = 8192"]
  Workers["Worker threads
maxThreads = 200"]
  Backlog["OS accept queue
acceptCount = 100"]
  RST["Kernel drops SYNs or sends RST"]
  Resp["HTTP response"]
  Client -->|connect| Accept
  Accept -->|poller has room| Poller
  Accept -.->|poller full| Backlog
  Backlog -.->|queue full| RST
  Poller -->|data ready| Workers
  Workers --> Resp

Common causes

Cause	What it looks like	First thing to check
Reverse proxy / load balancer keepalive pool	High connectionCount, low request rate, connections concentrated to one or few source IPs	`ss` source IP distribution; proxy keepalive config
Slowloris-style slow client attack	Connections from many IPs, near-zero bytes-received rate, requests never complete	bytesReceived vs connectionCount ratio
Stuck or half-open clients	ESTABLISHED connections that never send or close; FD count creeping up	`ss -tn state established`; FD count vs limit
maxConnections set too low	connectionCount hits a low ceiling quickly; configured value far below 8192	server.xml Connector attribute
maxConnections attribute typo	Changes to server.xml have no effect; ceiling unchanged	verify exact attribute name (case-sensitive)
File descriptor exhaustion first	connectionCount below maxConnections but accepts still fail; `Too many open files` in logs	`/proc/pid/limits` Max open files

Quick checks

# Connection count vs the NIO poller limit
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=ThreadPool,name=\"http-nio-8080\" connectionCount maxConnections"

# Accept queue depth (Recv-Q = current backlog, Send-Q = acceptCount)
ss -tnl 'sport = :8080'

# Count of established connections to the connector port
ss -tn state established '( dport = :8080 or sport = :8080 )' | wc -l

# Worker thread saturation (separate limit from connections)
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=ThreadPool,name=\"http-nio-8080\" currentThreadsBusy maxThreads"

# File descriptor count and process limit
TOMCAT_PID=$(pgrep -f 'catalina.startup.Bootstrap')
ls /proc/$TOMCAT_PID/fd | wc -l
cat /proc/$TOMCAT_PID/limits | grep "Max open files"

# Request throughput (low rate + high connections = idle keepalive or slow clients)
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=GlobalRequestProcessor,name=\"http-nio-8080\" requestCount bytesReceived"

# Top source IPs holding connections to the port
ss -tn state established 'sport = :8080' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head

How to diagnose it

Confirm the poller is actually full. Read connectionCount and maxConnections from the ThreadPool MBean. If connectionCount / maxConnections is above 0.90, the poller is saturated. If it is well below maxConnections, the problem is elsewhere (thread pool, backend, GC).
Check the accept queue. A non-zero Recv-Q on the listen socket means connections are waiting to be accepted. If Recv-Q is approaching Send-Q (which equals acceptCount, default 100), connections are being refused at the kernel level. There is no JMX counter for accept queue overflow; ss is the only reliable in-process check.
Distinguish idle keepalive from active attack. Compare connectionCount against request throughput. If connectionCount is high but requestCount is growing slowly, most connections are idle (keepalive pools, stuck clients, or slow-client attack). If requestCount is high and connections are high, you simply have heavy traffic and the poller limit is too low for the workload.
Inspect source IPs. If the majority of connections come from one or few IPs, suspect a reverse proxy or load balancer holding a large keepalive pool. If connections are spread across many IPs with minimal bytes received, suspect a Slowloris-style attack or broken clients.
Check file descriptors. Each connection consumes one FD. If FD count is approaching the process limit, you may hit FD exhaustion before or alongside poller saturation. Look for java.net.SocketException: Too many open files in the logs.
Rule out worker thread saturation. Check currentThreadsBusy vs maxThreads. If threads are at max but connections are below maxConnections, you have thread pool exhaustion, not poller saturation. The two require different fixes.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
connectionCount / maxConnections	Direct measure of poller saturation	Ratio sustained above 0.90
accept queue Recv-Q (`ss -tnl`)	Last buffer before RST; invisible to JMX	Sustained non-zero, approaching Send-Q
request throughput (requestCount rate)	Distinguishes idle keepalive from real load	High connections, low request rate
bytesReceived rate	Slow clients send almost nothing	High connectionCount, near-zero bytesReceived
file descriptor count / ulimit	Connections are FDs; FD exhaustion blocks accept()	Ratio above 0.80
currentThreadsBusy / maxThreads	Separate limit; rule it out	At max means thread exhaustion, not poller

Fixes

Raise maxConnections

If the workload legitimately holds many persistent connections (keepalive, WebSocket, server-sent events), the default 8192 may be too low. Raise maxConnections on the Connector in server.xml. The tradeoff: every connection is a file descriptor plus a small buffer, so you must raise the process FD limit to match. A reasonable target is maxConnections at roughly half of ulimit, leaving headroom for log files, JAR handles, and the selector FD itself. For NIO/NIO2 only, setting maxConnections to -1 disables connection counting entirely; use this with caution because it removes the only backstop before FD exhaustion.

Shorten idle connection timeouts

If the problem is idle keepalive connections accumulating, reduce connectionTimeout (default 60000ms) and keepAliveTimeout. A shorter timeout closes idle sockets faster and frees poller slots. The risk: legitimate clients on slow links may have connections closed mid-think-time. Coordinate this with your reverse proxy keepalive settings; if the proxy holds connections longer than Tomcat allows, you get connection-reset errors on the next proxied request.

Fix reverse proxy keepalive

If a single proxy or load balancer is holding a large connection pool, its keepalive configuration is the root cause. Tune the proxy’s upstream keepalive count, idle timeout, and max connections per worker so the pool stays well below Tomcat’s maxConnections. Behind nginx, for example, the upstream keepalive directive and keepalive_timeout must be coordinated with Tomcat’s connectionTimeout. Tomcat default connectionTimeout is 60s; the nginx upstream module’s keepalive_timeout (idle keepalive connections to upstream servers, since nginx 1.15.3) defaults to 60s, while nginx’s client-side keepalive_timeout defaults to 75s. If Tomcat closes first, nginx logs connection resets.

Raise the file descriptor limit

If FD exhaustion is the binding constraint, or will become so after raising maxConnections, raise the process limit. On systemd-managed Tomcat (Debian 10+, Ubuntu 20.04+), /etc/security/limits.conf is not consulted because systemd does not use PAM for service units. Use systemctl edit tomcat9.service and add LimitNOFILE=65535 under [Service], then restart. Verify with cat /proc/$(pgrep -f catalina)/limits. Do not use prlimit to change the FD limit on a running Tomcat: changing the limit underneath a running JVM is a known community-documented hazard that can break the NIO selector with errors like “Failed to register socket with selector from poller” (e.g. stackoverflow.com/q/68970415, spring-projects/spring-framework#23351); no single upstream JVM bug ID is canonically attached, so treat it as a hazard regardless of JDK version. Set the limit before the JVM starts.

Slow-client mitigation

For Slowloris-style attacks, the defense is connectionTimeout combined with external rate limiting or a WAF. connectionTimeout bounds how long Tomcat waits for a connection to start sending data after being accepted. A value of 20000ms is often sufficient for legitimate clients. For targeted attacks, identify and block source IPs at the firewall or load balancer rather than at Tomcat.

Prevention

Monitor connectionCount / maxConnections as a first-class ratio. Alert when sustained above 0.90.
Monitor accept queue Recv-Q via ss. Sustained non-zero means you are one step from RST.
Monitor file descriptor count against the process limit. Keep peak usage below 50% of ulimit.
Coordinate reverse proxy keepalive settings with Tomcat connectionTimeout. Mismatched timers cause resets and wasted connections.
Size maxConnections relative to your FD budget, not to an arbitrary number.
Verify the maxConnections attribute name in server.xml. It is case-sensitive; maxconnections is silently ignored.

How Netdata helps

The Tomcat collector pulls connectionCount and maxConnections per second from the ThreadPool MBean, so you watch the poller saturation ratio in real time rather than discovering it from client complaints.
Correlating connectionCount with request throughput and bytesReceived makes the idle-keepalive vs slow-client distinction immediate: high connections with low request rate and near-zero bytesReceived is the slow-client signature.
The OS file descriptor chart sits next to the Tomcat connection charts, so FD exhaustion shows up as a shared ceiling against the same workload.
Anomaly detection on the connectionCount baseline flags gradual accumulation (a growing reverse proxy pool, a slow leak) before the poller fills.
The accept queue depth is visible through OS-level socket statistics when that collector is enabled, closing the gap that JMX cannot see.