$ guides / tomcat / tomcat-connection-refused ▌

Operations Guides

Tomcat connection refused: maxConnections and acceptCount both exhausted

Clients are getting “connection refused” or connections hanging until timeout. The Tomcat JVM is up, the HTTP port is bound, heap looks fine, and there is nothing in catalina.out. Manager status shows the connector alive. From Tomcat’s perspective, nothing is wrong. From the kernel’s perspective, the listening socket’s accept queue is full and new SYNs are being dropped or RST’d.

This happens when two limits stack: the NIO poller has hit maxConnections (default 8192 for NIO) and the OS accept queue, bounded by acceptCount (default 100) and clamped by net.core.somaxconn, is also full. No JMX counter exposes this state. Tomcat logs nothing. Detection is either client-side connection error monitoring or ss -tnl showing Recv-Q stuck at the accept queue limit.

Tomcat is silent for a structural reason. Once maxConnections is reached, the acceptor thread stops calling accept() on the server socket. The kernel keeps completing TCP handshakes and queuing the resulting connections in the listen backlog. Those queued connections live entirely in kernel space. Tomcat cannot see them, count them, or log them. When the backlog fills, kernel behavior depends on net.ipv4.tcp_abort_on_overflow: with the default of 0, SYNs are silently dropped and clients retry until they time out; set to 1, the kernel sends RST immediately and clients get an immediate refusal.

What this means

A client connection passes through two gates before any worker thread processes it.

The kernel completes the TCP handshake and places the connection in the accept queue. The queue depth is bounded by min(acceptCount, net.core.somaxconn).
Tomcat’s acceptor thread calls accept(), removes the connection from the queue, and registers it with the NIO poller.
The poller watches for read readiness and dispatches to a worker thread when data arrives.

Two limits gate this path:

maxConnections (default 8192 for NIO): bounds how many connections the poller tracks. When reached, the acceptor stops calling accept().
acceptCount (default 100): passed to listen() as the backlog. The kernel caps the effective backlog at net.core.somaxconn.

When the acceptor stops draining, the OS queue fills. When the OS queue fills, the kernel refuses further connections. The accept queue itself is invisible to JMX. The only Tomcat-side signal is connectionCount approaching maxConnections, which tells you the poller is near its limit but says nothing about whether the OS queue is backing up behind it.

flowchart TD
    A["Client SYN"] --> B{"maxConnections reached?"}
    B -- No --> C["Poller tracks connection"]
    B -- Yes --> D{"acceptCount queue full?"}
    D -- No --> E["Kernel queues in backlog"]
    E --> C
    C --> F["Dispatch to worker thread"]
    D -- Yes --> G["Kernel drops SYN or sends RST"]
    G --> H["Client sees refused or timeout"]

Common causes

Cause	What it looks like	First thing to check
Thread pool exhaustion feeding connection pile-up	`currentThreadsBusy == maxThreads` sustained, then `connectionCount` climbs toward `maxConnections`	`jstack` to see what worker threads are blocked on
Slow client or keepalive flood	`connectionCount` high but request rate low, many idle keepalive sockets from few source IPs	`ss -tn state established 'sport = :8080'` grouped by peer
Backend dependency stall	Worker threads blocked on socket read to one backend, processing time climbing before refusal starts	Thread dump stack traces, backend latency metrics
`acceptCount` too low for burst traffic	Recv-Q hits the configured limit during normal peaks but `connectionCount` never approaches `maxConnections`	`ss -tnl 'sport = :8080'` Recv-Q vs Send-Q
`net.core.somaxconn` capping acceptCount	You raised `acceptCount` but Recv-Q still maxes out at 128 or 4096	`cat /proc/sys/net/core/somaxconn`
WebSocket connection leak	`connectionCount` climbs monotonically, never drops, no corresponding request throughput	Check Tomcat version against CVE-2024-23672 fixed releases

Quick checks

These are read-only and safe to run during an incident.

# Check accept queue depth on the listening socket.
# Recv-Q is current backlog depth; Send-Q is the configured max (acceptCount, clamped by somaxconn).
# If Recv-Q equals Send-Q, the queue is full and connections are being refused.
ss -tnl 'sport = :8080'

# Check effective somaxconn (caps acceptCount).
cat /proc/sys/net/core/somaxconn

# Check kernel overflow behavior (0 = silent drop, 1 = RST).
cat /proc/sys/net/ipv4/tcp_abort_on_overflow

# Current connection count vs maxConnections via JMX (requires JMX enabled).
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=ThreadPool,name=\"http-nio-8080\" connectionCount maxConnections"

# Thread pool state (if connections pile up because threads are stuck).
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=ThreadPool,name=\"http-nio-8080\" currentThreadsBusy maxThreads"

# Count established connections to the connector.
ss -tn state established '( sport = :8080 )' | wc -l

# Top peer IPs by connection count (slowloris or keepalive flood indicator).
# Note: cut -d: -f1 extracts IPv4 peer IPs only. For IPv6 peers, use a more robust parser.
ss -tn state established 'sport = :8080' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head

# Recent SYN drops or overflow events in kernel log.
# May require root. On some modern kernels this produces no output for accept backlog overflow.
dmesg -T | grep -iE 'syn|overflow|backlog' | tail

# Confirm Tomcat version (relevant for CVE-2024-23672 WebSocket DoS).
${CATALINA_HOME}/bin/version.sh

How to diagnose it

Confirm the accept queue is full. Run ss -tnl 'sport = :8080' and compare Recv-Q to Send-Q. Recv-Q stuck at or near Send-Q while clients report errors confirms the OS is refusing connections. If Recv-Q is zero, the problem is upstream of Tomcat: DNS, firewall, load balancer, or client-side.
Find the Tomcat-side limit driving it. Pull connectionCount and maxConnections via JMX. If connectionCount is at maxConnections, the NIO poller is full and the acceptor has stopped draining the OS queue. If connectionCount is well below maxConnections but the accept queue is still full, the acceptor thread itself is stalled, usually by a long GC pause.
Check whether the thread pool is the root driver. Pull currentThreadsBusy and maxThreads. If busy equals max, threads are stuck on a backend and connections are accumulating because requests cannot be processed. Take a thread dump with jstack <pid> and look for http-nio exec threads blocked on the same socket read or connection pool acquire. That identifies the failing dependency.
Check for keepalive or slow clients filling the poller. Compare connectionCount to request throughput. High connection count with low request rate means many idle keepalive connections. Group established connections by peer IP. A handful of IPs holding thousands of connections points to a misconfigured reverse proxy keepalive pool or a slowloris-style attack.
Verify the effective accept queue size, not just the configured one. acceptCount is clamped by net.core.somaxconn. If you set acceptCount=1000 but somaxconn=128, the effective backlog is 128. Check cat /proc/sys/net/core/somaxconn.
Correlate with GC activity if the issue is intermittent. A long full GC pauses the acceptor thread. Connections back up in the OS queue and, if the pause is long enough, the queue overflows. Check GC logs or run jstat -gcutil <pid> 1000 during the incident window.
Do not rely on kernel SYN flood messages. Modern kernels do not reliably log accept backlog overflow. Rely on ss or client-side monitoring instead.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Accept queue Recv-Q (`ss -tnl`)	Direct measurement of the OS backlog; the only signal for this failure mode	Recv-Q approaching Send-Q, sustained
`connectionCount` / `maxConnections` (JMX)	Tells you when the NIO poller is about to stop accepting	Ratio sustained above 0.8
`currentThreadsBusy` / `maxThreads` (JMX)	Thread exhaustion is the usual upstream cause of connection pile-up	Ratio sustained at 1.0
Client-side connection error rate	The only reliable end-to-end signal; Tomcat logs nothing here	Spike in “connection refused” or connect timeouts
`net.core.somaxconn`	Caps the effective acceptCount	Lower than configured acceptCount
GC pause time	Long pauses stall the acceptor and back up the OS queue	Full GC pauses long enough to fill the accept queue
File descriptor count	Each connection consumes an fd; FD exhaustion produces similar symptoms	Count above 80% of ulimit

Fixes

Thread pool exhaustion is the upstream cause

If currentThreadsBusy == maxThreads is driving the connection pile-up, the fix is not to raise maxConnections or acceptCount. Connections are piling up because requests cannot be processed. Raising limits just delays the refusal and makes the eventual failure larger.

Take a thread dump and identify what worker threads are blocked on. The stack trace tells you which backend is slow.
Set timeouts on outbound calls: JDBC query timeouts, HTTP client socket and connect timeouts, connection acquire timeouts. Defaults are often infinite, which guarantees thread consumption on any backend stall.
If a backend is genuinely down, shed load at the load balancer rather than letting Tomcat accept connections it cannot process.
Only after the backend issue is fixed, evaluate whether maxThreads is correctly sized for steady-state load. See the maxThreads and minSpareThreads guide.

Keepalive or slow clients filling the poller

If connectionCount is high relative to request rate, idle keepalive connections are consuming poller slots.

Tighten connectionTimeout (default 60000ms) and keepAliveTimeout so idle connections are reaped faster.
If behind a reverse proxy, coordinate the proxy’s upstream keepalive pool size with Tomcat’s maxConnections. A proxy holding 200 idle keepalive connections to each of 50 Tomcat instances consumes 10000 poller slots across the fleet.
For slowloris-style attacks, ensure connectionTimeout is set and consider rate-limiting at the edge.

acceptCount or somaxconn too low for burst traffic

If connectionCount never approaches maxConnections but the accept queue still fills during bursts, the OS backlog is the bottleneck.

Raise acceptCount on the Connector. This is the cheapest change and is safe up to the level your kernel supports.
Raise net.core.somaxconn to match. On systemd-managed hosts, set this via a sysctl drop-in under /etc/sysctl.d/ so it persists across reboots.
On kernel 5.4 and later, somaxconn defaults to 4096, which is usually sufficient. On older kernels the default is 128, which is far too low for production.

WebSocket connection leak

If connectionCount climbs monotonically and never drops, suspect a WebSocket leak.

Check whether the Tomcat version is vulnerable to CVE-2024-23672, a WebSocket DoS via incomplete cleanup. Fixed in 11.0.0-M17, 10.1.19, 9.0.86, and 8.5.99. Upgrade if below those.
For application-level WebSocket leaks, ensure server-side close handlers are invoked when clients disconnect uncleanly.

Disabling the limit entirely (NIO and NIO2 only)

For NIO and NIO2 connectors, setting maxConnections="-1" removes the poller-side limit and lets the acceptor drain the OS queue continuously. This shifts the bottleneck entirely to the OS accept queue and file descriptors. Use this only if you have monitoring for fd count and Recv-Q, and only if the upstream cause is already fixed. Removing the limit without fixing the root cause just moves the cliff.

Prevention

Monitor accept queue Recv-Q at the OS level. This is the only direct signal for this failure mode. There is no JMX counter.
Monitor client-side connection error rates. Because Tomcat logs nothing when the kernel refuses connections, client-side or synthetic check monitoring is the most reliable detection.
Treat connectionCount approaching maxConnections as a leading indicator, not a page on its own. It means the poller is about to stop draining the OS queue.
Set timeouts on all outbound calls. Infinite timeouts are the most common root cause of thread exhaustion that cascades into connection refusal.
Verify somaxconn is not at the legacy default of 128 on any production host.
Do not rely on kernel SYN flood messages. They are not a reliable signal for accept backlog overflow on modern kernels.

How Netdata helps

Netdata’s Linux networking collectors surface TCP listen socket backlog depth and overflow counters at per-second resolution. This is the direct signal for accept queue overflow and is not available via JMX.
The Tomcat collector surfaces connectionCount, maxConnections, currentThreadsBusy, and maxThreads per connector, so you can watch the poller fill up before the OS queue behind it overflows.
JVM collectors surface GC pause duration, letting you correlate acceptor stalls with full GC events in the same timeline.
ML anomaly detection on connectionCount and listen-socket backlog catches the slow monotonic climb of a WebSocket or keepalive leak before it hits the limit.
Correlating client-side synthetic check failures with server-side backlog depth and connectionCount in a single view shortens diagnosis from “Tomcat looks fine, must be the network” to “accept queue is full because the poller is full because the thread pool is stuck on the database.”