$ guides / tomcat / tomcat-accept-queue-overflow ▌

Operations Guides

Tomcat accept queue overflow: acceptCount, somaxconn, and Recv-Q

Clients start seeing “connection refused” or TCP timeouts, but the Tomcat JVM looks healthy. The thread pool has capacity. Heap is stable. JMX shows nothing wrong. The manager status page reports normal request counts. The application logs are quiet.

The problem is in a place Tomcat cannot see: the OS-level TCP accept queue, also called the listen backlog. When this queue fills, the kernel refuses new connections by sending RST or silently dropping the completed handshake. No JMX counter tracks this. No Tomcat log records it. The only direct signal is ss output showing Recv-Q climbing toward Send-Q on the listening socket.

What this means

Tomcat’s connector has a layered buffering model. Understanding which layer is full determines what clients experience:

flowchart TD
    A[Client SYN] --> B[SYN queue
tcp_max_syn_backlog]
    B -->|handshake complete| C{Accept queue
min acceptCount, somaxconn}
    C -->|acceptor calls accept| D{NIO poller
maxConnections 10000}
    D -->|data ready| E[Worker pool
maxThreads 200]
    C -->|queue full| F[Kernel drops silently
or sends RST]

The acceptCount attribute (default 100) is passed directly as the backlog argument to ServerSocket.bind(SocketAddress, int backlog) and then to the kernel’s listen() syscall. The kernel does not blindly honor it. Linux caps the effective backlog at min(acceptCount, net.core.somaxconn). If you set acceptCount="500" but somaxconn is 128, the effective backlog is 128.

On Linux 5.4 and later, net.core.somaxconn defaults to 4096. Before 5.4, the default was 128. This matters because many tuning guides assume 128, and many deployments assume the kernel will honor whatever acceptCount requests.

Reading ss output on a LISTEN socket

For LISTEN sockets, ss shows two columns that operators routinely misread:

Column	Meaning for LISTEN socket
`Recv-Q`	Current number of completed connections waiting for `accept()`. Not bytes.
`Send-Q`	Effective maximum backlog after kernel capping. Not bytes.

For established sockets, Recv-Q means unread bytes. For LISTEN sockets, it means queued connections. Confusing the two leads to wrong conclusions.

# Check accept queue depth on the Tomcat listen socket
ss -tnl 'sport = :8080'

Output looks like:

State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port
LISTEN  0       100     0.0.0.0:8080        0.0.0.0:*

Recv-Q of 0 means the queue is drained. A brief spike during a traffic burst is normal. The problem indicator is a sustained Recv-Q that stays non-zero for more than a few seconds, especially as it approaches Send-Q.

When Recv-Q reaches Send-Q, the backlog is full. New completed handshakes cannot be queued. The kernel either drops the connection silently (the client sees a timeout) or, if net.ipv4.tcp_abort_on_overflow=1 is set, sends RST immediately. The silent-drop behavior is the default and is what makes this failure mode hard to detect from the server side.

Common causes

Cause	What it looks like	First thing to check
`maxConnections` reached	`connectionCount` equals `maxConnections`, acceptor stops calling `accept()`, Recv-Q builds	JMX `Catalina:type=ThreadPool,name="http-nio-8080"` connectionCount vs maxConnections
Acceptor blocked by GC	Recv-Q spikes that coincide with stop-the-world pauses, then drain	GC log or `jstat -gcutil` for Full GC events
`somaxconn` cap below `acceptCount`	`acceptCount` raised in `server.xml` but `Send-Q` in `ss` stays lower than expected	`sysctl net.core.somaxconn`
Container `somaxconn` trap	Host `somaxconn` is correct but container shows a lower value	Read `/proc/sys/net/core/somaxconn` inside the container namespace
Accept queue too small for burst traffic	Recv-Q hits Send-Q during sharp traffic spikes, drains within seconds	Raise acceptCount after confirming somaxconn is not the real cap

Quick checks

# Accept queue depth and effective backlog on the listen socket
ss -tnl 'sport = :8080'

# Kernel somaxconn cap
sysctl net.core.somaxconn

# Confirm whether Tomcat has hit maxConnections (requires JMX access)
# connectionCount and maxConnections are on the ThreadPool MBean:
#   Catalina:type=ThreadPool,name="http-nio-8080"

# SYN queue and accept queue overflow counters from the kernel
nstat -az TcpExtListenOverflows TcpExtListenDrops

# Check if tcp_abort_on_overflow is set (makes overflow visible as RST)
sysctl net.ipv4.tcp_abort_on_overflow

# Current established connection count at the OS level
ss -tn state established '( sport = :8080 )' | wc -l

The TcpExtListenOverflows counter from nstat tracks accept queue overflows specifically. Any non-zero value that increases during a load window confirms the queue filled, even if a spot-check ss snapshot missed it.

How to diagnose it

Capture the accept queue state. Run ss -tnl 'sport = :8080' during the incident. Note Recv-Q and Send-Q. If Recv-Q is sustained non-zero, the acceptor is not draining.
Verify the effective backlog. Compare Send-Q from ss against your configured acceptCount in server.xml. If Send-Q is lower than acceptCount, the kernel capped it at somaxconn. Check sysctl net.core.somaxconn.
Check container isolation. If Tomcat runs in a container, read /proc/sys/net/core/somaxconn from inside the container, not from the host. Container namespaces inherit the kernel constant, not the host sysctl value. On kernel 5.4+ this is 4096. On older kernels it is 128.
Determine why the acceptor is not draining. Two upstream causes: maxConnections is reached (the acceptor voluntarily stops), or the acceptor thread is paused by a stop-the-world GC. Check connectionCount vs maxConnections via JMX on Catalina:type=ThreadPool,name="http-nio-8080". Check GC activity via jstat -gcutil or GC logs.
Confirm overflow at the kernel level. Run nstat -az TcpExtListenOverflows before and after a short interval. If the counter increases, the accept queue overflowed during that window.
Correlate with client-side errors. If you have access to load balancer or client logs, look for connection refused errors or SYN retransmissions timed to the same window.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
`ss -tnl` Recv-Q on LISTEN socket	Current accept queue depth. The only direct view of this queue.	Sustained non-zero, or approaching Send-Q
`ss -tnl` Send-Q on LISTEN socket	Effective backlog after kernel capping. Confirms whether somaxconn is silently limiting you.	Lower than configured acceptCount
`TcpExtListenOverflows` (nstat)	Cumulative count of accept queue overflows. Catches transient events you missed in ss.	Any non-zero value that increases during load
`connectionCount` / `maxConnections` (JMX)	When this ratio hits 1.0, the acceptor stops calling accept() and the queue builds.	Sustained at 1.0
GC pause time	Stop-the-world pauses freeze the acceptor thread. Completed handshakes accumulate in the backlog during the pause.	Full GC events coinciding with Recv-Q spikes
`net.core.somaxconn`	Hard kernel cap on the effective backlog.	Lower than acceptCount, especially in containers

Fixes

Raise acceptCount, but check somaxconn first

Increasing acceptCount in server.xml gives the OS more room to buffer completed handshakes. This only helps if somaxconn is not the real cap. If somaxconn is 128 and acceptCount is 100, raising acceptCount to 500 changes nothing until you also raise somaxconn.

<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
           acceptCount="200" ... />

Then raise the kernel cap:

# Runtime (does not persist across reboot, affects new listen() calls only)
sysctl -w net.core.somaxconn=4096

# Persistent
echo 'net.core.somaxconn=4096' >> /etc/sysctl.d/99-tomcat.conf
sysctl -p /etc/sysctl.d/99-tomcat.conf

Restart Tomcat after changing acceptCount so the connector re-binds the listen socket with the new backlog. Existing listening sockets are not affected by runtime sysctl changes.

Fix the container somaxconn trap

In Docker and Kubernetes, the container’s somaxconn does not inherit the host’s sysctl value. It defaults to the kernel constant SOMAXCONN compiled into the kernel. On kernel 5.4+ this is 4096. On older kernels it is 128, regardless of what you set on the host.

For Docker, use the --sysctl flag:

docker run --sysctl net.core.somaxconn=4096 ...

In Kubernetes, net.core.somaxconn is classified as an unsafe sysctl. You need the kubelet --allowed-unsafe-sysctls=net.core.somaxconn flag and a pod-level security context:

apiVersion: v1
kind: Pod
spec:
  securityContext:
    sysctls:
    - name: net.core.somaxconn
      value: "4096"

On kernel 5.4+ containers, the default is already 4096, so this is primarily a concern for older kernels.

Address maxConnections saturation

If connectionCount is hitting maxConnections (default 10000 for NIO ), the acceptor stops calling accept(). Raising acceptCount alone does not help because the acceptor is choosing not to drain. The root cause is usually too many concurrent connections, often from load balancer keepalive pools or slow clients holding connections open.

Options: raise maxConnections, reduce upstream keepalive counts, or investigate connection leaks. See Tomcat accepts connections but never responds: the TCP-connect trap for the connection saturation pattern.

Fix GC pauses freezing the acceptor

The acceptor thread is a Java thread. During a stop-the-world GC pause, it cannot call accept(). Completed handshakes accumulate in the backlog during the pause. If GC pauses are frequent or long enough, the backlog fills.

This is not an accept queue problem. It is a GC problem that manifests as accept queue pressure. See Tomcat GC death spiral: full GCs dominating and throughput collapsing for diagnosing the GC side.

Make overflow visible during diagnosis

Setting net.ipv4.tcp_abort_on_overflow=1 causes the kernel to send RST when the accept queue is full, instead of silently dropping. This affects all listening sockets on the system, not just Tomcat. It makes overflow immediately visible to clients as “connection refused” rather than a timeout. It does not fix the problem. Use it temporarily during investigation, not as a permanent setting.

# WARNING: system-global setting, affects all listening sockets
sysctl -w net.ipv4.tcp_abort_on_overflow=1

Prevention

Monitor the accept queue directly. The accept queue is invisible to JMX, so it requires OS-level instrumentation. A check that samples ss -tnl and records Recv-Q relative to Send-Q catches sustained queue buildup before clients see failures.

Track TcpExtListenOverflows. This kernel counter accumulates overflows. Any non-zero value means the accept queue filled at some point. Trend it over time to catch gradual degradation.

Set acceptCount and somaxconn together. If you raise one without the other, you have not actually raised the effective backlog. Verify by checking Send-Q in ss output after applying changes and restarting Tomcat.

Verify inside containers. Do not assume the host sysctl propagates. Check /proc/sys/net/core/somaxconn from inside the container where Tomcat runs.

Do not confuse thread exhaustion with connection exhaustion. When currentThreadsBusy equals maxThreads, Tomcat still accepts connections via the NIO poller (up to maxConnections). Connections are only refused when both maxConnections and the accept queue are full. See Tomcat HTTP Status 503 Service Unavailable: the connector is out of threads and Tomcat maxThreads and minSpareThreads: sizing the executor correctly for the thread pool side.

How Netdata helps

Per-second OS-level metrics on socket states surface the accept queue without relying on JMX.
Correlating Recv-Q with GC pause time reveals whether accept queue buildup is caused by stop-the-world freezes rather than genuine connection saturation.
Correlating Recv-Q with connectionCount and maxConnections distinguishes between “acceptor voluntarily stopped because maxConnections is reached” and “acceptor stuck for another reason.”
Kernel TCP counters like TcpExtListenOverflows catch transient overflows that a spot-check ss command would miss.
Anomaly detection on the Recv-Q baseline flags sustained queue depth even when absolute numbers look small relative to the backlog limit.