BIND tcp-clients exhaustion: the TCP connection limit, transfers, and truncation fallback

BIND is up. rndc status shows “running.” Your UDP health check returns NOERROR in under 2ms. But zone transfers are failing, some DNSSEC-validated queries time out, and clients receiving large responses report intermittent connection refused on TCP/53. If your monitoring only probes UDP, you will not know anything is wrong until a secondary’s zone expires or a downstream resolver escalates a ticket.

The likely cause is tcp-clients exhaustion. BIND caps concurrent inbound TCP connections at a configurable limit (default 150 in BIND 9.18 and 9.20). When that quota is full, new TCP connections to port 53 are refused. Zone transfers (AXFR/IXFR) cannot complete. DNSSEC answers that exceed the EDNS0 UDP buffer size are truncated, forcing clients to retry over TCP, and that retry fails too. The named process stays healthy and UDP resolution continues for everything that fits in a single UDP datagram.

The monitoring gap makes this hard to catch early. Most DNS health checks probe UDP only. TCP is the minority protocol for typical recursive traffic, normally under 5% of queries, so it is rarely checked independently. When the TCP quota fills, there is no BIND counter that surfaces “TCP is broken.” The QryTCP counter stops incrementing because connections are refused before the query is parsed. The evidence is in the OS-level TCP socket table, BIND’s internal quota tracking, and log messages reporting the quota is reached.

What this means

The tcp-clients option limits the total number of simultaneous TCP connections named will accept. This is a single shared pool. Zone transfers, client TCP queries, DNSSEC truncation fallback, and TCP-based control connections all draw from it. BIND cannot prioritize a zone transfer over a client query or vice versa.

When the pool is full, the listener stops accepting new TCP connections. The kernel may still complete the TCP handshake (the socket sits in the listening backlog), but named will not pick up the connection. From the client side, this looks like a connection timeout or connection refused, depending on timing and backlog depth.

The connection limit also interacts with file descriptor limits. Each accepted TCP connection consumes an FD. If FDs are the binding constraint rather than tcp-clients, the symptoms are similar but the fix is different.

flowchart TD
    A[Large response or DNSSEC answer] --> B[UDP truncated - TC bit set]
    B --> C[Client retries over TCP]
    C --> D[TCP connections accumulate]
    D --> E{tcp-clients quota full?}
    E -->|No| F[Connection accepted]
    E -->|Yes| G[New TCP connections refused]
    G --> H[Zone transfers and DNSSEC fallback fail]
    K[UDP health check] -.->|Still green| G

Common causes

Quick checks

# Check current TCP connections on port 53
ss -tan '( sport = :53 )' | head -30

# Count TCP connections by state
ss -tan '( sport = :53 )' | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn

# Check tcp-clients utilization via rndc
rndc status | grep -i tcp

# Count established TCP connections on port 53
ss -tn state established '( sport = :53 )' | wc -l

# Check file descriptor usage
CURRENT=$(ls /proc/$(pgrep -x named)/fd 2>/dev/null | wc -l)
MAX=$(grep "Max open files" /proc/$(pgrep -x named)/limits | awk '{print $4}')
echo "FDs: $CURRENT / $MAX"

# Check UDP vs TCP query ratio from statistics channel.
# Requires statistics-channels configured in named.conf, e.g.:
#   statistics-channels { inet 127.0.0.1 port 8653 allow { 127.0.0.1; }; };
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); ns=d.get('nsstats',{}); \
  u=ns.get('QryUDP',0); t=ns.get('QryTCP',0); \
  print(f'UDP: {u} TCP: {t} TCP%: {t/(u+t+1)*100:.1f}%')"

# Test TCP resolution directly
dig +tcp +time=2 +tries=1 @127.0.0.1 example.com A

# Test UDP for comparison
dig +time=2 +tries=1 @127.0.0.1 example.com A

# Check for quota-related log messages
journalctl -u named --since "30 min ago" | grep -i "tcp.*client\|quota"

How to diagnose it

1. Confirm the symptom. Run dig +tcp +time=2 +tries=1 @127.0.0.1 example.com A. If it times out or gets connection refused while the same query over UDP succeeds, the TCP path is the issue.

2. Check the tcp-clients quota. Run rndc status | grep -i tcp. The output shows the current count and configured limit. If the count is at or near the limit, the quota is exhausted.

3. Inspect who holds the connections. Run ss -tan '( sport = :53 )' and examine source IPs and connection states. Many ESTABLISHED connections from secondary nameserver IPs points to transfer pressure. Connections from many distributed, unknown IPs suggest an attack. Long-lived idle connections suggest a timeout configuration problem.

4. Check file descriptor usage. If the FD count is near the limit, FD exhaustion may be the binding constraint rather than tcp-clients. Both produce similar symptoms but require different fixes. Configure FD limits via ulimit or systemd LimitNOFILE.

5. Check the TCP share in BIND statistics. Pull QryTCP and QryUDP from the statistics channel. A TCP share above 10% (normal is under 5%) indicates elevated TCP fallback or transfer activity.

6. Correlate with DNSSEC. If ValAttempt is high and signed zones produce large responses, truncation is likely driving TCP fallback. Test with dig +dnssec +time=2 +tries=1 @127.0.0.1 <signed-zone> and look for the TC bit set in the response flags.

7. Check BIND version for known bugs. Run rndc status | head -1 to get the version. Several versions have quota accounting bugs that cause the counter to get stuck (see Fixes below).

Metrics and signals to monitor

Fixes

Raise tcp-clients

The direct fix. Increase tcp-clients in named.conf:

options {
    tcp-clients 500;
};

Apply with rndc reconfig. Consider the FD budget: each TCP connection uses an FD, so the new limit should not exceed what your FD budget supports. If Max open files is 1024, raising tcp-clients to 500 may just move the cliff from tcp-clients to FD exhaustion.

Tune TCP timeouts at runtime

BIND exposes rndc tcp-timeouts for runtime adjustment without restart:

# View current TCP timeout settings
rndc tcp-timeouts

Shortening tcp-idle-timeout causes idle connections to be reaped sooner, freeing quota slots. This is the least disruptive intervention during an active incident. Be cautious with tcp-initial-timeout: a bug in some 9.17.x releases caused it to apply to the entire connection lifetime rather than just the initial query, which could prevent large zone transfers from completing.

Address DNSSEC truncation at the source

If DNSSEC answers are driving TCP fallback, review EDNS0 buffer sizing:

options {
    edns-udp-size 1232;
    max-udp-size 1232;
};

The 1232-byte value follows DNS Flag Day 2020 guidance. BIND defaults both values to 4096, which can produce fragmented UDP responses that intermediate firewalls drop, forcing TCP retry. Setting to 1232 avoids fragmentation but may increase truncation for responses between 1232 and 4096 bytes. If truncation persists at 1232, the responses genuinely exceed the buffer and the fix is raising tcp-clients.

Manage zone-transfer scheduling

If secondary-initiated transfer bursts are consuming the quota:

• Spread NOTIFY timing or transfer schedules across secondaries.
• Use also-notify with staggered intervals.
• Ensure secondaries use IXFR (incremental) rather than AXFR for routine updates.

On the primary side, serial-query-rate and transfer timing are controlled by the secondary. The primary controls how many simultaneous transfers it will accept via tcp-clients, transfers-out, and transfers-per-ns.

Mitigate TCP-based attacks

If the connections are abusive rather than legitimate:

• Apply ACLs to restrict TCP/53 to known secondaries and client ranges.
• Use rate-limit (RRL) to constrain response volume. Note that RRL “slip” truncates responses, which can itself increase TCP retries if the TCP path is also constrained.
• Deploy firewall-level SYN cookie protection or connection rate limiting at the network edge.

Check for known version bugs

Several BIND versions have bugs in TCP client quota accounting:

• BIND 9.16.0: A libuv quota counter bug could exhaust TCP connections because the counter was not properly decremented when accepting connections on multiple interfaces. Workaround: raise tcp-clients or upgrade.

• BIND 9.18.27: The TCP4Clients statistic counter could report values far exceeding the actual count and the TCP high-water mark, due to a counter decrement failure on accept error.

• Before BIND 9.18.33 / 9.20.24: A bug where named stops accepting TCP connections even after quota pressure subsides. The listener does not recover until restart. Fixed in 9.18.33 and 9.20.24.

If connections are stuck at the limit and do not drain after the triggering load subsides, a version bug is the likely cause. Check the changelog for your specific release.

Prevention

• Monitor TCP independently. Add a TCP-specific health check (dig +tcp) alongside your UDP check. If TCP fails while UDP passes, you have a tcp-clients or FD problem. This is the single most important preventive step.
• Track tcp-clients utilization. Parse rndc status for the tcp-clients count and alert before it reaches 80% of the configured limit.
• Monitor FD usage. Track FD count against the Max open files limit. Peak usage should stay below 50% of the limit.
• Alert on TCP share. Monitor QryTCP as a percentage of total queries. Sustained TCP share above 10% warrants investigation.
• Set generous FD limits. The default ulimit -n of 1024 is too low for a DNS server handling TCP traffic. Configure at least 65536 via systemd LimitNOFILE or /etc/security/limits.conf.
• Stagger zone transfers. If you operate many secondaries, spread their transfer schedules to avoid simultaneous connection bursts that fill the shared pool.

How Netdata helps

• Per-second TCP connection tracking. Netdata collects TCP connection states at per-second resolution, surfacing burst patterns that minute-level polling misses.
• FD usage alongside connection counts. Per-process FD tracking lets you distinguish tcp-clients exhaustion from FD exhaustion without manual cross-referencing.
• UDP/TCP query ratio. The BIND collector surfaces QryTCP and QryUDP, making a protocol shift immediately visible as a ratio change rather than a raw counter delta.
• Anomaly detection on connection counts. Catches unusual TCP accumulation before the quota is exhausted, complementing static thresholds.
• Correlated dashboards. TCP connection count, FD usage, zone transfer counters, and QrySERVFAIL in a single view shortens the path from symptom to root cause.

Related guides

• BIND DNSSEC validation failing: ‘broken trust chain’, ValFail, and SERVFAIL for signed domains
• BIND cache eviction storms: DeleteLRU, an undersized max-cache-size, and the pressure spiral
• BIND cache hit ratio dropping: the leading edge of recursive pain
• BIND clients-per-query and max-clients-per-query: duplicate recursion for popular names
• BIND cold cache after restart: the warming storm and elevated upstream load
• BIND DNSSEC failing from clock drift: NTP, RRSIG inception/expiry windows, and SERVFAIL
• BIND dnssec-validation disabled: the security regression that ‘fixes’ SERVFAIL
• BIND dynamic update failures: UpdateFail, denied updates, and TSIG drift
• BIND forwarding loops: recursion that never terminates and burns recursive slots
• How BIND actually works in production: a mental model for operators
• BIND inline signing silently failed: missing keys and a zone served unsigned
• BIND journal (.jnl) corruption: dynamic-update and IXFR failures that block zone load