BIND random subdomain (water torture) attack: an NXDOMAIN flood that bypasses the cache

Your BIND resolver is up, the process is running, UDP and TCP both answer on port 53. But clients across the network are experiencing slow DNS or outright SERVFAIL. NXDOMAIN rate has spiked to several times baseline. Cache hit ratio is collapsing. Recursive clients are climbing toward the hard limit. Upstream query rate has ballooned to approach or exceed the inbound rate.

This is a random subdomain attack, also called water torture or PRSD (Persistent Random Subdomain). Attackers flood your resolver with queries for randomized subdomains of a real domain: a1b2c3.victim.com, x7y8z9.victim.com, q2w3e4.victim.com. Every name is unique, so the cache cannot help. Each query is a miss, each miss forces recursion, and each recursion consumes a recursive-client slot and CPU. The victim’s authoritative server is hammered, but the real damage is collateral: your resolver becomes so saturated that resolution for every client, for every domain, degrades.

The distinguishing feature is high query-name cardinality concentrated on one parent domain with near-zero repetition. Legitimate traffic repeats names (clients ask for google.com thousands of times). Attack traffic does not.

What this means

A warm BIND cache answers 80-95% of queries from memory in under a millisecond. The water torture attack defeats the cache entirely by ensuring no two queries ever match.

Each unique random subdomain triggers the full recursive pipeline: cache lookup (miss), iterative query to the victim’s authoritative nameservers, wait for NXDOMAIN response. Each in-flight query holds a slot in the recursive-clients table (default limit 1000, soft quota at 900). The victim’s authoritative server is also under attack, so responses may be slow or time out. Default resolver-query-timeout is 10 seconds. If the victim’s authoritative servers are unresponsive, a single failed query holds a recursive-client slot for the full window.

As slots fill, fewer are available for legitimate queries. At the soft quota (90% of limit), BIND begins rejecting new recursive queries. At the hard limit, every new recursive query receives SERVFAIL. The attack on one domain has now become a service-wide outage.

flowchart TD
    A[Random unique queries for victim.com subdomains] --> B[Every name is a cache miss]
    B --> C[Forced recursion to upstream NS]
    C --> D[RecursClients slots consumed]
    D --> E[NXDOMAIN rate and CPU spike]
    D --> F{recursive-clients near limit?}
    F -->|No| G[Degraded resolution for all clients]
    F -->|Yes| H[SERVFAIL for all recursive queries]

One detail that makes this attack particularly effective against BIND: clients-per-query and max-clients-per-query (BIND’s deduplication for concurrent identical recursive queries) provide no protection. These controls collapse multiple clients asking for the same name into a single upstream fetch. When every query name is unique, there is nothing to deduplicate.

Common causes

The first two are the classic pattern. The third is less common but mimics the same signal profile. In all cases, the damage path is identical: cache misses cascade into recursive-client exhaustion.

Quick checks

These commands assume BIND’s statistics channel is enabled on port 8653. Adjust the port to match your statistics-channels configuration. All commands are read-only.

# Check NXDOMAIN rate relative to other response codes
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  [print(f'{k}: {v}') for k,v in sorted(d.get('nsstats',{}).items()) if k.startswith('Qry')]"

# Check recursive clients as a gauge (absolute value, not a counter)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  print('RecursClients:', d.get('nsstats',{}).get('RecursClients', 'N/A'))"

# Check cache hit ratio per view
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  [print(f'{v}: Hits={cs.get(\"CacheHits\",0)} Misses={cs.get(\"CacheMisses\",0)} Ratio={cs.get(\"CacheHits\",0)/(cs.get(\"CacheHits\",0)+cs.get(\"CacheMisses\",1))*100:.1f}%') \
  for v,vd in d.get('views',{}).items() if (cs:=vd.get('resolver',{}).get('cachestats',{}))]"

# See what queries are in-flight and which upstream they are waiting on
rndc recursing | awk '{print $NF}' | sort | uniq -c | sort -rn | head -10

# Check RRL activity (only present if rate-limit is configured)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); ns=d.get('nsstats',{}); \
  print('RateDropped:', ns.get('RateDropped',0), 'RateSlipped:', ns.get('RateSlipped',0))"

# Check CPU utilization for named process
pidstat -p $(pidof named) 1 5

If RecursClients is above 900 (90% of the default 1000 limit), NXDOMAIN is dominating the response code distribution, and cache hit ratio has dropped well below baseline, you are under attack or experiencing a similar high-cardinality query pattern.

How to diagnose it

1. Identify the targeted domain. Dump the cache and analyze query name concentration:

   rndc dumpdb -cache
   # Dumps to the working directory (default: /var/named or as configured).
   # Can be memory and I/O intensive on large caches. Analyze the dump
   # for the most common parent domain.
   

   Alternatively, if query logging is available (enable it briefly; it degrades performance at high QPS), aggregate by parent domain:

   awk '{print $NF}' /var/log/named/queries.log | rev | cut -d. -f1-2 | rev | sort | uniq -c | sort -rn | head -10
   

2. Confirm the attack pattern. The signature is near-zero repetition among query names. Legitimate high-NXDOMAIN sources produce different distributions: Windows DNS suffix search lists generate predictable suffix-appended names, and DGA malware produces many different parent domains. Water torture concentrates randomness under one parent.

3. Assess collateral damage. Check whether RecursClients is approaching the limit and whether SERVFAIL is rising for unrelated domains. If both are true, the attack is already degrading resolution for all clients.

4. Check upstream victim status. Use rndc recursing to see whether in-flight queries are piling up against the victim’s authoritative nameservers. If the victim is also slow (because they are under the same attack), timeout duration amplifies slot consumption.

5. Check kernel UDP drops. The attack volume may exceed the kernel’s ability to buffer packets:

   cat /proc/net/snmp | grep Udp
   # Watch UdpRcvbufErrors for non-zero increments
   

   These drops are invisible to BIND. If present, your resolver is losing queries before it processes them.

Metrics and signals to monitor

Fixes

Apply RPZ to refuse the targeted domain

The fastest immediate mitigation. Configure a Response Policy Zone that returns NXDOMAIN for the victim domain before recursion is triggered. RPZ policy is evaluated early in the query pipeline, so the query never reaches the recursive fetch stage.

In the RPZ zone file, encode an NXDOMAIN policy with a CNAME . record for the target domain:

victim.com     CNAME .

Apply it with rndc reload of the RPZ zone. Confirm it is working by checking that RPZRewrites increases and NXDOMAIN responses for the targeted domain now come from policy rather than from upstream recursion.

Tradeoff: RPZ requires knowing the target domain. If the attacker shifts to a new domain, you must update the RPZ rule. RPZ datasets also consume memory, so factor that into capacity planning.

Enable rate-limit (RRL)

Response Rate Limiting throttles outgoing responses. RRL is built into BIND since version 9.9 and requires no compile flag. Configure it in named.conf:

rate-limit {
    responses-per-second 100;
};

The default responses-per-second is 0, meaning no limit. You must set it explicitly. The nxdomains-per-second option defaults to responses-per-second, so setting the parent value also limits NXDOMAIN responses.

RRL distinguishes between RateDropped (responses silently dropped) and RateSlipped (responses sent truncated, forcing TCP retry). Monitor both after enabling. Slipped responses cause TCP retry, which can create FD pressure if TCP capacity is also constrained.

Tradeoff: Overly aggressive RRL settings can throttle legitimate clients. Tune based on your normal traffic baseline.

Configure fetches-per-zone

fetches-per-zone caps the number of simultaneous iterative queries BIND will send for any single domain. This directly limits how many recursive slots a water torture attack can consume against the targeted domain.

options {
    fetches-per-zone 100;
};

The default is 0 (disabled). When the limit is exceeded, the default action is drop (the query is silently dropped). You can also specify fail to return SERVFAIL instead. This option was introduced in BIND 9.10.3.

Tradeoff: Setting this too low can affect legitimate high-traffic domains that genuinely generate many concurrent recursive queries. Test with your normal traffic patterns before applying in production.

Configure fetches-per-server

This limits simultaneous queries to any single upstream nameserver. The default is 0 (disabled). BIND can also adaptively adjust this quota downward based on upstream responsiveness via fetch-quota-params.

options {
    fetches-per-server 100;
};

The default action when the limit is exceeded is fail (returns SERVFAIL). If the victim’s authoritative servers are slow because they are also under attack, this limit prevents your resolver from piling up queries against them. But legitimate queries to those servers are also capped.

Flush cache for the targeted domain

If stuck cache entries are compounding the issue:

rndc flushtree victim.com

rndc flushtree clears cache entries for the specified domain and all its subdomains. It does not affect other cached data.

Do not rely on stale NXDOMAIN caching

Since BIND 9.18.0 (the change tracked as GL #3386, landing in 9.18.5 and 9.19.3), NXDOMAIN records are no longer retained past their normal negative cache TTL, even when stale answer serving is enabled.

This is a deliberate defense against memory exhaustion during water torture attacks. It also means you cannot count on stale NXDOMAIN entries to absorb repeat queries. The attack generates unique names regardless, so stale caching would not help even if it were available.

Prevention

• Pre-deploy RPZ infrastructure. Having RPZ configured and ready means you can add a refuse rule in seconds during an attack, without building the infrastructure under pressure.
• Pre-configure fetches-per-zone. Set a reasonable limit (for example, 100) before an attack occurs. The default of 0 means no protection.
• Monitor query name cardinality. This is a Level 4 maturity signal. Low entropy (repeated names) is normal. High entropy (random strings concentrated under one parent) indicates water torture. Sampling query logs periodically can detect this without the I/O cost of continuous query logging.
• Monitor NXDOMAIN as a ratio of total responses. A sustained spike above 3x baseline warrants investigation. The NXDOMAIN spike guide covers broader context on NXDOMAIN pattern analysis including DGA malware and Windows suffix search lists.
• Size recursive-clients appropriately. The default of 1000 is too low for busy resolvers. Increasing it without corresponding FD and memory capacity just delays the cliff. Scale it with available resources.
• Keep BIND patched. If using nxdomain-redirect as part of your mitigation strategy, ensure you are on BIND 9.18.24 or later to avoid CVE-2023-4408, which can cause named to crash with an assertion failure when nxdomain-redirect is enabled.

How Netdata helps

The attack produces a multi-signal signature that unfolds over seconds, not minutes. Per-second metric collection catches this before thresholds trip:

• NXDOMAIN rate correlation. Netdata surfaces QryNXDOMAIN alongside other response codes in real time. A sudden spike in NXDOMAIN share while NOERROR stays flat is the leading edge of the attack.
• Cache hit ratio trend. Per-second cache hit and miss collection shows the ratio collapse within seconds, not after a 5-minute polling interval.
• RecursClients as a gauge. Netdata collects RecursClients as an absolute value (not a counter), so you see the climb toward the limit in real time. Combined with the recursive-clients config value, the percentage-of-limit view is immediately available.
• Outbound query rate. The outbound-to-inbound ratio is the clearest signal that the cache has stopped providing value. Per-second resolution catches this before the recursive-client limit is reached.
• Anomaly detection. Netdata’s ML flags the sudden shift in response code distribution and cache behavior before explicit thresholds are crossed.

Related guides

• BIND cache eviction storms: DeleteLRU, an undersized max-cache-size, and the pressure spiral
• BIND cache hit ratio dropping: the leading edge of recursive pain
• BIND clients-per-query and max-clients-per-query: duplicate recursion for popular names
• BIND forwarding loops: recursion that never terminates and burns recursive slots
• How BIND actually works in production: a mental model for operators
• BIND lame delegations: ’lame server resolving’ and nameservers that are not authoritative
• BIND monitoring checklist: the signals every production resolver and authoritative server needs
• BIND monitoring maturity model: from survival to expert
• BIND ’no more recursive clients: quota reached’: the recursive-clients circuit breaker
• named not responding on port 53: total outage versus UDP-works-TCP-fails
• BIND resolver NumFetch per view: per-view recursive pressure in split-horizon setups
• BIND NXDOMAIN spike: DGA malware, water torture, and Windows suffix search lists