BIND NXDOMAIN spike: DGA malware, water torture, and Windows suffix search lists

The QryNXDOMAIN counter jumping above 3x baseline is a common alert trigger, but the raw rate tells you almost nothing. Two resolvers with identical NXDOMAIN rates can be in completely different states: one healthy, one under active attack.

The signal that matters is query-name cardinality and entropy. If the same names repeat, the spike is benign (Windows suffix search lists, new client rollouts). If nearly every query name is unique and random, you are looking at a water torture attack or DGA malware beaconing. This distinction determines whether you page someone at 3 a.m. or close the alert.

What this means

NXDOMAIN is a legitimate DNS response code meaning the queried name does not exist in the DNS hierarchy. Resolvers produce NXDOMAIN responses constantly: mistyped URLs, applications probing nonexistent configuration endpoints, Windows clients resolving through suffix search lists.

The problem is a sudden, sustained increase above 3x baseline. The raw rate is noise; query-name cardinality (how many unique names) and entropy (how random they look) are the signals that separate benign from malicious.

Common causes

Quick checks

These commands are safe and read-only. Run them on the resolver showing the NXDOMAIN spike. Replace the statistics channel port (8653 here) with whatever is configured in your named.conf.

# Check the QryNXDOMAIN counter alongside other response codes
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  ns=d.get('nsstats',{}); \
  [print(f'{k}: {v}') for k,v in sorted(ns.items()) if k.startswith('Qry')]"

# Check if RPZ is synthesizing NXDOMAIN (RPZRewrites increasing)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  print('RPZRewrites:', d.get('nsstats',{}).get('RPZRewrites',0))"

# Check recursive client pressure (are NXDOMAIN queries consuming slots?)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  print('RecursClients:', d.get('nsstats',{}).get('RecursClients','N/A'))"

# Check cache hit ratio (water torture drives it down: each unique name is a miss)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  [print(f'{v}: Hits={cs.get(\"CacheHits\",0)} Misses={cs.get(\"CacheMisses\",0)} Ratio={cs.get(\"CacheHits\",0)/(cs.get(\"CacheHits\",0)+cs.get(\"CacheMisses\",1))*100:.1f}%') \
  for v,vd in d.get('views',{}).items() if (cs:=vd.get('resolver',{}).get('cachestats',{}))]"
# Note: the walrus operator (:=) requires Python 3.8+.

# Dump the cache to inspect query-name patterns
rndc dumpdb -cache
# Output goes to the 'dump-file' path from named.conf (commonly
# /var/named/data/named_dump.db or /var/named/named_dump.db).

# Check in-flight recursive queries (what upstream is BIND waiting on?)
rndc recursing | head -20

# Check RRL activity if rate-limit is configured
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); ns=d.get('nsstats',{}); \
  print('RateDropped:', ns.get('RateDropped',0), 'RateSlipped:', ns.get('RateSlipped',0))"

# Check query type distribution (ANY spikes suggest amplification, not NXDOMAIN attack)
curl -s http://localhost:8653/json/v1/server | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  print(dict(sorted(d.get('qtypes',{}).items(), key=lambda x:-x[1])[:10]))"

How to diagnose it

Step 1: Confirm the spike is real. BIND’s statistics channel counters are cumulative since process start. A single snapshot is meaningless. Take two samples divided by the interval to compute the rate, then compare against your rolling baseline at the same time of day.

Step 2: Check RPZ first. If RPZ is configured, check whether RPZRewrites is increasing at the same rate as the NXDOMAIN spike. Some RPZ policy actions synthesize NXDOMAIN via the nxdomain policy action. If RPZ is the source, the spike is expected behavior from your security policy.

Step 3: Assess query-name cardinality. This is the most important step. Determine whether the NXDOMAIN queries target many unique names (high cardinality, near-zero repetition) or a small set of repeating names (low cardinality, high repetition). High cardinality points to attack. Low cardinality points to benign causes.

Use rndc dumpdb -cache to dump the cache, or if query logging is already enabled, sample the query log to find the dominant queried domains:

# WARNING: enabling query logging impacts performance at high QPS.
# If already enabled, sample the dominant queried domains.
# <!-- TODO: verify this extraction matches your BIND querylog format, which varies by version. -->
grep ': query:' /var/log/named/queries.log | \
  grep -oP '\([^)]+\)' | tr -d '()' | rev | cut -d. -f1-2 | rev | \
  sort | uniq -c | sort -rn | head -10

flowchart TD
    A["NXDOMAIN rate >3x baseline"] --> B{"RPZRewrites
increasing?"}
    B -- "Yes" --> C["RPZ policy action
(expected behavior)"]
    B -- "No" --> D{"Query-name
repetition high?"}
    D -- "Yes, names repeat" --> E["Benign:
Windows suffix list
or new client rollout"]
    D -- "No, unique names" --> F{"One parent
domain?"}
    F -- "Yes" --> G["Water torture
attack"]
    F -- "No, many domains" --> H["DGA malware
beaconing"]

Step 4: Determine the pattern. Based on the cardinality analysis:

• One parent domain, many unique subdomains: water torture attack against that domain. The resolver is being used as an amplifier to flood the victim’s authoritative server.
• Many random-looking domains, no parent concentration: DGA malware beaconing. An internal host is infected and trying to reach algorithmically generated C2 domains.
• Repeating names, correlated with Windows subnets: benign suffix search list behavior. No action needed beyond alert tuning.
• Repeating names, correlated with deployment timeline: new client rollout generating service discovery probes or internal suffix lookups.

Step 5: Identify the source. For water torture, the source is often external (botnet traffic hitting your resolver). For DGA, the source is typically an internal host infected with malware. Query log analysis or packet capture will reveal the source IP.

Step 6: Check downstream impact. If RecursClients is climbing and cache hit ratio is dropping, the NXDOMAIN spike is consuming resolver resources. Water torture attacks are particularly effective at this: each unique query is a cache miss that forces recursion, consuming a recursive-client slot for the full timeout duration. The default recursive-clients limit is 1000, with a soft quota at 900 (90%). Once the hard limit is reached, new recursive queries fail (SERVFAIL or timeout), affecting all resolution, not just the targeted domain.

Metrics and signals to monitor

Fixes

Water torture attack mitigation

Water torture attacks target one parent domain with random subdomains, using your resolver as an amplifier to flood the victim’s authoritative server. Each unique query is a cache miss consuming CPU and recursive-client slots.

1. Identify the target domain from the cache dump or query log analysis.
2. Apply RPZ to refuse or synthesize NXDOMAIN locally for the targeted domain. This stops BIND from forwarding queries upstream: the resolver answers NXDOMAIN from RPZ without consuming recursive-client slots or generating outbound traffic.
3. Enable RRL if not already configured. Response Rate Limiting can drop or slip (truncate) responses to abusive query rates. The nxdomains-per-second option targets NXDOMAIN response rates per client prefix. ISC recommends RRL primarily for authoritative servers; on recursive resolvers it risks false positives. Evaluate carefully for your deployment.

DGA malware remediation

DGA malware generates algorithmic domain names and queries them to contact C2 servers. The resolver sees NXDOMAIN for most queries because only a few generated names per cycle resolve to the active C2 server.

1. Identify the infected host from source IP analysis. DGA traffic originates inside your network.
2. Isolate the host and run malware remediation.
3. Deploy RPZ with threat intelligence feeds that include known DGA domains. RPZ can synthesize NXDOMAIN for known-bad domains, blocking the C2 lookup before it reaches upstream.
4. Monitor RPZRewrites as an early warning. A spike in RPZ rewrites can indicate a malware outbreak before anyone reports a problem.

Windows suffix search list noise

Benign behavior, not an attack. Windows clients with a DNS suffix search list generate NXDOMAIN for each non-matching suffix before finding the correct one. A 4-entry suffix list produces approximately 3 NXDOMAIN per successful lookup.

• Do not try to eliminate this behavior. It is correct Windows DNS client behavior defined by the suffix search list configuration.
• Adjust alerting to account for the expected NXDOMAIN floor from Windows subnets. Alert on deviation from baseline, not on absolute NXDOMAIN rate.
• If the suffix list is unnecessarily long, work with the endpoint team to trim it. Fewer suffixes means fewer NXDOMAIN queries per lookup.

RPZ-synthesized NXDOMAIN

If RPZ rules are the source of the NXDOMAIN spike, this is your security policy working as intended. The spike may indicate a malware outbreak (RPZ blocking C2 domains) or a newly loaded RPZ feed with aggressive rules.

1. Check RPZRewrites to confirm the correlation with the NXDOMAIN spike.
2. Investigate which RPZ rules are firing to understand the threat pattern.
3. If a new RPZ feed was recently added, the spike may be the feed catching up with accumulated threats. This should settle as the feed reaches steady state.

Prevention

• Monitor query-name entropy, not just NXDOMAIN rate. Track the ratio of unique query names to total queries over time. High cardinality with near-zero repetition is the signature of both water torture and DGA activity.
• Configure RPZ with threat intelligence feeds to intercept known DGA domains before they generate upstream traffic.
• Set up RRL on authoritative servers with nxdomains-per-second limits. Evaluate carefully for recursive resolvers due to false-positive risk.
• Baseline your NXDOMAIN rate per client subnet. Windows-heavy subnets have a higher NXDOMAIN floor. Separate alerting thresholds per subnet avoids false positives from expected behavior.
• Watch for Chrome startup bursts. Chrome generates 3 random single-label DNS queries at startup to detect NXDOMAIN substitution by ISPs. In environments with mass logins (shift start, morning VDI boot), this can produce a brief NXDOMAIN burst that resolves on its own.

How Netdata helps

• Netdata collects QryNXDOMAIN and all Qry* counters from the BIND statistics channel at per-second resolution, surfacing baseline deviations without manual counter arithmetic.
• Anomaly detection flags shifts in NXDOMAIN rate relative to the learned baseline, reducing false alerts from Windows suffix list noise that stays within expected patterns.
• Correlating QryNXDOMAIN with CacheMisses and RecursClients on a single dashboard shows whether a spike is consuming resolver resources or being absorbed by cache and policy.
• RPZRewrites and RateDropped/RateSlipped are collected alongside NXDOMAIN counters, letting you distinguish RPZ-synthesized NXDOMAIN from organic NXDOMAIN without running separate commands.
• Per-view statistics collection lets split-horizon deployments isolate spikes to specific client populations.

Related guides

• How BIND actually works in production: a mental model for operators
• BIND monitoring maturity model: from survival to expert
• BIND SERVFAIL responses: what a DNS SERVFAIL actually means and how to trace the cause