You are looking at coredns_dns_requests_total and the type="ANY" series has jumped from a flat zero to a sustained rate, or ANY queries have crept past a few percent of total traffic. That pattern is the classic fingerprint of DNS amplification and reflection abuse: an attacker sends small ANY queries with a spoofed source address, and your CoreDNS replies with large responses delivered to the victim’s IP. Your server is the reflector; someone else absorbs the blast.

This is not always an attack. Some debug tooling and monitoring scripts use ANY queries legitimately at low volume, and RFC 8482 acknowledges debugging as a valid use. The operational question is not “is ANY nonzero” but “did ANY change shape”: a spike from a zero baseline, or ANY crossing roughly 5% of total traffic, is unusual and worth investigating.

What this means

An ANY query asks the server for all records it holds for a name. A normal A or AAAA answer is small. An ANY answer can contain every record type for the name at once, which makes the response far larger than the query. That asymmetry is the whole attack:

  1. The attacker crafts an ANY query with the source IP set to the victim’s address (UDP makes source spoofing trivial on networks without egress filtering).
  2. CoreDNS receives a small query and sends a large response to the spoofed source.
  3. Thousands of open resolvers doing this at once produce a volumetric DDoS against the victim, and your egress bandwidth, conntrack table, and UDP buffers pay part of the cost.
flowchart LR
    A[Attacker] -->|"small ANY query, spoofed source = victim IP"| B[CoreDNS]
    B -->|"large ANY response"| V[Victim IP]
    B -.->|"also burns your egress, UDP buffers, conntrack"| N[Your node]

RFC 8482 exists precisely because of this. It deprecates the expectation that ANY returns everything and blesses minimal responses. CoreDNS behavior depends on which plugins you have configured: without anything specific in place, ANY answers come from whatever plugin serves the zone, at full size.

Two measurement realities matter for diagnosis. First, CoreDNS exposes no per-source-IP metrics, so identifying who is sending the flood requires log analysis or dnstap, not Prometheus. Second, UDP packets dropped by the kernel before reaching CoreDNS are invisible in CoreDNS metrics, so a severe flood can make your query rate look lower than reality while the node drowns. See the hub’s failure pattern catalogue for the UDP buffer cliff and conntrack exhaustion patterns.

Common causes

CauseWhat it looks likeFirst thing to check
Active reflection attackANY rate spiking from zero baseline, response size distribution shifting upward, egress saturatedcoredns_dns_requests_total{type="ANY"} rate and coredns_dns_response_size_bytes histogram
Reconnaissance or scanningModerate ANY rate against many names, possibly mixed with AXFR attemptscoredns_dns_requests_total{type="AXFR"} alongside ANY
Legitimate debug toolingLow, steady ANY baseline from one or two internal sourcesEnable the log plugin temporarily and aggregate by source IP
Misbehaving internal clientANY queries from a single pod, often a monitoring script or resolver testQuery logs grouped by source IP
CoreDNS reachable from untrusted networksANY flood combined with total QPS far above expected internal demandWhere the service is exposed: ClusterIP only, or node port / LoadBalancer / host port

Quick checks

All of these are read-only.

# ANY query rate and share of total traffic
curl -s http://localhost:9153/metrics | grep 'coredns_dns_requests_total' | grep 'type="ANY"'

# Response size distribution: is the histogram shifting upward?
curl -s http://localhost:9153/metrics | grep 'coredns_dns_response_size_bytes'

# AXFR attempts often travel with ANY reconnaissance
curl -s http://localhost:9153/metrics | grep 'type="AXFR"'

# Total query rate for context (is overall QPS also abnormal?)
curl -s http://localhost:9153/metrics | grep '^coredns_dns_requests_total'

Then check what the node sees, because CoreDNS metrics cannot:

# UDP buffer errors: packets dropped before CoreDNS ever counted them
netstat -su | grep -i "buffer errors"
cat /proc/net/snmp | grep Udp

# Conntrack pressure on the node (Kubernetes with iptables DNAT)
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max

Finally, see what an ANY response from your server actually looks like. This tells you the amplification factor you are offering:

# Compare ANY response size to a plain A response
dig @<coredns_ip> <a_name_you_serve> ANY +noall +answer
dig @<coredns_ip> <a_name_you_serve> A +noall +answer

If the ANY answer comes back with a full record set, your server is a good reflector. If it comes back with a single short HINFO record, someone already deployed the fix described below.

How to diagnose it

  1. Confirm the ANY share. Compute ANY as a percentage of total coredns_dns_requests_total. Under 5% and steady is plausibly baseline tooling. A spike from zero, or a sustained rate well above 5%, moves you to step 2.

  2. Correlate with response size. Check coredns_dns_response_size_bytes. A flood of ANY queries producing large answers shifts the histogram’s upper buckets. If ANY is up but response sizes are flat and small, your ANY responses are already minimal (or the queries are failing), and the reflection risk is low even though the query pattern is suspicious.

  3. Identify the source. Prometheus cannot help here. Temporarily enable the log plugin and aggregate by client IP. The log plugin’s combined format puts the client address in the second field, as IP:port:

    # Group queries by source IP (requires the log plugin)
kubectl logs -n kube-system <coredns-pod> | awk '{print $2}' | cut -d: -f1 | sort | uniq -c | sort -rn | head -20

    Many spoofed sources, or internet-routable sources you do not recognize, point to abuse. One internal IP points to a misbehaving or compromised workload. Note the tradeoff: the log plugin adds real overhead at high QPS, so enable it for the investigation and remove it after.

  4. Check exposure. In a default Kubernetes deployment, CoreDNS sits behind a ClusterIP and is only reachable inside the cluster. If your instance answers queries from untrusted networks (host port, node port, LoadBalancer, or a standalone deployment on a public interface), the attack surface is much larger and source spoofing is much easier.

  5. Check collateral damage. During a real flood, the first thing to break may not be CoreDNS itself. Look at netstat -su receive buffer errors and conntrack utilization from the quick checks. If those are climbing, the node is shedding packets for every workload on it, not just DNS.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
coredns_dns_requests_total{type="ANY"}The primary abuse indicatorANY > 5% of total traffic, or a spike from a zero baseline
coredns_dns_response_size_bytesConfirms whether ANY queries produce large (amplifying) responsesSustained shift in the distribution, or responses above 4096 bytes
coredns_dns_requests_total{type="AXFR"}Zone transfer attempts often accompany ANY reconnaissanceAny nonzero AXFR where transfers are not expected
Total coredns_dns_requests_totalSeparates an ANY-specific event from a general QPS surgeRate above 2-3x the rolling baseline with no known cause
Node UDP buffer errors (/proc/net/snmp, RcvbufErrors)Reveals floods the CoreDNS process never seesAny nonzero, incrementing count
Node conntrack usageUDP DNS flows consume conntrack entries; floods exhaust the tableUsage above 80% of nf_conntrack_max

Fixes

Reduce the amplification factor with the any plugin

Since CoreDNS 1.5.1, the built-in any plugin answers ANY queries with a minimal HINFO response per RFC 8482 instead of the full record set. Adding it to the Corefile is a one-word change in the server block:

. {
    any
    # ... rest of your plugin chain
}

The response becomes a short synthesized record ("ANY obsoleted" "See RFC 8482"), which collapses the amplification factor to roughly 1:1. Your server stops being an attractive reflector.

Tradeoffs to understand:

  • This is not a rate limiter. The attacker can still send the queries; you just stop multiplying their effect. Query volume still costs you CPU, sockets, and conntrack entries.
  • Legitimate dig ANY debugging against your server now returns the HINFO stub, not real data. That is the RFC-sanctioned behavior, but expect confused colleagues.
  • Many Kubernetes distributions ship a default Corefile without any. Check yours before assuming the protection is in place.

Rate limit with the rrl plugin

For volumetric mitigation, the rrl plugin provides BIND-style response rate limiting: it tracks response rates per category and can drop or truncate responses that exceed limits.

Tradeoffs:

  • rrl is an external plugin. It is not in standard CoreDNS builds, so you must compile a custom CoreDNS image with it added to plugin.cfg. For teams running the stock Kubernetes CoreDNS image, this is a real operational burden.
  • The rrl README documents a known wildcard-flooding weakness: an attacker spreading queries across unlimited unique names synthesized by a wildcard record keeps per-name rates under the limits. Mitigating this requires the metadata plugin and a minimum CoreDNS version the README leaves as TBD.
  • Rate limiting that is too aggressive will clip legitimate clients during bursts. Start permissive and tighten against observed baselines.

Remove the exposure

If CoreDNS is reachable from untrusted networks and does not need to be, fixing that beats every in-process mitigation:

  • Keep cluster DNS behind the ClusterIP; do not publish it via node port, host port, or LoadBalancer.
  • For standalone deployments, bind to internal interfaces only and filter port 53 at the network edge for untrusted sources.
  • Egress filtering (BCP 38) on your own network prevents your hosts from being the source of spoofed packets, which is the neighborly half of the same problem.

Survive the active flood

While a flood is ongoing, the risk is that the node fails before CoreDNS does. Watch UDP receive buffer errors and conntrack utilization, and be prepared to raise net.core.rmem_max and nf_conntrack_max as stopgaps. Both are covered in depth in the related guides below. Do not restart CoreDNS as a first response: it does nothing to stop spoofed traffic and adds a cold-cache thundering herd on top of the attack.

Prevention

  • Baseline the ANY share. Record your normal ANY percentage so “spike from zero” and “>5%” are detectable, not vibes.
  • Ship any in the Corefile by default. It is cheap, built in, and removes the reflection value of your server.
  • Alert on the pair, not the point. ANY rate up plus response size distribution shifting is a much stronger signal than either alone, and it filters out low-volume legitimate debugging.
  • Audit exposure after infrastructure changes. New load balancers, node ports, and firewall rules are how internal resolvers accidentally become public reflectors.
  • Monitor node-level signals. UDP buffer errors and conntrack utilization are where a flood actually hurts first, and neither appears in CoreDNS metrics.

How Netdata helps

  • Netdata charts coredns_dns_requests_total broken out by query type, so a rising type="ANY" series is visible next to A and AAAA traffic without writing PromQL during an incident.
  • The coredns_dns_response_size_bytes distribution is charted alongside query rates, making the “ANY up plus responses getting bigger” correlation a single-screen check.
  • Per-second collection catches short flood bursts that minute-resolution scraping averages into invisibility.
  • Node-level metrics (UDP errors, conntrack usage) are collected from the same hosts, so you can correlate the DNS-layer symptom with the kernel-layer damage in one view.
  • Baseline deviation on the ANY share is the kind of pattern ML anomaly detection flags well, because the normal value is a flat near-zero line.