CoreDNS is answering queries with REFUSED and clients cannot resolve names. Unlike SERVFAIL, which means “I tried and failed,” REFUSED means “I declined to try.” REFUSED is almost never an upstream or network problem: CoreDNS itself is deciding not to answer, and it does that for exactly three reasons.
Two are configuration problems: no server block matches the query’s zone, or the acl plugin is rejecting the source. One is a capacity problem: the forward plugin’s max_concurrent limit is shedding load. Determine which of the three you are dealing with before touching the Corefile. Editing the Corefile to fix a capacity REFUSED, or scaling replicas to fix a config REFUSED, wastes the incident.
The metric that separates them is coredns_forward_max_concurrent_rejects_total. If it increments alongside your REFUSED rate, you have a capacity problem. If it stays flat, you have a configuration problem.
What this means
Every query is matched against the server blocks in the Corefile, then run through the matched block’s plugin chain. REFUSED can be produced at three points along that path:
- Zone matching. If no server block matches the query’s zone, CoreDNS refuses the query outright. The classic case is a Kubernetes Corefile that serves
cluster.localbut has no catch-all.block with aforwardplugin. Queries for any external name get REFUSED. - The acl plugin. A
blockrule rejects the query based on source network or request type and returns REFUSED. - The forward plugin. If
max_concurrentis configured and in-flight forwarded queries hit that limit, additional queries are rejected with REFUSED. This is deliberate backpressure, counted incoredns_forward_max_concurrent_rejects_total.
flowchart TD
Q[Client query] --> Z{Server block matches zone?}
Z -- no --> R1[REFUSED: missing catch-all]
Z -- yes --> A{acl plugin block rule matches?}
A -- yes --> R2[REFUSED: acl rejection]
A -- no --> F{forward max_concurrent reached?}
F -- yes --> R3[REFUSED: capacity reject, coredns_forward_max_concurrent_rejects_total increments]
F -- no --> P[Normal plugin chain processing]Causes 1 and 2 are static: they refuse the same queries every time, regardless of load. Cause 3 is dynamic: it appears under load and disappears when load drops. That behavioral difference is often enough to identify the cause from the alert context alone.
Common causes
| Cause | What it looks like | First thing to check |
|---|---|---|
| No server block matches the query zone | REFUSED for every query in a specific zone, constant rate, independent of load. In Kubernetes, external names fail while cluster.local works | Does the Corefile have a catch-all . block with forward? |
| acl plugin rejecting a source | REFUSED for queries from specific source networks or of specific types. Other clients resolve fine | acl block rules in the Corefile, and which client IPs are affected |
forward max_concurrent limit hit | REFUSED spikes correlate with query rate or upstream latency. Comes and goes with load | coredns_forward_max_concurrent_rejects_total incrementing |
Quick checks
All of these are read-only and safe to run during an incident.
# Check REFUSED responses, broken down by zone and plugin
curl -s http://localhost:9153/metrics | grep 'coredns_dns_responses_total' | grep 'REFUSED'
# The decisive check: is the forward concurrency limit rejecting queries?
curl -s http://localhost:9153/metrics | grep 'coredns_forward_max_concurrent_rejects_total'
# Current query rate, to see if REFUSED correlates with load
curl -s http://localhost:9153/metrics | grep '^coredns_dns_requests_total'
# Inspect the running Corefile (Kubernetes)
kubectl get cm -n kube-system coredns -o yaml
# Reproduce from a client: which names get REFUSED?
dig @<coredns-ip> kubernetes.default.svc.cluster.local +time=2 +tries=1
dig @<coredns-ip> example.com +time=2 +tries=1
Two things to look for. First, the zone and plugin labels on coredns_dns_responses_total{rcode="REFUSED"} tell you which server block and which plugin produced the refusal. Second, whether the REFUSED counter moves in lockstep with coredns_forward_max_concurrent_rejects_total.
How to diagnose it
Confirm the REFUSED rate and scope. Pull
coredns_dns_responses_total{rcode="REFUSED"}and note thezonelabel values. REFUSED concentrated in one zone (or in the catch-all.zone) points at zone matching or forward. REFUSED tied to a specific client population points at acl.Check the concurrency reject counter. If
coredns_forward_max_concurrent_rejects_totalis incrementing, stop here: you have a capacity REFUSED. Skip to the capacity fix. Do not edit zone or acl configuration; the config is doing exactly what it was told to do.Check the Corefile for zone coverage. If the reject counter is flat, read the Corefile. Does a server block exist for the zone in the REFUSED metric’s
zonelabel? In Kubernetes, the common breakage is a Corefile with acluster.localblock but no.block, so every external name is refused. Also check for reload drift: ifcoredns_reload_failed_totalis nonzero, the running config may not be the one you are reading. CoreDNS keeps the old config on a failed reload, so the ConfigMap can look correct while the process runs something older.Check for acl rules. If zones are covered, look for an
aclplugin in the matched server block. Ablockrule matching the client’s source network returns REFUSED. Note that the plugin’s default action isallow: deny-by-default only exists if someone added an explicit catch-allblockrule. To identify refused source IPs, enable thelogplugin temporarily if it is not already on; CoreDNS exposes no per-client metrics. A config change triggers a CoreDNS reload or rolling restart, so expect a brief resolution blip when you apply it.Correlate with load for ambiguous cases. If REFUSED appears only during traffic spikes and the reject counter moves, it is capacity even if the absolute rate is low. If REFUSED is flat 24/7, it is config.
Metrics and signals to monitor
| Signal | Why it matters | Warning sign |
|---|---|---|
coredns_dns_responses_total{rcode="REFUSED"} | The pain signal. Rate and zone/plugin labels scope the cause | Any sustained nonzero rate during normal operations |
coredns_forward_max_concurrent_rejects_total | Separates capacity REFUSED from config REFUSED | Any nonzero sustained rate |
coredns_dns_requests_total | Load context. REFUSED that tracks query rate is capacity; flat REFUSED is config | REFUSED rising in lockstep with QPS |
coredns_forward_request_duration_seconds{to=...} | Slow upstreams cause in-flight queries to accumulate against max_concurrent | P99 upstream latency climbing before REFUSED spikes |
go_goroutines | Blocked forward queries accumulate as goroutines; a leading indicator for concurrency exhaustion | Count growing faster than QPS justifies |
coredns_reload_failed_total | A failed reload means the running Corefile is not the one you are reading | Any nonzero value |
Fixes
Missing catch-all zone
Add or repair the catch-all server block so external names have somewhere to go:
. {
forward . /etc/resolv.conf
# ... cache, errors, health, etc.
}
In Kubernetes, /etc/resolv.conf is only safe as a forward target if the node’s resolver points at a real upstream and not back at the cluster DNS Service IP. If it points back at CoreDNS, the loop plugin will detect the forwarding loop and crash the pod at startup: you trade REFUSED for CrashLoopBackOff. Verify the node’s resolv.conf before applying.
acl plugin rejecting legitimate clients
Adjust the acl rules so legitimate source networks are allowed before any broad block rule. Rule order matters: put specific allow rules for known-good networks ahead of wider blocks. If you intended deny-by-default, confirm the explicit catch-all block rule exists and the allow list is complete; the default action is allow, so a missing final block rule silently permits everything.
Test from an affected source after the change: dig @<coredns-ip> example.com from a client in the previously refused network should return NOERROR.
forward max_concurrent capacity rejects
This is the one cause where the Corefile edit is the second step, not the first. max_concurrent exists to protect the process from unbounded goroutine growth when upstreams are slow. Rejected queries are a symptom of backpressure, with two sub-cases:
- Upstream latency is the root cause. In-flight queries accumulate because responses are slow. Check per-upstream latency via the
tolabel on forward request duration. Fix the slow upstream (replace it, remove it, or investigate the network path) and the rejects stop. Raisingmax_concurrentwithout fixing the upstream delays the same cliff while consuming more memory: each concurrent query holds a goroutine and its buffers. - Query volume genuinely exceeds the limit. If upstreams are healthy and fast, raise
max_concurrentto at least your peak forwarded query rate multiplied by upstream latency, with headroom for bursts (3x is a reasonable starting margin). Ifmax_concurrentis not set at all, it defaults to unlimited and this cause cannot apply; recheck your diagnosis.
If you raised the limit, watch go_goroutines and heap afterwards. You traded fast rejection for resource consumption, and that trade needs monitoring.
Prevention
- Alert on the split, not just the symptom. Alert on any sustained REFUSED rate, and separately on any nonzero
coredns_forward_max_concurrent_rejects_total. The second alert is unambiguous: it is always a capacity or upstream problem. - Watch upstream latency as a leading indicator. Rising per-upstream P99 precedes concurrency rejects. Catching a degrading upstream early prevents the REFUSED spike entirely.
- Validate Corefile changes against zone coverage. Check every Corefile change for a catch-all block before rollout, and watch
coredns_reload_failed_totalafter every ConfigMap update so silent reload drift does not mask your intent. - Size max_concurrent deliberately. If you set it, derive it from measured peak forward rate and upstream latency rather than copying a default. Revisit it as cluster traffic grows.
- Keep acl rules explicit and ordered. Document which source ranges must resolve, and review allow lists when network topology or pod CIDRs change.
How Netdata helps
- Netdata charts
coredns_dns_responses_totalby RCODE, so REFUSED is visible as its own line instead of being buried in an aggregate error count. - The
zoneandpluginlabel breakdowns show which server block is refusing, narrowing the cause before you open the Corefile. coredns_forward_max_concurrent_rejects_totalis charted alongside forward latency and query rate, making the config-vs-capacity split a glance rather than an investigation.- Per-upstream forward latency (
tolabel) is correlated on the same dashboard, so a slow upstream driving concurrency rejects shows up in the same view as the rejects. - Goroutine and Go memory charts expose in-flight query accumulation before the concurrency limit starts shedding load.
- Anomaly detection on the REFUSED rate catches low-level, persistent config refusals that sit below absolute thresholds but represent broken clients.






