You find it in the Traefik log, usually right after a deploy, a restart storm, or a cluster rebuild: an ACME order rejected with an HTTP 429 and the message “too many certificates already issued” for your domain. Traefik cannot get a new certificate, and it will keep retrying and keep getting rejected.

Two very different situations sit behind this error. If your existing certificates are still valid and stored in acme.json, traffic keeps flowing and you have days or weeks to fix the renewal path before expiry. If acme.json was lost along with the certificates, you are rate limited and serving nothing valid on HTTPS, which is an outage on a timer.

The quota is almost never consumed by organic growth. It is consumed by your own infrastructure repeating orders: restarts without persistent storage, crash loops, multiple Traefik instances renewing the same names independently, or CI/CD spinning up new subdomains. Finding the repeater is the actual fix. Everything else is waiting.

What this means

Let’s Encrypt rejected the order with rateLimited (HTTP 429). Traefik has no Prometheus metric for ACME failures, so the log line is the primary evidence and traefik_tls_certs_not_after is the consequence you monitor. The limits that matter for this error, per the Let’s Encrypt rate limits documentation:

LimitValueWindowWhat typically trips it
New certificates per registered domain507 daysSubdomain sprawl, multiple instances each reissuing
New certificates per exact set of identifiers57 daysCrash loops, lost acme.json, repeated reinstalls of the same host
New orders per account3003 hoursMass configuration churn across many domains

These budgets refill continuously (token bucket) rather than resetting on a fixed weekly boundary. The exact-set limit refills at roughly one certificate per 34 hours; the registered-domain limit at roughly one per 202 minutes. Three operational consequences:

  • Existing certificates keep working for their full lifetime. The rate limit blocks new issuance, not validation of what you already serve.
  • Revoking certificates does not reset rate limits. Do not revoke in the hope of unlocking issuance.
  • The per-registered-domain limit is global across all ACME accounts. Creating a fresh account does not bypass it.

One nuance on renewals: Let’s Encrypt exempts proper renewals from some limits, but renewals without ARI (ACME Renewal Info) still count against the exact-set limit.

flowchart TD
  A[Restarts without persisted acme.json
or multiple instances renewing] --> B[Burst of new certificate orders] B --> C{Let's Encrypt quota} C -->|under limit| D[Certificate issued] C -->|limit exceeded| E[HTTP 429 rateLimited] E --> F[Issuance and renewals blocked] F --> G[Existing certs keep serving until expiry] G --> H[traefik_tls_certs_not_after approaches now] H --> I[Clients see TLS errors]

Common causes

CauseWhat it looks likeFirst thing to check
acme.json not persisted across restartsNew certificates issued on every container start; issuance timestamps cluster at deploy timesDoes acme.json survive a restart (volume mount)?
Crash-looping TraefikDozens of duplicate certificates for the same exact names within hoursContainer restart count
Multiple Traefik instances, each with its own acme.jsonSame domains renewed independently per replica; duplicates in certificate transparency logsHow many replicas run the same certificates resolver
Rapid subdomain creation (per-branch previews, CI/CD)Many distinct certificates under one registered domain in daysCount distinct names issued this week
Renewals for removed routesTraefik renews certificates for domains no longer routedCompare acme.json domains against active routers
Corrupted acme.jsonTraefik hard-fails at startup, or silently loses track of issued certs and re-requestsFile integrity, permissions, recent restores

Quick checks

All read-only. Adjust paths and label selectors to your deployment.

# 1. Find the rate limit errors and which limit class was hit
docker logs traefik 2>&1 | grep -iE "rate ?limit|too many certificates"
# or: journalctl -u traefik | grep -i "too many certificates"
# or: kubectl logs -n <ns> deploy/traefik | grep -i "too many certificates"

# 2. Check expiry of the certificate actually being served
echo | openssl s_client -servername app.example.com -connect app.example.com:443 2>/dev/null \
  | openssl x509 -noout -dates -issuer

# 3. Check the expiry metric for all stored certificates (Traefik v2.5+)
curl -s http://localhost:8080/metrics | grep traefik_tls_certs_not_after

# 4. Count stored certificates per domain in acme.json
jq -r '.[] | .Certificates[]?.domain.main' /path/to/acme.json | sort | uniq -c | sort -rn

# 5. Verify acme.json permissions (Traefik requires 600)
stat -c '%a %U %n' /path/to/acme.json

# 6. Check whether acme.json is on a durable mount
docker inspect traefik | jq '.[0].Mounts'

# 7. Check the restart count (crash loops burn duplicate-cert quota)
docker inspect -f '{{.RestartCount}}' traefik
# or: kubectl get pods -n <ns> -l app.kubernetes.io/name=traefik

# 8. Count recent issuance from the outside via certificate transparency logs
curl -s "https://crt.sh/?q=%25.example.com&output=json" | jq -r '.[].not_before' | sort | tail -20

Check 8 gives you the ground truth Let’s Encrypt sees: every certificate ever issued for your domain, with timestamps. Group by day and by name to see the burn pattern.

How to diagnose it

  1. Identify which limit you hit. The error text names the limit class: “exact set of domains” (the 5-per-week duplicate limit) or the registered domain (the 50-per-week limit). The exact-set limit points at repeated reissuance of one name; the registered-domain limit points at volume across many names.

  2. Reconstruct the issuance history. From the CT log output in check 8, count certificates per day for the last 7 days and group duplicates by common name. A flat line of one cert every container restart is a persistence problem. A sawtooth across many new names is subdomain sprawl.

  3. Correlate issuance timestamps with restarts. Line the CT timestamps up against deploy events and RestartCount from check 7. If every restart produced a new order, acme.json is not surviving.

  4. Verify persistence directly. Check 6 should show acme.json (or its parent directory) on a named volume or durable host path. Community reports indicate bind-mounting the single file is less reliable than mounting the whole directory, particularly around atomic file replacement. Permissions must be 600 or Traefik refuses to use the file.

  5. Count ACME writers. List every running Traefik instance using the same certificates resolver. Each one keeps its own acme.json and renews on its own schedule, multiplying consumption by replica count.

  6. Check your time budget. From check 3, find the smallest traefik_tls_certs_not_after among affected names. More than 14 days: you can wait for quota refill and fix calmly. Under 7 days, or certificates already lost: you need a stop-gap from the fixes below, this week.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
traefik_tls_certs_not_afterOnly built-in signal that renewal is failing; there is no ACME error metricAny active cert under 30 days un-renewed; under 14 days means renewal has been failing for over two weeks
Weekly issuance count (CT logs or log aggregation)Measures quota consumption directlyMore than 25 certificates per registered domain per week (50% of the limit)
process_start_time_seconds / restart countCrash loops convert directly into duplicate-cert ordersAny restart followed within minutes by a new issuance
traefik_config_reloads_total rateConfig churn adds and removes domains, which triggers ordersReload storms coinciding with new hostnames
Traefik log ACME linesThe only place 429s, challenge failures, and storage errors appearAny rateLimited, too many certificates, or “Error renewing certificate” line

Fixes

Stop the quota burn first

Nothing below helps while orders are still firing. Fix persistence, stop the crash loop, or scale down to one ACME-owning instance before anything else. Every additional order attempt while at the limit just confirms you are still at the limit.

Restore persistent ACME storage

Put acme.json (or the directory holding it) on a named volume, set permissions to 600, and restart once. Verify the file survives a second restart before declaring victory. Tradeoff: the first start after fixing storage may still 429 if you are inside the window, because Traefik must re-request what it lost. That is expected. It will succeed when budget refills.

Do not delete acme.json and restart to “start clean” while rate limited. That converts a renewal problem into a full re-issuance problem against an empty quota.

Consolidate to a single ACME owner

In current Traefik versions the acme.json file cannot be shared across instances, so each replica renews independently and burns quota multiplicatively. (Older v1-era deployments coordinated via a KV store; that mechanism was removed.) Practical patterns:

  • Run one replica responsible for the certificates resolver, and distribute the resulting certificates to other TLS terminators through your own mechanism.
  • Terminate TLS at a single layer (one Traefik, or a load balancer in front) instead of at every replica.
  • If you run multiple independent ingresses for isolation, give each its own registered domain where possible so quotas do not overlap.

Wait out the window, when certs are still valid

If acme.json is intact and the served certificates have time left, the correct fix is patience plus monitoring. The exact-set budget refills at roughly one per 34 hours, the registered-domain budget at roughly one per 202 minutes. Traefik retries renewal on its own; watch traefik_tls_certs_not_after jump back to a fresh 90-day value as confirmation. Do not revoke anything: revocation does not reset the limits, and it destroys a certificate that was still buying you time.

Stop-gaps when you are blocked with no valid certificate

  • Restore acme.json from backup. This is the fastest path back and the reason backups of this file matter.
  • Deploy a manually obtained certificate through Traefik’s dynamic TLS configuration as a bridge until ACME issuance recovers.
  • Point caServer at a different ACME-compatible CA temporarily. The per-domain Let’s Encrypt limit does not follow you to another CA, but the trust chain and account setup differ, so treat this as a measured change, not a panic move.
  • Do not use the staging endpoint as a production fix. Staging certificates are issued by an untrusted test CA; browsers will reject them. Staging is for testing only.

Reduce future consumption

Traefik renews certificates for domains it still has configured, even ones no longer routed or used. Remove stale domains and routers from your configuration so renewals stop. Where you control many subdomains under one registered domain, a wildcard certificate collapses N certificates into one. Note that wildcards require the DNS-01 challenge, so your DNS provider integration becomes part of the renewal path.

Prevention

  • Quota headroom target. Keep weekly issuance under 50% of the limit (25 per registered domain) so the other half is reserved for emergency reissuance and recovery.
  • Tiered expiry alerting. Alert at 30 days (renewal window open, verify automation), 14 days (renewal failing), 7 days (act now). Traefik starts renewing around 30 days before expiry, so a cert sitting at 7 days has had renewal failing for over three weeks.
  • Restart and crash-loop alerting. Page on restart patterns, not just process death; each restart without persistent storage is a quota withdrawal.
  • acme.json integrity and backups. Back it up, checksum it, and alert on permission drift away from 600. Corruption at startup hard-fails Traefik; corruption mid-operation fails silently.
  • Staging-first testing. Test every ACME configuration change against https://acme-staging-v02.api.letsencrypt.org/directory before touching production. Staging has far higher limits and burns nothing real.
  • Single ACME writer by design. Make certificate ownership an explicit architectural decision in any multi-instance deployment, not an emergent property of replica count.
  • Issuance tracking. There is no built-in counter for quota consumption. Track weekly issuance from CT logs or your own log pipeline so you see the trend before the 429.

How Netdata helps

  • Netdata charts traefik_tls_certs_not_after per certificate, giving you a live countdown for every CN and SAN Traefik manages, which is the earliest reliable indicator that renewal has stopped working.
  • Process restart tracking lets you correlate crash loops with the exact timestamps of duplicate issuance, confirming or eliminating the persistence failure mode in minutes instead of log archaeology.
  • Config reload metrics (traefik_config_reloads_total, traefik_config_last_reload_success) expose the domain churn that drives order bursts during mass deploys.
  • Because Traefik emits no ACME failure metric, pairing Netdata’s metric alerts with log monitoring for “too many certificates” and “Error renewing certificate” closes the gap between a silent 429 and the eventual expiry page.
  • Long retention makes the weekly issuance trend visible, so you can enforce the 50% headroom policy as a measured number rather than a guess.