$ guides / traefik / traefik-high-cpu-usage ▌

Operations Guides

Traefik high CPU usage: TLS handshakes, regex rules, and config rebuilds

Traefik is pegging cores and request latency is climbing. The process is alive, /ping returns 200, and traffic is still flowing, but p99 latency is creeping up and TLS connections are slower to establish. You need to know which of the three usual suspects is burning the CPU: cryptographic work, middleware processing, or configuration rebuilds.

Traefik CPU is dominated by TLS handshake computation (especially with RSA certificates), then middleware chain processing (gzip compression, JWT validation, regex-based routing rules), then configuration rebuilds whose cost scales with routers times middlewares times services. The diagnosis is almost entirely correlation: match the CPU curve against the TLS handshake rate, the config reload rate, and the request rate, and the cause usually names itself.

First: measure CPU against the container’s cgroup CPU limit, not against host cores. A Traefik container limited to 2 cores on a 64-core host that shows “3% of host CPU” is actually saturated. Also note that Go’s default GOMAXPROCS comes from host core count, not the cgroup quota, so a throttled Traefik can overschedule and burn its CPU budget in bursts unless GOMAXPROCS is set explicitly.

What this means

Traefik does three kinds of expensive work, and each has a distinct signature:

TLS handshakes. Every new TLS connection costs roughly 1-5 ms of CPU on modern hardware. That is small per connection, but it is the single largest CPU consumer in a typical Traefik deployment. RSA key exchange is far more expensive than ECDSA: switching from RSA-2048 to ECDSA P-256 certificates can cut handshake CPU by 10-20x. Session resumption eliminates most of the handshake cost for returning clients, so anything that defeats resumption (short connection lifetimes, disabled session tickets, clients that do not reuse connections) forces every connection to pay full price.

Middleware and rule evaluation. Every request traverses the middleware chain. Regex-based routing rules, gzip compression of large responses, and JWT validation are the expensive links. One correction to the usual folklore: Go’s RE2 regexp engine does not backtrack catastrophically, so the regex hazard is steady per-request evaluation cost multiplied across routers, not exponential blowups on crafted input.

Configuration rebuilds. Every provider event (pod churn in Kubernetes, container events in Docker, file changes) triggers a rebuild of the routing table. Rebuild cost is proportional to routers times middlewares times services, and the rebuild is effectively single-threaded: with more than about 5,000 routes, one rebuild can consume 50-200 ms of CPU. At high provider churn rates, Traefik spends more time rebuilding than routing.

The useful mental model: CPU correlating with TLS request rate means crypto-bound; CPU correlating with config reloads means rebuild storm; sustained high CPU at low request rate means middleware or routing overhead.

flowchart TD
  A[High CPU on Traefik process] --> B{Correlate with signals}
  B -->|CPU tracks TLS request rate| C[TLS handshake bound]
  B -->|CPU tracks config reloads| D[Rebuild storm]
  B -->|High CPU at low request rate| E[Middleware / regex bound]
  C --> C1[RSA certs? Session resumption off? High new-connection rate?]
  D --> D1[Provider churn? Throttle too low? Huge route table?]
  E --> E1[gzip on large responses? Expensive regex? JWT validation?]

Common causes

Cause	What it looks like	First thing to check
RSA certificates under high new-connection rate	CPU tracks `traefik_entrypoint_requests_tls_total`; profile shows `crypto/rsa.decrypt` on top	TLS cert key type and handshake rate
Session resumption defeated	Every connection pays full handshake cost; CPU high even at modest request rates	Upstream LB keep-alive behavior, session ticket settings
Config rebuild storm	CPU spikes align with `traefik_config_reloads_total` increments; latency jitter, not sustained increase	Reload rate and provider event rate
Regex routing rules	Sustained CPU at low or moderate request rate; cost grows with the number of regex matchers evaluated per request	Router rules using HostRegexp, PathRegexp, or regex middlewares
gzip compression on large responses	CPU tracks response bytes more than request count	Which services route through a compress middleware
JWT validation or auth middleware on hot paths	CPU scales with request rate even for cheap requests	Middleware chain on the busiest routers
Large routing table	Baseline CPU climbs over weeks as routes accumulate; rebuilds get slower	Router, service, and middleware counts

Quick checks

These are read-only and safe to run during an incident.

# Process CPU (rate this counter; also compare against the cgroup limit)
curl -s http://localhost:8080/metrics | grep process_cpu_seconds_total
top -p "$(pgrep -d, traefik)" -bn1

# Container CPU limit (cgroup v2)
cat /sys/fs/cgroup/cpu.max

# TLS request rate by version and cipher
curl -s http://localhost:8080/metrics | grep traefik_entrypoint_requests_tls_total

# Config reload activity
curl -s http://localhost:8080/metrics | grep traefik_config_reloads_total

# Request rate, to separate "busy" from "inefficient"
curl -s http://localhost:8080/metrics | grep traefik_entrypoint_requests_total

Measure the TLS handshake cost from a client. Handshake time is time_appconnect minus time_connect:

# TLS handshake timing from the client side
curl -w "time_appconnect: %{time_appconnect}\ntime_connect: %{time_connect}\n" \
  -o /dev/null -s https://your-traefik-host/

Same-datacenter clients should see handshakes under about 10 ms. Consistently above 50-100 ms for nearby clients points at CPU saturation on the Traefik side.

If the cause is not obvious from correlation, capture a CPU profile. The pprof endpoints are served by Traefik’s API, and enabling the API may require a restart, so treat this as a planned step, not a first response:

# 30-second CPU profile via pprof (requires the Traefik API enabled)
curl -o cpu.pb.gz "http://localhost:8080/debug/pprof/profile?seconds=30"

A crypto-bound profile is dominated by TLS and RSA functions (crypto/rsa.decrypt is the classic signature). A middleware-bound profile shows regexp evaluation, compression, or auth code. A rebuild-bound profile shows router and middleware construction.

How to diagnose it

Establish the CPU baseline against the real limit. Rate process_cpu_seconds_total and express it as a fraction of the cgroup CPU limit. If you are reading host-level CPU, stop and redo this first; a throttled container looks idle on host-wide graphs.
Correlate CPU with the TLS request rate. Chart rate(process_cpu_seconds_total) next to rate(traefik_entrypoint_requests_tls_total). If the curves move together, you are crypto-bound. Confirm by checking the certificate key type (RSA vs ECDSA) and the new-connection rate from upstream load balancers and clients.
Correlate CPU with config reloads. Chart CPU next to rate(traefik_config_reloads_total). Spikes aligned with reload increments mean rebuild cost. Check what the provider is doing: pod churn, autoscaling events, mass deployments. Also check routing table size, since rebuild cost scales with it.
Check the middleware path. If CPU is high but both TLS rate and reload rate are flat, look at the request path itself. Compare traefik_entrypoint_request_duration_seconds with traefik_service_request_duration_seconds: if entrypoint latency is high while service latency is normal, the overhead is inside Traefik’s own processing (TLS, middleware, compression), not the backends.
Audit regex rules. List routers and middlewares that use regex matchers (HostRegexp, PathRegexp, QueryRegexp, HeaderRegexp, RedirectRegex, ReplacePathRegex). Look for unnecessary wildcards and overlapping alternations. RE2 will not blow up exponentially, but every regex matcher on a router is evaluated per request, and the cost adds up across a large table.
Profile if still ambiguous. Take the 30-second pprof CPU profile above and attribute CPU to crypto, regexp, compression, or config construction. This settles arguments quickly.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
`process_cpu_seconds_total` (rate vs cgroup limit)	The primary saturation signal	Sustained above 70% of limit for 5 minutes; above 90% for 2 minutes
`traefik_entrypoint_requests_tls_total`	TLS session rate by version and cipher; the main CPU driver	Rising rate alongside rising CPU; spike in unusual ciphers
`traefik_config_reloads_total`	Rebuild frequency; each rebuild costs CPU proportional to table size	Rate approaching or exceeding 1/second sustained
`traefik_entrypoint_request_duration_seconds` vs `traefik_service_request_duration_seconds`	The gap isolates Traefik-internal overhead from backend time	Entrypoint latency high while service latency normal
`go_goroutines`	Concurrent work; rebuild storms and stuck connections both inflate it	Growth without corresponding traffic growth
`traefik_service_responses_bytes_total`	Large responses through a compress middleware are expensive	CPU tracking response bytes more than request count

Fixes

TLS handshake bound

Switch from RSA to ECDSA P-256 certificates. This is the highest-leverage change: 10-20x cheaper handshakes. For ACME-managed certs this means requesting ECDSA keys; for manual certs, reissue with ECDSA. Client compatibility for ECDSA is a non-issue for anything remotely modern, but verify if you serve very old clients.

Restore session resumption. Resumed sessions skip most of the handshake cost. Make sure nothing in your TLS options disables session tickets, and check that upstream load balancers and clients keep connections alive instead of opening a new connection per request. A load balancer in front of Traefik with keep-alive disabled turns every request into a full handshake.

Scale out or offload. TLS handshake capacity is a hard CPU ceiling. If you are saturated after the above, add replicas and spread the handshake load, or terminate TLS further upstream. There is no config knob that makes RSA cheap.

Config rebuild storm

Increase providersThrottleDuration. The default is 2 seconds; for busy Kubernetes environments raise it to 5-10 seconds. This batches provider events into fewer rebuilds and is the standard fix for churn-driven rebuild load.

Reduce provider churn and table size. Batch mass deployments instead of triggering thousands of individual pod events. Scope providers to the namespaces Traefik actually serves. If the routing table has grown very large, rebuild cost per event grows with it, and splitting across multiple Traefik instances by namespace or shard becomes worth considering.

Middleware and regex bound

Simplify regex rules. Trim unnecessary wildcards and alternations. Prefer plain Host and PathPrefix matchers over regex where they express the same intent; every regex matcher adds per-request evaluation cost.

Reconsider compression placement. gzip on large responses is CPU-expensive at proxy scale. Options: compress at the backend, compress only for content types and sizes where it pays off, or drop compression for already-compressed media. Check whether CPU correlates with traefik_service_responses_bytes_total before touching anything.

Lighten the auth path. JWT validation and auth middlewares run on every request on their routers. If CPU scales linearly with request rate on auth-protected routes, that chain is the cost; reduce what runs per request or scope the middleware to the routers that actually need it.

Prevention

Alert on CPU against the cgroup limit: ticket at sustained >70% for 5 minutes, page at >90% for 2 minutes. Host-level CPU alerts are meaningless for a throttled container.
Baseline the TLS handshake rate and alert on divergence. A sudden jump in new TLS sessions (reconnection storm after a network blip, CDN cache purge, client deploy that disabled keep-alive) is your earliest warning of crypto saturation.
Alert on reload rate. Sustained traefik_config_reloads_total above roughly 1/second deserves investigation before it becomes a rebuild storm. Set providersThrottleDuration deliberately instead of inheriting the 2-second default.
Prefer ECDSA certificates from day one. There is rarely a reason to deploy new RSA-2048 certs on a high-connection-rate edge.
Track routing table growth. Router, middleware, and service counts are a slow-moving CPU liability: both per-request matching cost and per-rebuild cost grow with them.
Keep a CPU profile from a healthy period. Comparing a bad-time pprof profile against a known-good baseline makes attribution a two-minute job.

How Netdata helps

Netdata charts process_cpu_seconds_total alongside container cgroup CPU usage and limits, so you see saturation against the real ceiling rather than against idle host cores.
Per-second collection catches rebuild-storm CPU spikes and TLS reconnection bursts that minute-resolution monitoring averages away.
Correlating Traefik CPU with traefik_entrypoint_requests_tls_total (by TLS version and cipher) and traefik_config_reloads_total on one dashboard is the fastest path to the crypto-vs-rebuild-vs-middleware attribution this article describes.
Go runtime metrics (go_goroutines, heap, GC pauses) are collected from the same endpoint, so goroutine growth and GC pressure are visible next to the CPU curve when the cause is a leak rather than traffic.
ML-based anomaly detection on the CPU and TLS-rate metrics flags divergence from the learned baseline, which is how you catch a slow-building handshake or churn problem before the latency alerts fire.