nginx SSL_do_handshake() failed — diagnosing TLS handshake errors

You see [crit] or [error] entries for SSL_do_handshake() failed in /var/log/nginx/error.log. The message might involve a client connecting to nginx, or nginx connecting to an upstream server. These errors mean the TLS negotiation never completed, so no application data was exchanged. The impact ranges from a few browsers showing security warnings to all proxied traffic returning 502 Bad Gateway.

Before chasing certificates, determine which side of a connection is failing. nginx logs two variants: while SSL handshaking to client and while SSL handshaking to upstream. The first affects visitors hitting your server directly. The second affects nginx as a reverse proxy calling a backend over HTTPS. The symptoms, root causes, and fixes are different.

What this means

SSL_do_handshake() is the OpenSSL routine nginx calls to perform the TLS handshake. When it fails, the underlying TCP connection is established but the cryptographic negotiation breaks down. nginx logs the failure and closes the connection.

For client-facing listeners, the failure is usually caused by a certificate problem, a protocol or cipher mismatch, or a mutual TLS requirement that the client does not satisfy. For upstream proxy connections, the failure is usually caused by missing SNI, an untrusted upstream certificate, or a TLS version mismatch between nginx and the backend.

Because the handshake fails before HTTP begins, these errors do not produce standard HTTP status codes in the access log. The only evidence is in the error log and in secondary signals like connection count or upstream latency.

flowchart TD
    A[SSL_do_handshake failed in error log] --> B{Which direction?}
    B -->|to client| C[Check certificates protocols ciphers mTLS]
    B -->|to upstream| D[Check proxy_ssl SNI protocols trust]
    C --> E{Alert 40 or no shared cipher?} --> F[Align ssl_protocols and ssl_ciphers]
    C --> G{Cert verify error?} --> H[Check expiry chain and hostname]
    C --> I{mTLS required?} --> J[Verify ssl_verify_client and client certs]
    D --> K{Unrecognized name?} --> L[Set proxy_ssl_server_name and proxy_ssl_name]
    D --> M{Protocol mismatch?} --> N[Match proxy_ssl_protocols to upstream]
    D --> O{Cert untrusted?} --> P[Configure proxy_ssl_trusted_certificate]

Common causes

Quick checks

Run these read-only commands to scope the problem before making changes.

# Count handshake failures and identify direction (client vs upstream)
grep -c 'SSL_do_handshake() failed' /var/log/nginx/error.log
grep 'SSL_do_handshake() failed' /var/log/nginx/error.log | tail -20

# Extract current TLS configuration from the running config
nginx -T 2>/dev/null | grep -E 'ssl_protocols|ssl_ciphers|proxy_ssl_protocols|proxy_ssl_ciphers'

# Find certificate paths and verify expiration dates
nginx -T 2>/dev/null | grep ssl_certificate | grep -v ssl_certificate_key | while read -r _ path; do
  echo "Cert: $path"
  openssl x509 -noout -dates -subject -in "$path" 2>/dev/null
done

# Test SNI and TLS negotiation against an upstream target
# Replace TARGET with the actual upstream hostname
openssl s_client -connect TARGET:443 -servername TARGET </dev/null 2>/dev/null | \
  openssl x509 -noout -subject -dates

# Check TLS version distribution from access logs
# Requires $ssl_protocol in the log_format
tail -10000 /var/log/nginx/access.log | \
  awk '{for(i=1;i<=NF;i++) if($i ~ /^TLSv/) count[$i]++} END {for(v in count) print v": "count[v]}'

How to diagnose it

1. Determine the direction. Read the error log line carefully. while SSL handshaking to client means the visitor’s browser or client cannot agree on TLS with nginx. while SSL handshaking to upstream means nginx cannot establish TLS to a backend it is proxying to. If you see both, treat them as separate incidents.

2. Check for configuration drift. If the errors started immediately after a reload, compare the current configuration with the last known good state. nginx -T dumps the effective configuration. Look for recent changes to ssl_protocols, ssl_ciphers, ssl_certificate, or any proxy_ssl_* directive.

3. For client-side failures, inspect the alert details. OpenSSL passes specific error strings through nginx. no shared cipher means the client’s cipher list and nginx’s ssl_ciphers have no overlap. tlsv1 alert protocol version or version too low means the client tried to use a TLS version that nginx disabled. peer did not return a certificate means a location requires mTLS but the client presented none.

4. Verify certificate health. Even if the certificate has not expired, check the full chain. The leaf certificate must come first in the file, followed by intermediates. Some clients reject misordered chains. Confirm the certificate covers the hostname clients use. If you recently renewed, ensure nginx was reloaded so the new file is loaded into memory.

5. For upstream proxy failures, test SNI explicitly. Many CDNs and APIs require SNI. Run openssl s_client -connect upstream:443 without -servername and then with it. If the connection fails without SNI but succeeds with it, add proxy_ssl_server_name on; and set proxy_ssl_name to the hostname expected by the upstream certificate.

6. Align upstream TLS parameters. If the upstream enforces TLS 1.3 and your proxy_ssl_protocols omits it, the handshake fails. Similarly, if the upstream uses a self-signed certificate or an internal CA, nginx rejects it unless you configure proxy_ssl_trusted_certificate to point to the correct CA bundle. Do not set proxy_ssl_verify off in production as a quick fix; it removes protection against man-in-the-middle attacks.

7. Determine if the traffic is legitimate. A low, steady rate of handshake failures from diverse IPs is often scanning or outdated bots. Correlate the error timestamps with your request rate. If real user traffic is unaffected and the errors correlate with no access log entries, the failures are likely benign. If the errors spike along with 502 responses or user complaints, the configuration is wrong.

Metrics and signals to monitor

Fixes

Protocol or cipher mismatch (client-facing)

Align ssl_protocols and ssl_ciphers with your client base. Modern nginx defaults to TLSv1.2 TLSv1.3, which is safe for nearly all legitimate traffic. If you maintain a custom cipher list, ensure it includes ciphers that your least-capable supported client can use. Remember that ssl_ciphers does not restrict TLS 1.3 ciphersuites; those are negotiated internally by OpenSSL. To restrict TLS 1.3 ciphers explicitly, use ssl_conf_command Ciphersuites ... on nginx 1.19.4 and later.

SNI mismatch (upstream proxy)

If the upstream is a CDN, API gateway, or multi-tenant TLS endpoint, it probably requires SNI. Add proxy_ssl_server_name on; and set proxy_ssl_name to the hostname in the upstream certificate’s CN or SAN. Without this, nginx presents no Server Name Indication, and the upstream may close the connection.

Client certificate (mTLS) failures

Ensure ssl_client_certificate points to the CA that issued the client certificates, or ssl_trusted_certificate if you are using a chain. Verify that ssl_verify_client is set to the level you actually require: optional allows requests without a cert but verifies it if present, while on rejects any connection without a valid cert. If clients stopped presenting certificates after a renewal, check that their trust store includes your new issuer.

Certificate expiry or chain issues

Renew and deploy certificates before expiration. After replacing files on disk, run nginx -t and then reload nginx. Verify the file order: leaf certificate first, intermediate certificates second. A misordered chain causes alert 40 on strict clients and some upstreams. Use openssl x509 -in <file> -text -noout to inspect dates, subject, and issuer.

Upstream TLS misalignment

Set proxy_ssl_protocols to include the versions the upstream accepts. If the upstream uses a private CA, place the CA file in a known path and reference it with proxy_ssl_trusted_certificate. Keep proxy_ssl_verify on in production. If you are debugging in a non-production environment and must disable verification temporarily, document the exception and revert it immediately.

Prevention

Monitor certificate expiration independently of your renewal automation. An expired certificate is an instant outage that no amount of load balancing can mask.

Log $ssl_protocol and $ssl_cipher in your access log format. Review the distribution before removing support for older protocols. A shift in client capabilities is a leading indicator that a planned hardening change will cause failures.

Test TLS configuration changes against a representative client profile before deploying to production. If you serve embedded devices or legacy enterprise tools, they may not support the latest OpenSSL defaults.

Keep the shared SSL session cache configured and adequately sized. A healthy session cache reduces full handshake volume, which lowers CPU usage and reduces the blast radius of any handshake-related misconfiguration.

Always run nginx -t before reloading. A syntax error in a server block will fail the reload, but the previous config remains active. That safety feature can hide the fact that your intended fix never took effect.

How Netdata helps

• Correlate nginx error log rate with worker CPU and connection state breakdown to distinguish a TLS configuration error from an SSL CPU saturation event.
• Track TLS version distribution via access log parsing to detect client capability shifts before they become outages.
• Monitor SSL session cache hit rate to spot when handshake overhead is climbing, giving you early warning before CPU saturation amplifies failures.
• Alert on certificate expiration independently of your renewal pipeline, so automation gaps do not become outages.
• Correlate upstream connect time spikes with proxy SSL handshake errors to isolate upstream-side failures without guessing.

Related guides

• nginx 413 Request Entity Too Large: client_max_body_size explained
• nginx 499 status code: why clients close connections before the response
• nginx 500 Internal Server Error: how to diagnose it
• nginx 502 Bad Gateway: causes and how to fix it
• nginx 503 Service Temporarily Unavailable: causes and fixes
• nginx 504 Gateway Time-out: causes and fixes
• NGINX access log performance: buffering, sampling, and the event loop
• NGINX active connections climbing: reading, writing, waiting explained
• nginx: bind() to 0.0.0.0:80 failed (98: Address already in use)
• NGINX backend cascade failure: when slow upstreams take down everything
• NGINX proxy cache hit rate is low: measuring and improving it
• nginx: configuration file test failed - finding the syntax error