ProxySQL Client_Connections_aborted rising: clients rejected or crashing on connect

Client_Connections_aborted is a cumulative counter in stats_mysql_global. What matters is the rate of change, not the absolute value. A sustained non-zero rate means clients are failing to establish or maintain connections through ProxySQL.

The diagnostic question is whether aborts come from ProxySQL actively rejecting connections (authentication failures, connection limits), clients disconnecting improperly (timeouts, crashes, protocol mismatches), or the Client Error Limit feature banning source addresses. The Access_Denied_* counters in the same stats table identify which. If aborts rise while the Questions rate drops, clients are experiencing a real outage. If Questions is stable, the impact may be limited to a subset of clients.

What this means

Client_Connections_aborted counts client connections that failed during establishment or were closed improperly. Compare it against Client_Connections_created, which counts successful new connections. If both rise together, you have connection churn (rapid connect/disconnect cycling). If aborted rises while created stays flat, existing connections are being killed or rejected at a higher rate.

ProxySQL tracks three authentication rejection counters alongside the abort counter:

• Access_Denied_Wrong_Password - wrong password attempts
• Access_Denied_Max_Connections - rejected because global mysql-max_connections was reached
• Access_Denied_Max_User_Connections - rejected because a per-user connection limit was hit

Correlating these with Client_Connections_aborted is the fastest path to root cause. If none are rising, the aborts are not auth-related. Look at protocol issues, client-side timeouts, or the Client Error Limit feature.

Common causes

Quick checks

All queries run against the admin interface on port 6032. These are read-only and safe to run during an incident.

# Check abort rate and access denied breakdown
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT Variable_Name, Variable_Value
  FROM stats_mysql_global
  WHERE Variable_Name IN (
    'Client_Connections_aborted',
    'Client_Connections_created',
    'Client_Connections_connected',
    'Access_Denied_Wrong_Password',
    'Access_Denied_Max_Connections',
    'Access_Denied_Max_User_Connections'
  );"

Take two readings 30-60 seconds apart and compute the delta. A rate of 0 with a high absolute value means the problem has already passed.

# Check per-user connection utilization
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT username, frontend_connections, frontend_max_connections
  FROM stats_mysql_users
  ORDER BY frontend_connections DESC;"

# Check Client Error Limit activity
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT Variable_Name, Variable_Value
  FROM stats_mysql_global
  WHERE Variable_Name LIKE '%client_host_error%';"

# Check global connection limit settings
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT variable_name, variable_value
  FROM global_variables
  WHERE variable_name IN (
    'mysql-max_connections',
    'mysql-connect_timeout_client',
    'mysql-client_host_error_counts',
    'mysql-client_host_cache_size'
  );"

# Check per-errno error breakdown (ProxySQL 2.x)
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT * FROM stats_mysql_errors
  ORDER BY last_seen DESC LIMIT 20;"

# Check recent client sessions and their source addresses
mysql -u admin -p -h 127.0.0.1 -P 6032 -e "
  SELECT user, cli_host, hostgroup, db, command, time_ms
  FROM stats_mysql_processlist
  ORDER BY user;"

# Check for unhealthy connection log entries
grep "Closing unhealthy client connection" /var/lib/proxysql/proxysql.log | tail -20

How to diagnose it

flowchart TD
    A["Client_Connections_aborted rate > 0"] --> B{"Access_Denied_Wrong_Password rising?"}
    B -->|Yes| C["Credential mismatch: check mysql_users"]
    B -->|No| D{"Access_Denied_Max_Connections rising?"}
    D -->|Yes| E["Global limit: check mysql-max_connections"]
    D -->|No| F{"Access_Denied_Max_User_Connections rising?"}
    F -->|Yes| G["Per-user limit: check stats_mysql_users"]
    F -->|No| H["Non-auth: check Client Error Limit, timeouts, protocol"]

1. Determine the rate. Two readings 30-60 seconds apart give the delta. A sustained rate confirms an active issue.

2. Correlate with Access_Denied_* counters. Apply the same delta approach to all three. Whichever rises in step with the aborts identifies the rejection cause.

3. If Access_Denied_Wrong_Password is rising, compare the password in mysql_users against what the application uses. Check whether a credential rotation was performed on the MySQL backend but not propagated to ProxySQL. Also verify the monitor credentials (mysql-monitor_username / mysql-monitor_password) are valid, since monitor failures cause a different cascade.

4. If Access_Denied_Max_Connections is rising, check Client_Connections_connected against mysql-max_connections (default 2048). If connected is near the limit, investigate whether a single application is consuming disproportionate connections, or whether the limit is too low for current load. See ProxySQL client connections at mysql-max_connections for frontend saturation details.

5. If Access_Denied_Max_User_Connections is rising, query stats_mysql_users to find which user is at their per-user limit. The max_connections column in mysql_users controls this.

6. If no Access_Denied_* counter is rising, the aborts are not authentication rejections. Check these possibilities:

   • Client Error Limit: If mysql-client_host_error_counts is greater than 0 (default 0, meaning disabled), ProxySQL bans source addresses that exceed the error threshold. Check client_host_error_killed_connections in stats_mysql_global for evidence.
   • Protocol mismatch: Version-specific issues with caching_sha2_password (MySQL 8 default auth plugin) cause silent auth failures on ProxySQL versions that lack full support.
   • Unhealthy disconnects: If mysql-log_unhealthy_connections is true (default), check the ProxySQL log for “Closing unhealthy client connection” entries.

7. Check error codes. If available, per-hostgroup and per-user error breakdowns provide MySQL errno detail. Error 1045 means access denied. Error 1040 means too many connections from the backend. ProxySQL internal errors (9000+ range) indicate proxy-side failures.

8. Verify Questions rate. If Questions drops while aborts rise, the service is degraded for clients. If Questions is stable, the impact may be limited to a subset of connections.

Metrics and signals to monitor

Fixes

Credential mismatch

If Access_Denied_Wrong_Password is the cause, update the credentials in ProxySQL’s mysql_users table:

UPDATE mysql_users SET password='...' WHERE username='...';
LOAD MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS TO DISK;

If the monitor credentials are also stale, update them separately:

SET mysql-monitor_password='...';
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES TO DISK;

The three-layer config model (MEMORY, RUNTIME, DISK) means the change is not live until LOAD TO RUNTIME and not durable across restarts until SAVE TO DISK. Missing either step is a common operational error.

Global connection limit

If Client_Connections_connected is near mysql-max_connections (default 2048), increasing the limit provides immediate relief:

SET mysql-max_connections=4096;
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES TO DISK;

Before raising the limit, verify the host has sufficient file descriptors. Each connection consumes approximately one file descriptor. Check ulimit -n and current FD usage via /proc/<pid>/fd. If FD exhaustion is the actual bottleneck, raising mysql-max_connections will not help.

Also investigate whether a single application or user is consuming disproportionate connections. A connection leak in one service can starve all others sharing the ProxySQL instance.

Per-user connection limit

If Access_Denied_Max_User_Connections is the cause, adjust the max_connections column for the affected user in mysql_users, or redistribute connections across multiple users.

Client Error Limit

If client_host_error_killed_connections is rising and the Client Error Limit feature is enabled (mysql-client_host_error_counts > 0), review whether the threshold is too aggressive for legitimate traffic patterns. A misconfigured application that generates auth errors can get its source IP banned, which then affects all clients behind the same NAT or load balancer.

Authentication plugin incompatibility

If you recently upgraded MySQL backends to 8.x and see auth failures cascading, check whether caching_sha2_password is the default auth plugin on the backend. ProxySQL support for this plugin is version-dependent:

• Pre-2.0.2: no support for caching_sha2_password
• 2.0.2 through 2.5.x: frontend support requires cleartext password storage and admin-hash_passwords set to false
• 2.6.0+: full support for both frontend and backend

If you cannot upgrade ProxySQL immediately, configure the MySQL backend to use mysql_native_password for accounts that connect through the proxy.

Security: check for vulnerability exploitation

If aborts spike without any credential change, deployment, or capacity change, and you are running ProxySQL versions 2.0.18 through 3.0.8, check for CVE-2026-48773. This pre-authentication heap memory corruption vulnerability in first-packet handling allows a remote unauthenticated client to trigger an oversized first packet length, causing recv() to write past a fixed input buffer. If you suspect exploitation, upgrade immediately and review ProxySQL logs for abnormal connection patterns from unexpected source addresses.

Prevention

• Correlate aborts with access denied counters in alerts. A bare Client_Connections_aborted alert is ambiguous. Alerting on aborts plus a specific Access_Denied_* counter gives the on-call engineer immediate context.
• Set per-user connection limits. Without per-user limits in mysql_users, a single misbehaving application can consume all frontend connections. Set max_connections per user based on expected usage.
• Monitor per-user utilization. Track frontend_connections relative to frontend_max_connections from stats_mysql_users. Alert above 90% to catch saturation before rejections.
• Gate cold-start alerts with uptime. Use ProxySQL_Uptime > 600 as a condition to avoid false alarms during restart, when empty connection pools cause brief connection failures.
• Keep credentials in sync across systems. Credential rotation should update the MySQL backend, ProxySQL mysql_users, and ProxySQL monitor credentials in the same change window.
• Verify auth plugin compatibility before MySQL upgrades. Check the ProxySQL version’s support for the target MySQL authentication plugins before upgrading backends.

How Netdata helps

• Per-second collection of Client_Connections_aborted and all Access_Denied_* counters, so rate-of-change is visible without manual delta calculations.
• Abort rate correlated with Questions rate on the same dashboard, making it obvious whether the service is degraded or only a subset of clients is failing.
• Per-user connection tracking from stats_mysql_users, surfacing which users are approaching their frontend_max_connections limit before rejections begin.
• Anomaly scoring on the abort rate, separating a genuine spike from normal post-deployment churn or periodic credential rotation windows.
• Composite alerting that can gate on ProxySQL_Uptime and Client_Connections_non_idle to suppress cold-start and idle-instance false positives.

Related guides

• ProxySQL backend connection pool exhausted: queries queuing for a free connection
• ProxySQL backend flapping between ONLINE and SHUNNED: monitor-induced oscillation
• ProxySQL OFFLINE_SOFT vs OFFLINE_HARD vs SHUNNED: what each backend status means
• ProxySQL backend SHUNNED: why a healthy backend gets pulled out of rotation
• ProxySQL client connections at mysql-max_connections: frontend saturation and rejected clients
• ProxySQL ConnERR climbing: backend connection errors and how to localise them
• ProxySQL ConnPool_get_conn_failure rising: the most direct pool-starvation signal
• ProxySQL hostgroup_locked connections: reading the multiplexing-health ratio
• How ProxySQL actually works in production: a mental model for operators
• ProxySQL error 9001 Max connect timeout reached while reaching hostgroup
• ProxySQL monitor check failures: connect, ping, read-only, and replication-lag probes failing
• ProxySQL MySQL_Monitor_Workers is zero: health checks stopped and status is stale