ProxySQL monitoring checklist: the signals every production proxy needs

ProxySQL sits between your applications and MySQL-compatible backends, fully parsing the MySQL wire protocol and making per-query routing, caching, and connection-pooling decisions.

This checklist defines the monitoring signals every ProxySQL deployment needs, organized into four maturity levels: survival, operational, mature, and expert. Each level builds on the previous one. If you track query digest anomalies at level 4 but cannot tell whether your backends are ONLINE at level 1, you are debugging blind.

All ProxySQL stats are read through the admin interface on port 6032 as SQL queries against stats_* tables. The data-plane listener is port 6033. If you use the built-in Prometheus exporter (disabled by default, port 6070 when enabled via admin-restapi_enabled), it reads from the same underlying tables.

flowchart TD
    L4["Level 4: expert
Processlist, digest anomalies,
per-thread CPU, monitor freshness"]
    L3["Level 3: mature
Multiplexing ratio, config drift,
query cache, FD utilization"]
    L2["Level 2: operational
Pool saturation, latency histograms,
monitor checks, slow queries, CPU"]
    L1["Level 1: survival
Process, port 6033, ONLINE backends,
ConnERR, aborted, RSS"]
    L4 --> L3
    L3 --> L2
    L2 --> L1

Level 1: survival

These six signals answer one question: is ProxySQL alive and serving traffic? If any fail, you have an active outage.

The “zero ONLINE” page needs conditions. Alert only when: zero ONLINE backends persist for more than 60 seconds (an alert-debounce window, not a multiple of the defaults: mysql-monitor_ping_interval is 8000ms and mysql-monitor_connect_interval is 60000ms), ProxySQL_Uptime exceeds 600 (past cold start), and Client_Connections_non_idle > 0 or Questions rate is nonzero (there is actual traffic). Without these conditions, you will page on idle instances, cold starts, and backup windows.

The port 6033 check is not a ProxySQL stats metric. It must be an external TCP connect check, not a query against the admin interface. A hung listener with a running process will pass an admin-interface check but fail a port 6033 check.

SHUNNED is not a page. A backend in SHUNNED state is ProxySQL’s protection mechanism, often transient and self-correcting when replication catches up or the network blip clears. Page on zero ONLINE backends in a hostgroup, not on individual SHUNNED events. Paging on every SHUNNED transition causes alert fatigue.

Level 2: operational

These signals tell you whether ProxySQL is healthy under load, not just alive. Skip this level and you will not know you have a problem until it becomes an outage.

ConnFree == 0 does not mean saturation. ProxySQL can still create new connections up to max_connections configured in mysql_servers. True saturation means ConnERR is rising because ProxySQL tried and failed to open a connection. The most direct saturation indicator is ConnPool_get_conn_failure in stats_mysql_global, covered in level 3.

Latency_us in stats_mysql_connection_pool is ping latency, not query latency. It is measured by the monitor module’s ping check, representing network round-trip and protocol overhead. Actual query execution latency lives in the stats_mysql_commands_counters histogram buckets (cnt_100us through cnt_INFs). Building dashboards on Latency_us thinking it represents user-facing query performance is the most common ProxySQL monitoring mistake.

Stats tables reset on restart. stats_mysql_connection_pool, stats_mysql_query_digest, and most other stats tables are in-memory only. Baseline comparisons against pre-restart data are invalid. External collection must persist this data if you need historical context.

Level 3: mature

These signals answer: is ProxySQL actually doing its job efficiently? At this level you move from “is it broken?” to “is it providing value?”

The hostgroup_locked ratio is the most important ProxySQL-specific signal. Compute it as Client_Connections_hostgroup_locked / Client_Connections_connected. When this ratio approaches 1.0, every client has a dedicated backend connection. The proxy is adding latency and operational complexity with zero pooling benefit. Common causes: ORMs setting session variables (SET NAMES, SET sql_mode, SET time_zone) on every connection, long transactions, prepared statements in certain configurations, LOCK TABLES, user-defined variables, and GET_LOCK(). When multiplexing breaks, backend connection demand scales linearly with client count instead of sublinearly, and the capacity plan that assumed 10:1 multiplexing is fiction.

ConnPool_get_conn_failure is the most direct pool starvation signal. It tells you ProxySQL actually tried to get a backend connection and failed, not just that the pool is busy. This is more actionable than watching ConnFree == 0, which can occur during legitimate burst traffic without actual query failures.

Configuration drift has no metric. The three-layer model (MEMORY, RUNTIME, DISK) means a change can exist in MEMORY without being loaded to RUNTIME, or loaded to RUNTIME without being saved to DISK. A restart reverts to DISK state. There is no stats table that reports layer divergence. You must compare mysql_servers (memory layer) against runtime_mysql_servers (runtime layer) and disk state directly. This is one of the most dangerous silent failures: everything looks fine until ProxySQL restarts and comes back with stale or missing configuration.

File descriptor exhaustion is a binary cliff. ProxySQL uses approximately one FD per connection (client and backend), plus monitor and admin connections. At the ulimit, all new connections and health checks fail simultaneously. ProxySQL does not expose FD counts in stats tables. This must be a host-level check via /proc/<pid>/fd. Set ulimit -n to at least 4x peak expected client connections.

Memory collection interval. admin-prometheus_memory_metrics_interval is the documented variable; it is fixed at 61 seconds by minimum and maximum in current ProxySQL admin documentation.

Polling stats_memory_metrics is expensive. ProxySQL collects jemalloc metrics on a separate interval because the collection itself has measurable cost. Polling more frequently than every 60 seconds can impact production query performance.

Level 4: expert

These signals exist for teams who have been paged at 3 a.m. by ProxySQL enough times to want deeper visibility. They require custom queries beyond what standard monitoring agents collect.

The stats_mysql_errors table is gold during incidents. It shows the actual MySQL error number, the hostgroup it came from, the user and schema context, first and last occurrence, and count. ProxySQL internal errors in the 9000+ range always indicate proxy-side failures, not backend issues. Standard monitoring rarely collects this table.

stats_mysql_query_digest grows unboundedly. Use stats_mysql_query_digest_reset (which reads and clears the table in one operation) for periodic collection. Without this, the table accumulates entries in memory. Also note that queries served from the query cache still appear in this table, routed to hostgroup -1.

How Netdata helps

Netdata’s ProxySQL collector reads from the admin interface on port 6032 at per-second resolution, turning the stats tables in this checklist into continuous time-series data. The signals most improved by per-second collection and cross-signal correlation:

• Backend health transitions with ConnERR context. When a backend moves to SHUNNED, the exact ConnERR spike that triggered it is visible in the same time window, not minutes later when a 60-second poller catches up.
• Multiplexing ratio as a derived chart. Netdata tracks Client_Connections_hostgroup_locked alongside Client_Connections_connected, making the pinning ratio a visible trend rather than something you compute from two separate admin queries.
• Questions rate and Slow_queries as per-second derivatives. Cumulative counters become immediate rate-of-change charts. A traffic drop or slow-query spike is visible within seconds, not after the next scrape interval.
• Memory metrics at safe cadence. The collector respects the cost of stats_memory_metrics collection, polling jemalloc metrics at intervals that do not add load to a busy proxy.
• ML anomaly detection on workload-dependent signals. Questions rate, connection counts, and ConnERR are inherently workload-dependent, which makes absolute thresholds brittle across environments. Anomaly detection flags deviations from learned baselines without requiring per-environment threshold tuning.

Related guides

• How ProxySQL actually works in production: a mental model for operators
• ProxySQL monitoring maturity model: from survival to expert
• ProxySQL backend SHUNNED: why a healthy backend gets pulled out of rotation
• ProxySQL zero ONLINE backends in a hostgroup: total outage for that traffic class
• ProxySQL OFFLINE_SOFT vs OFFLINE_HARD vs SHUNNED: what each backend status means
• ProxySQL backend flapping between ONLINE and SHUNNED: monitor-induced oscillation
• ProxySQL monitor check failures: connect, ping, read-only, and replication-lag probes failing
• ProxySQL MySQL_Monitor_Workers is zero: health checks stopped and status is stale
• ProxySQL backend connection pool exhausted: queries queuing for a free connection
• ProxySQL error 1040 Too many connections: the backend MySQL rejecting the pool
• ProxySQL error 9001 Max connect timeout reached while reaching hostgroup
• ProxySQL ConnPool_get_conn_failure rising: the most direct pool-starvation signal