$ guides / proxysql / proxysql-query-execution-interrupted ▌

Operations Guides

ProxySQL error 1317 Query execution was interrupted: timeout-killed queries

Error 1317 (“Query execution was interrupted”) means ProxySQL killed a backend query that exceeded a configured timeout. When a query runs past its limit, ProxySQL opens a separate connection to the backend, issues KILL QUERY <thread_id>, and returns error 1317 to the client.

The question is whether 1317 is a healthy safety valve catching runaway queries, or a symptom of backend degradation killing queries that should normally succeed. Raising a timeout is cheap, but masking backend slowness with a higher ceiling only delays the problem.

The error surfaces in stats_mysql_errors with hostgroup, hostname, port, username, and schemaname context. The diagnostic path: confirm the error, identify which timeout fired, determine whether the killed queries are genuinely slow, and decide whether to tune the timeout or fix the backend.

What this means

ProxySQL tracks execution time for every query it sends to a backend. Four mechanisms can trigger a kill:

mysql-default_query_timeout (global variable, default 86400000 ms = 24 hours). Applies to all queries not matched by a more specific rule.
mysql_query_rules.timeout (per-rule override). When a query matches a rule with a timeout value set, that value replaces the global default for matching queries.
Query annotations. Individual queries can carry a timeout in a SQL comment: SELECT /*+ ;query_timeout=100 */ .... Overrides both the global and rule-level timeout for that specific query.
mysql-max_transaction_time (global variable, default 14400000 ms = 4 hours). Sessions with an open transaction exceeding this duration are killed, regardless of individual query duration.

When any threshold fires, ProxySQL returns error 1317 to the client. Queries killed by timeout are not retried, even if mysql-query_retries_on_failure (default 1) is set. This was an intentional change introduced in ProxySQL 1.4.15 and 2.0.6 to prevent retrying queries that a DBA manually killed on the backend. The tradeoff: queries interrupted during a graceful backend shutdown, which some MySQL versions report as 1317 instead of 1053, also do not get retried.

flowchart TD
    A["Error 1317 in
stats_mysql_errors"] --> B{"Which timeout
is in effect?"}
    B --> C["Global:
mysql-default_query_timeout"]
    B --> D["Per-rule:
mysql_query_rules.timeout"]
    B --> E["Annotation:
query_timeout in SQL comment"]
    B --> F["Transaction:
mysql-max_transaction_time"]
    C --> G{"Are Slow_queries and
per-backend latency rising?"}
    D --> G
    E --> G
    F --> G
    G --> H["Yes: backend degraded
locks, I/O, bad plan"]
    G --> I["No: timeout threshold
too low for workload"]
    G --> J["Overflow bug?
Check ProxySQL version"]

Common causes

Cause	What it looks like	First thing to check
Backend genuinely slow	1317 correlates with rising Slow_queries, high per-backend Latency_us, histogram tail shift toward cnt_1s or cnt_10s	`stats_mysql_query_digest` for the slowest queries by sum_time
Timeout too aggressive	1317 appears after a config change; queries that previously completed now get killed	`runtime_mysql_query_rules` for recently changed timeout values
Timeout overflow bug (pre-2.0.12)	Long timeout values above ~35 min in mysql_query_rules cause queries to die after ~2 seconds due to integer overflow	ProxySQL version; upgrade to 2.0.12 or later
Backend shutdown ambiguity	1317 during maintenance; some MySQL versions report 1317 instead of 1053 on graceful shutdown	Whether a backend was being drained or restarted when errors appeared

Quick checks

All commands connect to the ProxySQL admin interface (default port 6032) and are read-only. Replace <password> with your admin credentials, or omit -p to be prompted interactively.

# Error 1317 entries with full hostgroup/user context
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT * FROM stats_mysql_errors WHERE errno=1317 ORDER BY last_seen DESC LIMIT 20;"

# Global timeout and retry variables
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT * FROM global_variables WHERE variable_name IN ('mysql-default_query_timeout','mysql-max_transaction_time','mysql-query_retries_on_failure');"

# Query rules with a timeout set
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT rule_id, match_digest, match_pattern, timeout, destination_hostgroup, active FROM runtime_mysql_query_rules WHERE timeout IS NOT NULL AND timeout > 0 ORDER BY rule_id;"

# Slow queries vs total questions for ratio context
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT Variable_Name, Variable_Value FROM stats_mysql_global WHERE Variable_Name IN ('Slow_queries','Questions','generated_error_packets');"

# Command latency histogram tail
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT Command, Total_cnt, Total_Time_us, cnt_100ms, cnt_500ms, cnt_1s, cnt_5s, cnt_10s, cnt_INFs FROM stats_mysql_commands_counters WHERE Total_cnt > 0;"

# Per-backend monitor latency (ping, NOT query latency)
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT hostgroup, srv_host, srv_port, status, Latency_us, ConnUsed, ConnFree, ConnERR FROM stats_mysql_connection_pool;"

# Top queries by total execution time
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT hostgroup, schemaname, username, digest_text, count_star, sum_time, max_time FROM stats_mysql_query_digest ORDER BY sum_time DESC LIMIT 20;"

# ProxySQL version (check for overflow bug)
mysql -u admin -p<password> -h 127.0.0.1 -P 6032 \
  -e "SELECT * FROM global_variables WHERE variable_name='admin-version';"

How to diagnose it

Confirm the error and get context. Query stats_mysql_errors for errno=1317. Note count_star (cumulative error count), hostgroup, username, and schemaname. If the error is concentrated in one hostgroup or user, the problem is specific to that traffic class. Stats are in-memory and reset on ProxySQL restart.
Determine which timeout fired. Cross-reference the affected hostgroup and query patterns against runtime_mysql_query_rules. If a rule with a low timeout matches the killed queries, that rule is the source. If no rule has a timeout, the global mysql-default_query_timeout applies. Check whether queries carry /*+ ;query_timeout=N */ annotations in the application code.
Check whether the killed queries are genuinely slow. Look at stats_mysql_query_digest for the specific query patterns being killed. Compare max_time against the configured timeout. If max_time consistently exceeds the timeout, the query is genuinely slow and the timeout is catching a real problem. If max_time is well below the timeout but queries still die, suspect the overflow bug or another mechanism.
Check per-backend latency and health. Use stats_mysql_connection_pool to see if one backend has significantly higher Latency_us than others in the same hostgroup. A backend with ConnERR > 0 or SHUNNED status may be the root cause, with 1317 as a downstream symptom. Latency_us is the monitor’s ping latency, not query execution time. Use it to detect network or protocol-level degradation, not query slowness.
Check the histogram tail. In stats_mysql_commands_counters, look for counts accumulating in cnt_1s, cnt_5s, cnt_10s, or cnt_INFs buckets. A shift toward these buckets confirms backend-level execution degradation. The histogram is cumulative since stats reset, so compare two consecutive readings to get interval-based rates.
Check the ProxySQL version. If mysql_query_rules.timeout is set above approximately 2,100,000 ms (about 35 minutes) and queries die after roughly 2 seconds, you are hitting the integer overflow bug documented in GitHub issue #2765, fixed in ProxySQL 2.0.12.
Distinguish safety valve from symptom. If Slow_queries is rising and histogram tail buckets are accumulating, the backend is degraded. Fix the backend: query optimization, index tuning, lock investigation. If queries complete normally elsewhere and only specific patterns hit 1317, the timeout threshold may be too low for the workload.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
`stats_mysql_errors` (errno=1317)	Direct count of timeout-killed queries per hostgroup/user	Sustained increase above baseline
`Slow_queries` rate	Correlates with timeout kills; confirms backend slowness	Rising rate proportional to 1317 count
`stats_mysql_commands_counters` histogram tail	Shows queries landing in multi-second buckets	cnt_1s, cnt_5s, cnt_10s counts increasing
Per-backend `Latency_us`	Network/protocol latency divergence between backends	One backend more than 5x peers in same hostgroup
`stats_mysql_query_digest` max_time	Shows whether killed queries are genuinely slow	max_time approaching or exceeding configured timeout
`ConnERR` per backend	Backend unreachable or rejecting connections	Sustained increase on specific backend
`mysql-default_query_timeout` and rule `timeout`	The actual threshold in effect	Unexpectedly low value, or recent config change

Fixes

Backend is genuinely slow

The timeout is working correctly. The fix is on the backend, not in ProxySQL.

Identify slow query patterns via stats_mysql_query_digest sorted by sum_time descending.
Check the MySQL backend directly: SHOW PROCESSLIST, SHOW ENGINE INNODB STATUS, slow query log, EXPLAIN on the top digests.
Look for lock contention, missing indexes, table scans on large tables, or I/O saturation.
If the slow query is an intentional batch or ETL job, route it to a dedicated hostgroup with a higher timeout rather than raising the global limit for all traffic.

Timeout is too aggressive

If queries that previously completed are now killed after a config change:

Check runtime_mysql_query_rules for the affected query patterns and their timeout values.
Raise the timeout to accommodate the slowest acceptable query in that class. Tradeoff: a higher timeout means runaway queries run longer before being killed, consuming backend resources and holding connections.
Avoid setting mysql-default_query_timeout extremely high as a blanket fix. Set targeted per-rule timeouts that match the expected execution profile of each query class.
After changing: LOAD MYSQL QUERY RULES TO RUNTIME; SAVE MYSQL QUERY RULES TO DISK;

Timeout overflow bug (pre-2.0.12)

If mysql_query_rules.timeout is set above approximately 2,100,000 ms (about 35 minutes) and queries die after roughly 2 seconds, this is integer overflow, not a real timeout. Upgrade ProxySQL to 2.0.12 or later. As a workaround on older versions, keep per-rule timeout values below the overflow threshold and rely on mysql-default_query_timeout for long-running queries.

Backend shutdown reported as 1317

Some MySQL versions return error 1317 instead of 1053 during a graceful shutdown. ProxySQL does not retry 1317-killed queries. If this happens during planned maintenance:

Accept that queries in flight during shutdown will fail.
If using OFFLINE_SOFT to drain a backend, wait for ConnUsed to reach zero before switching to OFFLINE_HARD.
Application-level retry logic should treat 1317 as retryable during maintenance windows, since ProxySQL will not retry it.

Prevention

Baseline the Slow_queries to Questions ratio. A sustained increase is the earliest indicator that backend degradation will start producing 1317 errors.
Set per-rule timeouts deliberately. Match the timeout to the query class: short timeouts for OLTP reads, longer timeouts for reporting queries.
Monitor stats_mysql_errors for new error codes. A new errno appearing that was absent from baseline signals a new failure mode.
Keep ProxySQL updated. The overflow bug and retry behavior are version-dependent. Document version-specific behavior in your runbook.
Watch for repeated KILL QUERY entries in the ProxySQL log. When a query times out, ProxySQL may log multiple KILL attempts for the same thread_id, each followed by error 1317. This is normal behavior, not a bug, but can inflate log volume.

How Netdata helps

Per-second Slow_queries rate reveals backend degradation developing before it produces enough 1317 errors to trigger application-level alerts.
stats_mysql_errors breakdown by errno shows 1317 counts with hostgroup and user context, so you can see whether the problem is isolated to one traffic class or systemic.
stats_mysql_commands_counters histogram shows the tail-latency shift toward cnt_1s and cnt_10s buckets, confirming that killed queries were genuinely slow rather than victims of an overly aggressive timeout.
Per-backend Latency_us and ConnERR correlate 1317 errors with specific backend degradation, separating “backend is slow” from “timeout is wrong.”
Questions rate provides the denominator for error-rate calculations. A spike in 1317 means something different at 10,000 queries per second than at 100.
ML anomaly detection on Slow_queries and generated_error_packets can flag a degradation trend before it crosses a static threshold, providing lead time before 1317 errors reach application-visible volume.