PgBouncer avg_wait_time high: the latency the pool itself is injecting

Your application latency is up, PostgreSQL looks fine, and then PgBouncer’s stats show avg_wait_time at tens or hundreds of milliseconds. That number answers “where did the latency come from”: it is time clients spent queued inside PgBouncer waiting for a server connection, before their query even reached PostgreSQL.

avg_wait_time is purely PgBouncer-induced latency. A direct connection to PostgreSQL would not have it. When it is high, the pool is not keeping up with demand, and every queued client pays that delay on top of normal query execution time.

This guide covers how to read the metric correctly (including a version difference that changes what the number means), how to tell pool queuing apart from backend slowness, and how to fix the root cause rather than just widening the pool.

What this means

avg_wait_time comes from SHOW STATS and SHOW STATS_AVERAGES, measured in microseconds, averaged over the stats period (stats_period, default 60 seconds). It is the average time clients spent in the FIFO wait queue before being assigned a server connection.

Interpretation bands:

• Sub-millisecond: healthy. Clients are assigned server connections essentially instantly.
• Sustained above 10ms: the pool is routinely saturated. Queuing happens often enough to show up in the average.
• Above 100ms (100,000 microseconds) sustained: visible application latency. If your application has a 200ms response budget, 100ms of queue wait has consumed half of it before the query starts.

Three properties matter before you act on this metric:

1. It is an average, and averages hide bursts. High traffic volume dilutes a few very slow waiters. You can see avg_wait_time = 5ms while the P99 wait during a burst was 500ms. PgBouncer exposes no percentiles.
2. It can be skewed by timed-out clients. Clients that wait until query_wait_timeout (default 120s) and get disconnected have their entire wait accumulated into the stats. A handful of timed-out waiters can drag the average up even if most clients were served quickly.
3. The calculation changed in PgBouncer 1.23.0. Before 1.23.0 (July 2024), avg_wait_time was total accumulated wait time divided by the wall-clock duration of the stats period, not by the number of clients served. One client waiting 10 minutes would report roughly 10 seconds of “average” wait time in the period it was finally assigned a backend, regardless of what any other client experienced. The reported value had little relation to what a typical client waited. 1.23.0 made it a true per-client average (total wait divided by server_assignment_count). If you are on an older version, treat avg_wait_time as unreliable and lean on maxwait from SHOW POOLS instead. See the 1.23.0 changelog entry and PR #727 for details.

One check before anything else: confirm the database is not administratively paused. During PAUSE, cl_waiting spikes and wait times climb by design. Every PgBouncer alert and every diagnosis should start with SHOW DATABASES and the paused / disabled columns.

Common causes

Quick checks

All commands run against the PgBouncer admin console and are read-only.

# Check administrative state first: paused or disabled databases
psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -Atc "SHOW DATABASES;"

# Current per-database averages: wait_time, query_time, xact_time (microseconds)
psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -Atc "SHOW STATS_AVERAGES;"

# Who is waiting right now, and how long has the oldest waiter been blocked
psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -Atc "SHOW POOLS;"
# cl_waiting, sv_active, sv_idle, sv_login, maxwait, maxwait_us per pool

# Per-client wait times: find the worst current waiters
psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -Atc "SHOW CLIENTS;"
# state, wait, wait_us per client

# Which server connections are holding the pool: oldest active request_time
psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -Atc "SHOW SERVERS;"

# Event loop health: time a trivial admin command
time psql -h /var/run/postgresql -p 6432 -U pgbouncer pgbouncer -c "SHOW LISTS;" > /dev/null

# Process CPU: is the single thread saturated
ps -p $(pgrep -f pgbouncer) -o %cpu,rss

# Log: wait timeouts and connection failures are log-only signals
grep -E "query_wait_timeout|login failed|connect failed" /var/log/pgbouncer/pgbouncer.log | tail -20

Two caveats. Column positions in SHOW STATS are version-dependent (1.23 added server_assignment_count, 1.24 added prepared statement counters), so reference columns by name or use SHOW STATS_AVERAGES for the simpler layout. And there are no error counters in any SHOW command; timeouts, rejections, and login failures only exist in the log.

How to diagnose it

The core question is: pool problem or backend problem? Three signals answer it.

flowchart TD
  A[avg_wait_time high] --> B{Database paused or disabled?}
  B -- yes --> C[Expected: maintenance state. RESUME when done]
  B -- no --> D{cl_waiting greater than 0 now?}
  D -- no --> E[Burst already drained or pre-1.23 metric artifact. Check maxwait and version]
  D -- yes --> F{avg_query_time elevated?}
  F -- yes --> G[Backend is slow: connections held longer, pool starves. Fix PostgreSQL]
  F -- no --> H{avg_xact_time much greater than avg_query_time?}
  H -- yes --> I[Idle-in-transaction: app holds connections without querying. Fix app]
  H -- no --> J[Pool undersized or CPU-bound event loop. Check sv_active vs pool_size and process CPU]

1. Rule out maintenance. SHOW DATABASES: if paused = 1 or disabled = 1, the queuing is intentional. Do not escalate.
2. Confirm live queuing. SHOW POOLS: cl_waiting > 0 means clients are blocked right now, and maxwait is the age of the oldest waiter. If cl_waiting is zero but avg_wait_time was high, the burst already drained, or you are on a pre-1.23.0 version looking at a distorted average. Cross-check with maxwait and your version.
3. Split pool latency from backend latency. Compare avg_wait_time against avg_query_time in SHOW STATS_AVERAGES. High wait with normal query time means PostgreSQL is fast and the pool is the bottleneck. High query time means PostgreSQL is slow, connections are held longer, and the queue is a symptom. This split is the most common misdiagnosis: teams see latency, blame the database, and the database was at 5ms while the queue was at 2000ms.
4. Check for idle-in-transaction. If avg_xact_time is far larger than avg_query_time (a 10x gap is a strong signal), clients are holding server connections while doing non-database work. Confirm in SHOW SERVERS (active connections with old request_time) and on PostgreSQL itself (pg_stat_activity rows in idle in transaction state).
5. Identify the culprits. In SHOW SERVERS, find active connections with the oldest request_time; follow the link column to SHOW CLIENTS to get the source address of the application holding them.
6. Check supply. Is sv_active pinned at pool_size with sv_idle = 0? If yes, demand exceeds the pool. Also sum the sv_* columns against pool_size: if the total regularly exceeds it, the reserve pool is being drawn, which means the base pool has been undersized for a while.
7. Check the event loop. If all pools degrade together and the admin console itself is slow (over ~200ms for a trivial SHOW), look at process CPU. PgBouncer is single-threaded; at 100% of one core (often from TLS handshakes or connection churn) everything queues behind the event loop, not just behind the pool.
8. Check the log for query_wait_timeout. Each occurrence confirms a client waited the full timeout (default 120s) and was disconnected. These waits also inflate avg_wait_time, so their presence changes how you read the average.

Metrics and signals to monitor

Fixes

Backend is slow (avg_query_time elevated)

The queue is a symptom. Fix PostgreSQL: find the slow queries (pg_stat_statements), lock contention, or I/O saturation. If specific long-running queries are identifiable and safe to cancel, canceling them on the PostgreSQL side frees server connections immediately. Coordinate with the owning team first; killing a query is disruptive to whoever ran it. Raising pool_size here is the wrong move: it just sends more concurrent load to an already struggling backend.

Idle-in-transaction (avg_xact_time » avg_query_time)

This is an application fix, not a PgBouncer fix. Find the clients via SHOW SERVERS link to SHOW CLIENTS, then fix the code path that opens a transaction and does non-database work before committing. As a guardrail, PostgreSQL’s idle_in_transaction_session_timeout can auto-terminate these sessions; clients get an error, so roll it out deliberately.

Pool genuinely undersized

Increase pool_size (per-database override or default_pool_size) and run RELOAD. Constraints to respect: the sum of all pools across all PgBouncer instances targeting one PostgreSQL must fit within that server’s max_connections with headroom for superuser and direct connections (keep pool capacity under ~80% of it). If reserve_pool_size > 0 and the reserve is drawn regularly, treat that as proof the base pool has been undersized; size the base pool so the reserve is only touched by genuine spikes.

Event loop saturated

If CPU on the single thread is the limit, scaling the pool does nothing. The supported path is multiple PgBouncer processes sharing the port via so_reuseport, each with independent pools and stats (monitoring must aggregate across them). Offloading TLS termination to a layer in front of PgBouncer also cuts CPU sharply if TLS handshakes are the driver.

Clients timing out and retrying

If application timeouts are shorter than query_wait_timeout, applications give up and retry while their original requests still hold queue slots, amplifying the queue. Align the timeouts, and make sure retries back off rather than stampede.

Prevention

• Alert on the right combination. cl_waiting > 0 sustained, with maxwait above your application tolerance, and the database not paused. Raw cl_waiting > 0 pages on harmless bursts; avg_wait_time alone lags and averages over bursts.
• Baseline the wait/query/xact trio. The diagnostic split only works if you know normal. Track avg_wait_time, avg_query_time, and avg_xact_time per database over time and alert on deviation from baseline, not just absolute thresholds.
• Watch headroom, not just pain. sv_idle = 0 with cl_waiting = 0 is one slow query away from a cascade. Trend sv_active / pool_size; above 85% sustained, grow capacity before the cliff.
• Upgrade to 1.23.0 or later. The pre-1.23.0 avg_wait_time calculation is misleading enough to hide real saturation and invent fake saturation. On older versions, alert on maxwait instead.
• Size against PostgreSQL. Re-derive the pool-size budget whenever you add PgBouncer instances or application capacity, and keep total pool capacity below 80% of PostgreSQL max_connections.
• Audit for idle-in-transaction in application code, especially ORM transaction scoping, before it shows up as a queue.

How Netdata helps

• Netdata collects PgBouncer stats per database and graphs avg_wait_time alongside avg_query_time and avg_xact_time, so the “pool or backend” split is one glance instead of three admin-console queries during an incident.
• Per-second sampling of cl_waiting and maxwait catches short queuing bursts that a 60-second stats_period average smooths away.
• Server connection state breakdowns (sv_active, sv_idle, sv_login) per pool make it visible when one (database, user) pool saturates while the aggregate looks fine.
• Process-level CPU per core exposes single-thread event loop saturation, the cause that pool tuning cannot fix.
• Historical retention lets you compare today’s wait-time baseline against last week’s, which is how you catch slow creep before it becomes a queue.

Related guides

• How PgBouncer actually works in production: a mental model for operators
• PgBouncer monitoring checklist: the signals every connection pooler needs
• PgBouncer monitoring maturity model: from survival to expert
• PgBouncer pool exhaustion: clients queue, wait times climb, and the retry cascade