$ guides / uwsgi / uwsgi-too-many-open-files ▌

Operations Guides

uWSGI too many open files: file descriptor exhaustion and EMFILE

When the uWSGI process hits its file descriptor limit (RLIMIT_NOFILE), accept() and open() start returning EMFILE (errno 24). New connections are silently rejected. Logging fails. Application code throws exceptions that look like disk or network failures. The uWSGI stats endpoint does not report file descriptor usage, so there is no uWSGI-level signal pointing to the real problem. Operators typically chase disk space, I/O, or network connectivity before realizing the process has simply run out of file descriptors.

What this means

Every TCP connection, UNIX socket, log file, stats endpoint connection, spooler task file, database connection, and application file costs one file descriptor. The kernel enforces a per-process soft limit. When that limit is reached, accept(), open(), socket(), pipe(), and any other fd-allocating syscall returns EMFILE.

The cascade is what makes this hard to diagnose. Workers appear idle because accept() fails before they can pick up new connections, but existing connections may still be processing, so throughput does not drop to zero immediately. Logging failures look like disk problems. Application open() failures look like permission errors.

flowchart TD
    A["fd count reaches ulimit -n"] --> B["accept/open/socket fail EMFILE"]
    B --> C["Connections rejected"]
    B --> D["Logging stops working"]
    B --> E["App open() calls throw"]
    C --> F["Looks like network failure"]
    D --> G["Looks like disk failure"]
    E --> H["Looks like permission error"]
    F --> I["Misdiagnosed as disk or network"]
    G --> I
    H --> I

Common causes

Cause	What it looks like	First thing to check
Database connection pool leak	Workers accumulate socket fds over time; count grows monotonically per worker	Per-worker fd count over time; compare workers that have served many requests vs recently respawned ones
File handle leak in app code	Workers accumulate regular file fds; `lsof` shows many open handles to the same paths	`ls -la /proc/<pid>/fd` and look for repeated file paths
Deleted-but-open files	fds point to `(deleted)` entries: files were unlinked but never closed	`ls -la /proc//fd
Too many workers times connections per worker	Total fds across all workers approaches the per-process limit under load	Sum fds across all workers; compare to `ulimit -n`
Subprocess fd leaks	Workers have pipe fds to child processes that exited but were not reaped	`lsof -p <pid>` and look for pipe or chr fd types
Log rotation without fd close	Old log fds remain open after rotation; count jumps at each rotation cycle	fd count jumps at each log rotation interval

Quick checks

All commands below are read-only and safe on production systems. Replace the pidfile path with your --pidfile directive.

# Soft limit for the current shell (not necessarily what uWSGI inherited)
ulimit -n

# Actual limit for a running uWSGI master process
cat /proc/$(cat /tmp/uwsgi.pid)/limits | grep "Max open files"

# Count open fds for the master process
ls /proc/$(cat /tmp/uwsgi.pid)/fd | wc -l

# Count open fds per worker, with soft limit for comparison
for pid in $(pgrep -P $(cat /tmp/uwsgi.pid)); do
    count=$(ls /proc/$pid/fd 2>/dev/null | wc -l)
    limit=$(awk '/^Max open files/ {print $4}' /proc/$pid/limits)
    echo "pid=$pid fds=$count limit=$limit"
done

# Find deleted-but-open file descriptors (leaked fds after unlink)
ls -la /proc/<worker_pid>/fd | grep "(deleted)"

# System-wide fd usage vs kernel limits
cat /proc/sys/fs/file-nr

# Detailed fd listing with lsof (slower; use /proc/<pid>/fd for exact counts)
lsof -p <worker_pid>

How to diagnose it

Confirm the limit. Check /proc/<pid>/limits for the actual RLIMIT_NOFILE value. In containerized deployments, verify what the container runtime actually set. systemd services do not read /etc/security/limits.conf; use LimitNOFILE in the unit file or a drop-in override.
Measure per-process usage. Run ls /proc/<master_pid>/fd | wc -l and per-worker using pgrep -P. If usage is above 80% of the soft limit, you are in the danger zone.
Establish whether the count is growing. A single measurement tells you the current state. Two measurements over time tell you whether there is a leak. If the count grows monotonically per worker and never drops until respawn, you have a leak. Sample at 30-second intervals during active traffic.
Categorize the open fds. Use ls -la /proc/<pid>/fd to determine what type is dominating. Socket fds point to connection pools. Regular file fds point to file handling. Pipe fds point to subprocess management. Deleted fds point to unlinked-but-unclosed files.
Check for worker-to-worker divergence. If one worker has significantly more fds than others, that worker hit a code path that leaks. If all workers grow at the same rate, the leak is in shared initialization code or connection pool setup common to every worker.
Correlate with respawn cycles. If max-requests is configured, workers respawn periodically. If fd count resets to baseline after respawn and then grows again, the leak is per-worker (in request handling). If it persists across respawns, the leak is in the master or inherited across forks.
Verify it is not the high-fd-limit memory blowup. If uWSGI runs in a container with a very high fd limit (some runtimes set RLIMIT_NOFILE to very large values), uWSGI may allocate internal data structures sized by that detected limit, causing memory exhaustion rather than fd exhaustion. The symptom is OOM kills at startup, not EMFILE during traffic. Check startup logs for “detected max file descriptor number” and inspect the reported value. If it is in the billions, this is your problem.

The underlying issue remains unfixed upstream: unbit/uwsgi issue #2299 (“High memory usage if fs.nr_open is very high”) is still open, the commits referenced in its thread are in downstream projects, and no mitigation appears in the 2.0.29-2.0.31 changelogs. --max-fd (or a sane RLIMIT_NOFILE) remains the fix.

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Per-worker fd count (`/proc/<pid>/fd`)	Directly tracks the resource being exhausted	Growth above 80% of soft limit, or monotonic growth without reset
Per-worker fd count trend (rate of change)	Distinguishes a leak from a temporary spike	Sustained positive slope over hours with no plateau
`RLIMIT_NOFILE` from `/proc/<pid>/limits`	The ceiling; must be known to interpret fd count	Soft limit below what workers need under peak load
System-wide fd usage (`/proc/sys/fs/file-nr`)	Rules out system-level fd exhaustion	Allocated count approaching system-wide limit
Worker RSS	Correlates with fd-driven memory issues	RSS growing in lockstep with fd count
Exception rate (from uWSGI stats)	Application errors from failed open/socket calls	Spike with no deployment change or downstream event
Write errors (from uWSGI stats, per-core)	Client disconnects from rejected connections	Spike correlating with external connection failure reports

Fixes

Raise the file descriptor limit

The immediate mitigation is to raise ulimit -n. How depends on how uWSGI is started:

Standalone (shell): Set ulimit -n 65536 in the shell before starting uWSGI.
systemd: Set LimitNOFILE=65536 in the [Service] section of the unit file or a drop-in override (systemctl edit uwsgi). The file /etc/security/limits.conf does not apply to systemd-managed services.
Docker: Use --ulimit nofile=65536:65536 in docker run or the equivalent in the compose file. Verify inside the container with cat /proc/1/limits because different runtimes apply different defaults.

Raising the limit does not fix a leak. It buys time. Use the raised limit as breathing room while you find and fix the leak.

Fix file descriptor leaks in application code

If fd count grows monotonically per worker and resets on respawn, the application is leaking fds. Common patterns:

Opening files without closing them (missing close() calls, not using context managers like with open(...) as f:).
Database connections borrowed from a pool but never returned. This usually happens when exception handling exits the function before the connection is released.
Subprocess pipes not closed after the child exits. Each subprocess.Popen call that is not properly cleaned up leaks two fds (stdin and stdout pipes).
Temporary files created with mkstemp or NamedTemporaryFile but never unlinked and closed.

To localize the leak, run ls /proc/<pid>/fd | wc -l before and after hitting specific endpoints. The endpoint that causes the count to grow without recovering is the culprit.

Tune connection pool sizes

Each database or downstream API connection holds a socket fd. If each worker maintains a pool of N connections, the steady-state fd cost from pools alone is workers x N, plus per-request connections, logging, and the stats server. If workers x pool_size is already a significant fraction of the fd limit, reduce the pool size per worker or raise the limit.

Use –max-fd to cap internal allocation

The --max-fd option tells uWSGI the maximum number of file descriptors to plan for internally. This is useful when the process inherits an extremely high fd limit from the container runtime or systemd (values in the millions or higher). uWSGI allocates internal arrays sized by the detected limit. Setting --max-fd 65536 caps those allocations without changing the kernel limit.

Enable close-on-exec

The close-on-exec option sets FD_CLOEXEC on file descriptors so they are not inherited by subprocesses. Without it, every fd the worker has open at fork() + exec() time is duplicated into the child. If the application spawns subprocesses frequently (shellouts, worker pools, image processing), child processes accumulate copies of the parent’s fds.

Use max-requests as a safety valve

If you cannot immediately find the leak, configure --max-requests so workers recycle after serving N requests. This resets the fd count periodically. Each recycling event increments respawn_count in the stats. This is a band-aid: the leak still occurs within each cycle, and workers near the end of their cycle operate with the highest fd counts. Set max-requests high enough to avoid excessive churn but low enough that fd count never approaches the limit.

Prevention

Monitor per-worker fd count at the OS level. This is the only signal that catches fd exhaustion before it becomes an outage. Without it, you are blind to the primary resource exhaustion mode that uWSGI cannot report itself.
Verify the fd limit is actually applied. Check /proc/<pid>/limits on the running process. 65536 is a reasonable production starting point, but the value set in configuration may differ from what the process actually inherits from the container runtime or service manager.
Audit connection pool sizing. Calculate workers x pool_size and ensure headroom for request-handling fds, logging, stats server, and spooler files.
Track fd count trends, not just snapshots. A point-in-time value of 500 open fds is meaningless without knowing the slope. Collect at regular intervals and alert on sustained growth.
Watch for deleted-but-open fds. These accumulate silently and only appear in /proc/<pid>/fd with a (deleted) suffix. A growing count is a definitive leak in file handling code.

How Netdata helps

Per-process file descriptor monitoring. Netdata collects open fd count per process from /proc/<pid>/fd, providing continuous visibility without manual polling. This is the signal uWSGI does not expose.
Correlation with uWSGI worker metrics. When fd count rises alongside worker busy ratio, response time, or exception rate, the correlation is visible on a single timeline.
Trend and anomaly detection. Per-second granularity and ML-based anomaly detection catch slow fd leaks (growth over hours or days) that point-in-time manual checks miss.
Memory pressure detection. Per-process RSS monitoring and OOM event tracking surface the memory blowup from oversized internal allocations before the process is killed.
System-wide fd usage. Netdata monitors /proc/sys/fs/file-nr, so you can distinguish per-process exhaustion from system-wide limits.