$ guides / uwsgi / uwsgi-file-descriptor-limits ▌

Operations Guides

uWSGI file descriptor limits: raising ulimit -n and systemd LimitNOFILE

The default per-process file descriptor limit on many Linux distributions is 1024. For a uWSGI instance running multiple workers, each holding connections, sockets, log files, and database handles, that ceiling is too low for production.

File descriptor exhaustion in uWSGI is silent. When the limit is hit, accept() and open() calls fail with EMFILE. New connections are rejected with no uWSGI-level error, no log entry, and no stats counter reflecting the problem. Clients see connection resets or timeouts. The master process stays alive. Worker status looks normal. The only evidence is at the OS level.

This guide covers raising the limit durably across the service manager, the OS, and uWSGI itself, plus leak detection, headroom sizing, and verifying the running limit via /proc.

The limit hierarchy

File descriptor limits cascade through several layers. Each layer can constrain the effective limit, and the lowest value wins.

flowchart TD
    A["Kernel fs.file-max\nhost-wide ceiling"] --> B["RLIMIT_NOFILE\nper-process limit"]
    B --> C["systemd LimitNOFILE\nOR shell ulimit -n"]
    C --> D["uWSGI --max-fd\noptional internal cap"]
    D --> E["Master process\ninherits effective limit"]
    E --> F["Worker processes\ninherit at fork"]

Setting ulimit -n in your shell has no effect on a systemd-managed service. systemd applies its own limits from the unit file at service start, overriding whatever the shell environment provided. Always verify the running limit via /proc/<pid>/limits, not ulimit -n.

What proper fd limits give you

Beyond preventing EMFILE, a correctly sized limit provides:

Headroom for traffic bursts. A burst that doubles normal connection count should not approach the ceiling.
Leak detection window. A steadily climbing per-worker fd count over hours or days indicates a leak. A generous limit gives you time to detect and respond before exhaustion. A tight limit means you hit the wall first.
Coverage for all fd consumers. File descriptors are not just connections. They include the listening socket, log files, the stats server socket, spooler files, pipes, and anything the application opens directly (database connections, cache connections, temp files).

Prerequisites

Before choosing a target limit, gather these values:

Input	Where to find it	Why it matters
Worker count	uWSGI config (`--processes`) or stats `workers[]`	Each worker has its own fd table
Expected connections per worker	Traffic patterns, upstream proxy config	One fd per active connection
Non-connection fd consumers	Application code, database pool size, logging	Often overlooked, can be significant
Current effective limit	`/proc/<pid>/limits` for the running process	Baseline for sizing the increase

Checking the current limit

The shell ulimit -n shows the limit for your current shell session, not the limit of a running uWSGI process managed by systemd. The authoritative source is /proc/<pid>/limits.

# Check the limit of a running uWSGI master process
awk '/^Max open files/ {print $4}' /proc/$(cat /tmp/uwsgi.pid)/limits

# Check per-worker fd usage and effective limit
for pid in $(pgrep -P $(cat /tmp/uwsgi.pid)); do
    count=$(ls /proc/$pid/fd 2>/dev/null | wc -l)
    limit=$(awk '/^Max open files/ {print $4}' /proc/$pid/limits)
    echo "pid=$pid fds=$count limit=$limit"
done

If you do not have a pidfile, find the master PID with pgrep:

# Find the uWSGI master process
pgrep -f 'uwsgi.*master'

Setting the limit under systemd

For uWSGI running under systemd, the service unit file controls the fd limit. The LimitNOFILE directive in the [Service] section sets RLIMIT_NOFILE for the process.

The cleanest approach is a systemd override, which persists across package updates:

# Create or edit an override without modifying the original unit file
systemctl edit uwsgi

In the editor, add:

[Service]
LimitNOFILE=65536

Apply the change:

# Reload systemd unit files
systemctl daemon-reload

# Restart the service to pick up the new limit
# WARNING: this interrupts in-flight requests unless using chain reload
systemctl restart uwsgi

Setting the limit from a shell

If uWSGI is started from a shell or init script rather than systemd, set the limit before launching:

# Set the soft and hard limit for the current shell session
ulimit -n 65536

# Then start uWSGI
uwsgi --ini /etc/uwsgi/app.ini

This does not persist across reboots. For persistence without systemd, use /etc/security/limits.conf:

# /etc/security/limits.conf
uwsgi  soft  nofile  65536
uwsgi  hard  nofile  65536

/etc/security/limits.conf applies to PAM-based logins and may not affect system services started by systemd. Verify the effective limit after restart.

uWSGI’s –max-fd option

uWSGI accepts --max-fd to cap the maximum number of file descriptors it will use internally. This matters when the OS reports a very high RLIMIT_NOFILE and you want to prevent uWSGI from sizing internal data structures based on that large number.

# uWSGI configuration
max-fd = 65536

In Emperor mode, --max-fd set on the Emperor process is expected to propagate to vassals, avoiding per-vassal configuration.

--max-fd works by calling setrlimit() to set both the soft and the hard RLIMIT_NOFILE to the requested value. The uWSGI documentation states that it requires root privileges; in practice setting a value within the current hard limit works without root, while raising the hard limit above its inherited value needs root (or CAP_SYS_RESOURCE).

Emperor propagation is via rlimit inheritance: the Emperor forks vassals, so a vassal that does not set max-fd of its own inherits the Emperor’s raised RLIMIT_NOFILE. A vassal whose own configuration sets max-fd applies that value instead.

Verifying the running limit

After restarting or reloading uWSGI, verify that the new limit is in effect:

# Verify the master process limit (soft and hard)
awk '/^Max open files/ {print "soft="$4, "hard="$5}' /proc/$(cat /tmp/uwsgi.pid)/limits

# Verify a worker process limit
WORKER_PID=$(pgrep -P $(cat /tmp/uwsgi.pid) | head -1)
awk '/^Max open files/ {print "soft="$4, "hard="$5}' /proc/$WORKER_PID/limits

Workers inherit the master’s limits at fork time. If the master was started with the correct limit, workers should match.

If the limit did not change, check:

systemd: confirm the override is active. Run systemctl cat uwsgi to see the merged configuration including overrides.
shell: confirm ulimit -n was run in the same session or script that starts uWSGI.
limits.conf: confirm the PAM user matches the uWSGI service user.

The headroom rule

Keep fd usage below 50-80% of the soft limit under normal operating conditions. This provides margin for:

Traffic bursts that temporarily increase connection counts
Application code that opens additional fds during specific operations (file processing, bulk database transactions)
Slowly developing fd leaks that need time to detect before they cause incidents

The degradation curve for fd exhaustion is a cliff. Below the limit, everything works. At the limit, new accept() and open() calls fail with EMFILE. There is no gradual degradation and no uWSGI-level signal.

Sizing a target limit

Estimate peak fd usage per worker:

peak_fds_per_worker = max_concurrent_connections + database_pool_size + logging_fds + application_fds + overhead

Then apply headroom:

target_limit = peak_fds_per_worker * 1.5

The 1.5x multiplier provides 33% headroom above peak. For conservative deployments, use 2x.

Deployment profile	Suggested limit	Rationale
Small (2-4 workers, behind nginx)	65536	Comfortable headroom, low overhead
Medium (8-16 workers, direct or proxied)	65536	Sufficient for most workloads
High-connection (async/gevent, many connections per worker)	131072 or higher	Async workers multiplex many connections

Detecting fd leaks

A steadily climbing per-worker fd count that never stabilizes or decreases indicates a leak. Common causes:

mmap’d files not closed. The fd is opened, the file is mapped, but the fd is never closed after mmap returns. VSZ grows alongside fd count in this pattern.
Database connections not returned to the pool. Each leaked connection holds an fd.
Temp files opened but not closed. Common in file upload processing or report generation.
Socket connections without timeouts. Sockets that never close accumulate indefinitely.

Track the correlation between fd count and virtual memory size:

# Compare fd count and VSZ per worker over time
for pid in $(pgrep -P $(cat /tmp/uwsgi.pid)); do
    fds=$(ls /proc/$pid/fd 2>/dev/null | wc -l)
    vsz=$(awk '/^VmSize/ {print $2}' /proc/$pid/status)
    echo "$(date +%s) pid=$pid fds=$fds vsz_kb=$vsz"
done

VSZ growing alongside fd count is a strong indicator of mmap-related leaks. VSZ stable with growing fd count points to socket or pipe leaks. Either pattern warrants investigation before the limit is reached.

Common pitfalls

systemd LimitNOFILE overrides shell ulimit

The most common mistake. You run ulimit -n 65536 in your shell, restart the service with systemctl restart uwsgi, and assume the new limit is in effect. It is not. systemd applies LimitNOFILE from the unit file or override, ignoring the shell environment entirely. Always verify via /proc/<pid>/limits.

setrlimit after privilege drop

If uWSGI is configured with --uid and --gid to drop privileges after startup, and the process attempts to raise its fd limit after the drop, the setrlimit call fails with Operation not permitted. Raising the hard limit requires elevated privileges.

The fix is to ensure the limit is set before privilege drop occurs. uWSGI itself applies --max-fd during initialization, before --uid/--gid take effect, so an instance started as root can raise the limit and then drop privileges. Once the process is running as an unprivileged user, raising the hard limit fails with Operation not permitted. systemd handles this correctly because LimitNOFILE is applied before exec. For Emperor mode, set the limit on the Emperor process (which runs as root) so vassals inherit it.

High OS limits causing internal allocation overhead

Setting an extremely high RLIMIT_NOFILE can cause uWSGI to allocate oversized internal data structures, because several subsystems size tables from uwsgi.max_fd (derived from the soft limit): the poll/event queue allocates a struct pollfd array and a pointer array of max_fd entries, and the async subsystem allocates two struct wsgi_request * arrays of max_fd entries each. On a container with a limit in the billions, that alone reaches multiple gigabytes. Use --max-fd to cap uWSGI’s internal allocation independently of the OS limit.

Not accounting for all fd consumers

Database connection pools, Redis connections, log files, the stats server socket, and application-opened files all consume fds. Counting only incoming connections underestimates actual usage. Audit all fd consumers before sizing the limit. A worker with 50 concurrent connections but a 20-connection database pool and 5 log files needs at least 76 fds, not 50.

Signals to monitor

Signal	Why it matters	Warning sign
Per-worker fd count (`/proc/<pid>/fd`)	Primary utilization metric	Steady upward trend indicates a leak
fd count vs. soft limit ratio	Headroom indicator	Above 50-80% under normal load
Worker VSZ	Correlates with mmap’d fd leaks	VSZ growing in lockstep with fd count
Worker RSS	Memory pressure from leaked resources	Steady growth without recycling
System-wide fd usage (`/proc/sys/fs/file-nr`)	Host-wide fd consumption	Approaching `fs.file-max`

How Netdata helps

Per-process fd counts. Netdata collects open fd counts per process group, replacing manual /proc/<pid>/fd polling with continuous collection.
VSZ/RSS correlation. Memory metrics appear alongside fd counts. VSZ rising in lockstep with fd count signals an mmap-related leak.
Anomaly detection on fd trends. Netdata’s anomaly detection flags unusual fd growth patterns, catching acceleration before exhaustion.
Per-second resolution. fd leaks can accelerate nonlinearly. Per-second collection catches the inflection point where growth rate changes.
System-wide fd context. Netdata tracks /proc/sys/fs/file-nr alongside per-process metrics, so you can distinguish a uWSGI-specific problem from host-wide fd pressure.