ZFS capacity planning: runway estimation before the pool fills

ZFS does not fail gracefully at full. Long before ENOSPC, the metaslab allocator starts working harder to find free space, write latency climbs, TXG syncs stretch, and the pool slides into the capacity-fragmentation cliff described in how ZFS actually works in production. By the time applications see errors, you are already in emergency territory, and the recovery options (destroying snapshots under I/O pressure, expanding a pool mid-incident) are the worst versions of themselves.

The fix is boring: measure the allocation rate, divide free space by it, and act on the answer while you still have weeks. Most teams get this wrong because naive capacity math uses the wrong inputs. Pool-level FREE is not the space you can write to. Snapshots hold blocks that file deletion does not release. Compression ratio drift changes the physical cost of the same logical growth. And space you “freed” yesterday may still be sitting in the asynchronous reclaim backlog.

This article is the working reference for computing real ZFS runway: which numbers to collect, how to adjust them, and the headroom thresholds that should trigger planning, action, and emergency response.

Why naive capacity math fails on ZFS

Four ZFS-specific behaviors break the “used divided by total, extrapolate the slope” approach that works on simpler filesystems:

Pool free is not writable free. zpool list reports raw pool allocation and free space. The number that gates new writes is the per-dataset available property from zfs list or zfs get available, which accounts for quotas, reservations, and the slop reserve. ZFS reserves roughly 3.125% of pool size (floor 128 MiB, capped at 128 GiB since OpenZFS 2.1) as slop space for metadata and administrative operations. Regular writes fail when free space drops below the slop threshold. Pool-level FREE does not reflect this; available does. Runway computed from zpool list FREE is systematically optimistic.

Snapshots hold deleted data. Copy-on-write means a snapshot pins every block it references. Deleting 500 GB of files frees nothing if a snapshot references those blocks. Your pool can fill on a flat dataset because snapshot retention is growing underneath it.

Compression drift changes the exchange rate. zpool list shows post-compression allocation. If incoming data is less compressible than historical data (a common shift when a workload starts writing media, encrypted, or pre-compressed data), raw space consumption accelerates while logical growth looks unchanged. A declining compressratio means your runway is shrinking faster than the write rate suggests.

Reclaim is asynchronous. Destroying a snapshot or dataset does not return space instantly. The bytes move into the pool’s freeing property and are reclaimed in the background. During heavy reclaim, effective free space is lower than it looks, and on a nearly-full pool the reclaim work itself competes for the I/O bandwidth you need to survive.

The signals you need first

Collect these before computing anything. All commands are read-only.

# Pool-level allocation, capacity, fragmentation, and async reclaim backlog
zpool list -H -o name,size,alloc,free,cap,frag,freeing

# Per-dataset writable space and usage (this is the number that gates writes)
zfs list -o name,used,avail -r tank

# Space breakdown: live data vs snapshots vs reservations vs children
zfs list -o space -r tank

# Snapshot space per dataset
zfs get -r usedbysnapshots tank

# Snapshots ranked by space held
zfs list -t snapshot -o name,used,refer -s used -r tank | tail -20

# Compression trend per dataset
zfs get -r compressratio,logicalused,used tank

The columns that matter for runway:

One sample of each tells you where you are. Runway requires the trend, so sample on a fixed cadence (daily is enough for capacity work; hourly if you are already in the action zone) and keep history.

Computing the daily allocation rate

Measure physical allocation, not logical writes. Logical bytes lie because of compression.

# Daily allocation rate: sample allocated bytes at the same time each day
zpool list -Hp -o name,alloc,free

# rate(bytes/day) = (alloc_today - alloc_7_days_ago) / 7

Use a 7-day window minimum. A single day is dominated by whatever batch job ran. Two to four weeks of daily samples gives you both the baseline rate and, more importantly, whether the rate itself is accelerating. An accelerating weekly growth rate is the earliest warning you get.

Watch for step changes rather than smooth slopes. A new tenant, a new backup schedule, a replication target pointed at the pool: these reset the rate overnight, and any runway computed on pre-change data is fiction.

Adjust the raw rate for compression drift. If compressratio on a dataset fell from 1.8x to 1.4x over the quarter, the same logical ingest now costs about 29% more physical space. Either compute the rate from alloc (which already absorbs this) and accept that it bakes in recent drift, or model it explicitly if the data mix is still changing.

The runway formula

The base formula:

Runway (days) = free space (bytes) / daily allocation rate (bytes/day)

Apply three corrections before trusting the number:

1. Use available, not pool FREE. For the dataset(s) that actually grow, take zfs get available. On a pool with reservations or one approaching the slop boundary, the difference is material.
2. Subtract the freeing backlog. If zpool get freeing shows a large pending reclaim, that space is spoken for operationally even though it will eventually return. Treat it as unavailable until the backlog drains. A freeing value that stays large for days means async reclaim is falling behind, which is itself a warning sign on a busy pool.
3. Model snapshot growth separately. Snapshots do not grow linearly with writes. As the active dataset diverges from a retained snapshot, that snapshot’s held space grows even if write volume is flat. See the next section.

flowchart LR
  A[Dataset write growth] --> R[Net daily allocation rate]
  S[Snapshot retention minus pruning] --> R
  C[Compressratio drift] --> R
  F[freeing backlog] --> AV[Effective free bytes]
  V[zfs available] --> AV
  R --> RW[Runway in days]
  AV --> RW
  RW --> P{Headroom policy}
  P --> H1[Comfortable: 30%+ free]
  P --> H2[Acceptable: 15-25% with plan]
  P --> H3[Emergency: under 5%]

Model the growth components separately

A single blended rate hides the failure modes that actually bite. Model dataset growth plus snapshot retention minus scheduled pruning as separate terms.

Dataset growth is the easy term. Trend usedds (or used minus usedbysnapshots) per growing dataset. This is usually the stable, forecastable component.

Snapshot retention is the term that surprises people. Estimate it per dataset as usedbysnapshots trended over time, and sanity-check it against your retention policy. If you keep 30 daily snapshots of a dataset with 2% daily churn, steady-state snapshot space is roughly 30 times the daily churn for that dataset, and it only stabilizes once the retention window is full. A pool that is three weeks into a new 30-day snapshot schedule has not seen its steady state yet.

Scheduled pruning is the subtraction term, and it only counts if pruning is actually running. Verify: compare snapshot counts against what the policy should produce (zfs list -t snapshot -o name,creation -s creation), and confirm the oldest snapshots are disappearing on schedule. A pruning cron that has been failing silently for a month converts your model from “growth minus pruning” to just “growth.”

Clones and holds block reclaim entirely. A clone pins its origin snapshot; a zfs hold tag blocks destruction. If large snapshots refuse to age out, check both before concluding your growth model is wrong.

Headroom thresholds and what each one demands

These map to concrete operational posture, not vibes.

Between comfortable and emergency sit the action triggers that keep you out of the under-5% row: 75-80% full is the planning threshold (evaluate growth, plan expansion), 85-90% is the action threshold (prune snapshots, execute the expansion), and 96% with active writes and no reclaim in progress is a page.

Two things move a pool from one row to the next faster than the average rate suggests: snapshot schedules reaching steady state, and compressratio declines. Both are why the trend of the rate matters as much as the rate.

Why the last 20 percent is not like the first 80

Runway math assumes degradation is linear until full. ZFS is not. The degradation mechanism is per-metaslab: when an individual metaslab drops below 4% free (metaslab_df_free_pct), its allocator switches from first-fit (fast) to best-fit (slow). As the pool fills, more metaslabs cross that line, and write latency and TXG sync duration climb non-linearly. The commonly cited “80% rule” is a conservative approximation; the real threshold is workload-dependent. Write-heavy random workloads feel it earlier; mostly-sequential workloads tolerate more.

Fragmentation compounds this. Fragmentation rises as the pool fills, and ZFS cannot be defragmented in place. A scrub does not defragment. The only fix for a badly fragmented pool is zfs send | zfs recv into a new pool, which is exactly the kind of project you cannot execute during a capacity emergency. Trend frag alongside capacity: fragmentation rising while capacity climbs means the cliff arrives before your runway math says it should.

The practical consequence: do not let runway approach zero before acting, because the pool’s usable performance runs out before its bytes do. Your planning trigger should fire at the 75-80% boundary, and the expansion or migration should complete while the pool is still in the comfortable or acceptable band. For the full failure mechanics of the full-pool endgame, including the slop reserve and why deletion itself can fail, see ZFS No space left on device.

A worked pass in fifteen minutes

Once a week, or daily once you are past 75%:

1. Snapshot the state. Record zpool list -Hp -o name,alloc,free,cap,frag,freeing, per-dataset available, and usedbysnapshots totals.
2. Update the rate. Recompute the 7-day and 28-day allocation rates. Flag if the 7-day rate exceeds the 28-day rate by more than half: the growth is accelerating.
3. Check the model terms. Dataset growth on trend? Snapshot space matching retention-policy expectations? Pruning running? Freeing backlog draining?
4. Compute runway from available minus the freeing backlog, divided by the current rate.
5. Check compression. Is any growing dataset’s compressratio declining? If yes, bias the rate upward.
6. Act on the threshold. Runway under 90 days: start the expansion or archival project. Under 30 days: it is a project with a deadline. Under 7 days to 96% full: open a ticket and act this shift.

Signals to watch in production

For where these fit in a broader monitoring practice, see the ZFS monitoring checklist and the monitoring maturity model, which places capacity growth trending and runway estimation at Level 3.

How Netdata helps

Runway estimation fails when it depends on someone remembering to run zpool list weekly. Netdata collects the ZFS capacity signals continuously, so the trend exists before you need it:

• Pool allocation, capacity, and fragmentation over time, so the growth rate and its acceleration are visible on a chart instead of in a spreadsheet.
• The freeing backlog alongside free space, so you can see whether reclaim is keeping up or silently falling behind.
• Per-dataset space usage, which helps separate “dataset is growing” from “snapshots are pinning deleted data” when combined with usedbysnapshots.
• TXG sync duration and pool latency correlated against fill level, so you can see the allocator-pressure performance cost arriving before capacity is exhausted.
• Threshold alerts on capacity crossing the planning and action bands, so the 75-80% trigger fires whether or not anyone ran the weekly pass.

The value is correlation: capacity climbing, snapshot space climbing, and TXG sync stretching in the same window tells you which term of the growth model is misbehaving, in one view.

Related guides

• ZFS No space left on device: ENOSPC, the slop reserve, and the pool you cannot delete from
• How ZFS actually works in production: a mental model for operators
• ZFS monitoring checklist: the signals every production pool needs
• ZFS monitoring maturity model: from survival to expert
• ZFS pool ONLINE with non-zero errors: why zpool status -x lies