Disk space on Netdata

How to monitor the disk usage on your infrastructure

Tue, 25 Oct 2022 00:00:00 +0000

The most important part of disk usage monitoring is to check the utilization of each filesystem and each mount point which can reveal existing or impending issues with the storage space on your infrastructure.

What Does Disk Usage (DU) Mean?

Disk usage (DU) refers to the portion or percentage of computer storage that is currently in use. It contrasts with disk space or capacity, which is the total amount of space that a given disk is capable of storing. Disk usage is a crucial metric to any computing system, as it gives the user the information needed not only for storage, but also software requirements and overall operation. Although it usually refers to a computer’s hard disk, it may also refer to external storage, such as a USB drive or compact disc (CD).

PostgreSQL disk full: emergency recovery and root cause analysis

Mon, 01 Jan 0001 00:00:00 +0000

When df -h shows 100% utilization on the PostgreSQL data volume, queries fail with “could not write to file”. If pg_wal fills, the server enters PANIC and refuses to restart until space is freed. The fastest way to make the incident worse is to delete WAL files from pg_wal manually. PostgreSQL needs those files for crash recovery; removing them causes data inconsistency that forces a restore from backup. Identify which subsystem is consuming space, reclaim it safely, and fix the root cause before the cycle repeats.

Reading docker system df: where Docker disk usage actually lives

Mon, 01 Jan 0001 00:00:00 +0000

You run docker system df but the numbers do not add up to what df -h reports. Maybe the Build Cache row is empty while /var/lib/docker/buildkit/ consumes tens of gigabytes. Maybe RECLAIMABLE is high but docker system prune barely frees space because overlay2 layer sharing masks the real unique cost. Or the daemon returns Error response from daemon when the disk is already full. This guide shows how to read docker system df precisely, what it hides, and how to triage the real consumers on the host.

vCenter /storage/seat full: stats, events, alarms, and tasks outgrowing their partition

Mon, 01 Jan 0001 00:00:00 +0000

The /storage/seat partition on the vCenter Server Appliance (VCSA) holds the vPostgres tables for Stats, Events, Alarms, and Tasks: vpx_event, vpx_event_arg, vpx_task, and the vpxd_hist_stat* rollup tables. In modern VCSA it is a dedicated mount, so it can fill while /storage/db, /storage/log, and / all show healthy utilization. Operators checking only / or the VAMI dashboard’s “VCDB” usage will miss it until vpxd refuses to start.

When /storage/seat crosses 95% utilization, vpxd refuses to come up to avoid database corruption. Without vCenter: DRS stops scheduling, vMotion is gone, HA cannot be reconfigured, no provisioning, no management operations. VMs on ESXi hosts keep running because the data plane is independent of vCenter, but everything that touches vCenter is broken.