$ guides / lvm / lvm-dmeventd-not-running ▌

Operations Guides

LVM dmeventd not running: the auto-extend safety net is offline

You found this page because systemctl is-active lvm2-monitor.service returned something unexpected, or pgrep dmeventd came back empty, or lvs -o+seg_monitor showed “not monitored” on a thin pool you assumed was protected.

Here is the uncomfortable part: nothing is broken right now. No errors in lvs. No failed services your alerting noticed. No I/O hangs. The system looks healthy, and it is, right up until the moment a thin pool hits 100%, a mirror leg dies, or a snapshot overflows. Then you discover, during the incident, that the daemon which was supposed to fire the safety net has been dead for weeks.

This article covers what dmeventd actually protects, how to confirm whether it is running and whether your volumes are registered with it, and how to restore monitoring without disrupting production.

What this means

dmeventd is the device-mapper event daemon. LVM registers certain logical volumes with it, and the kernel’s device-mapper subsystem notifies dmeventd when those devices cross thresholds or change state. It is the only component in the LVM stack that reacts to events in real time. Everything else (lvs, vgs, your cron jobs) is polling.

Three safety nets depend on it:

Thin pool auto-extend. When a thin pool’s data or metadata usage crosses the configured threshold, dmeventd runs lvextend --use-policies to grow the pool before it fills. Without dmeventd, the pool grows to 100% and every thin LV in it stops accepting writes. With the default queue-on-full behavior, writes queue in the kernel for the no_space_timeout window (60 seconds by default) and then fail. Processes pile up in D state. If your root filesystem or a database lives in that pool, this is a full outage.
Mirror and RAID failure events. When a leg of a mirrored or dm-raid volume fails, dmeventd handles the event. Without it, the array runs degraded and you get no signal from the LVM layer. You keep operating on one leg with zero redundancy, and the second failure is total data loss.
Snapshot overflow handling. Traditional COW snapshots are monitored so overflow events are caught. An overflowing snapshot is invalidated instantly and irreversibly, taking your restore point with it.

The trap is that the daemon being down removes your reaction capability, not your current functionality. This is one of the most common gaps found in postmortems of thin pool exhaustion incidents.

Common causes

Cause	What it looks like	First thing to check
lvm2-monitor.service never enabled	Service inactive or disabled since install; dmeventd never started at boot	`systemctl is-enabled lvm2-monitor.service`
Service failed at boot	Status shows failure, often a dependency error on dm-event.socket	`journalctl -u lvm2-monitor.service -b`
dmeventd crashed	Service shows `active (exited)` (normal for a oneshot), but `pgrep dmeventd` returns nothing	`pgrep -x dmeventd`
Monitoring never enabled on the LV	dmeventd runs, but the pool shows “not monitored” in `seg_monitor`	`lvs -o+seg_monitor`
Threshold left at the default	dmeventd runs, pool is monitored, but `thin_pool_autoextend_threshold` is 100, so auto-extend is disabled	`grep thin_pool_autoextend /etc/lvm/lvm.conf`
Auto-extend configured but VG is full	dmeventd fires and fails because there are no free extents to extend into	`vgs -o vg_name,vg_free`
Container-based LVM	LVM tooling runs inside a container that cannot reach the host’s dmeventd; monitoring silently disabled	Where does `lvm` actually execute?

One nuance that trips people up: lvm2-monitor.service is a oneshot unit. It runs vgchange --monitor y to register eligible LVs with dmeventd, then exits. A status of active (exited) is normal and does not mean monitoring stopped. The actual daemon is a separate long-running process. This is why you must check both the service and the process.

Quick checks

All read-only. Run these before changing anything.

# 1. Is the monitoring service active and enabled?
systemctl is-active lvm2-monitor.service
systemctl is-enabled lvm2-monitor.service

# 2. Is the daemon process actually alive? (this is the real test)
pgrep -x dmeventd

# 3. Which LVs are registered for monitoring?
lvs -o lv_name,vg_name,seg_monitor

# 4. Is auto-extend actually configured? (default threshold is 100 = disabled)
grep -E 'thin_pool_autoextend_threshold|thin_pool_autoextend_percent' /etc/lvm/lvm.conf

# 5. If auto-extend fired, could it succeed? Is there VG headroom?
vgs -o vg_name,vg_size,vg_free

# 6. How close is the pool to needing it?
lvs -o lv_name,data_percent,metadata_percent

# 7. What has dmeventd been saying? (it logs auto-extend attempts to syslog)
journalctl -t dmeventd --since "7 days ago" | tail -50

Two important interpretations:

Check 2 is the ground truth. The oneshot service can report active (exited) while the daemon is dead. Only pgrep -x dmeventd tells you the process is alive.
Check 4 is the trap inside the trap. The default thin_pool_autoextend_threshold is 100, which disables auto-extend entirely. Many operators have a running dmeventd and monitored pools and are still unprotected, because nobody ever set the threshold below 100. The minimum effective value is 50; values below 50 are treated as 50. thin_pool_autoextend_percent defaults to 20, meaning each triggered extension grows the pool by 20% of its current size.

How to diagnose it

Work through these in order. Each step narrows which layer of the safety net is missing.

Confirm the daemon state. Run pgrep -x dmeventd. Empty output means no monitoring is happening anywhere on this host, regardless of what any service unit says. If it returns a PID, the daemon is alive and you move to checking registration and configuration.
Check per-volume registration. Run lvs -o lv_name,vg_name,seg_monitor. Any thin pool, mirror, RAID volume, or traditional snapshot showing “not monitored” is unprotected even with a healthy daemon. Registration is done per-LV via lvchange --monitor y, or in bulk by the lvm2-monitor service at boot.
Verify the auto-extend policy. Read /etc/lvm/lvm.conf for thin_pool_autoextend_threshold and thin_pool_autoextend_percent. If the threshold is 100 (or unset, which means the default of 100), auto-extend is disabled and dmeventd will only emit warnings, not act. The dmeventd thin plugin logs warnings to syslog at 80%, 85%, 90%, and 95% pool usage, so even a fully “working” default setup gives you log lines and nothing else.
Verify the extension can succeed. Auto-extend consumes VG free extents. Run vgs -o vg_free. If the VG has no free space, dmeventd will attempt the extension, fail, and retry with increasing backoff while the pool keeps filling. This is a classic silent failure: auto-extend “configured” but unable to fire. Check syslog for failed lvextend attempts from dmeventd.
Correlate with current exposure. Run lvs -o lv_name,data_percent,metadata_percent and lvs -o lv_name,copy_percent,lv_attr for mirrors/RAID. If any pool is above 80% data or 75% metadata, or any mirror shows copy_percent < 100 or a dead leg, your unprotected window is not theoretical. Restore monitoring before doing anything else.

flowchart TD
  A[pgrep dmeventd returns PID?] -->|no| B[Start lvm2-monitor.service]
  A -->|yes| C[lvs -o+seg_monitor: all pools monitored?]
  B --> C
  C -->|no| D[lvchange --monitor y on affected LVs]
  C -->|yes| E[thin_pool_autoextend_threshold below 100?]
  D --> E
  E -->|no| F[Set threshold in lvm.conf, e.g. 80]
  E -->|yes| G[vgs: free extents for extension?]
  F --> G
  G -->|no| H[Add PV or free VG space - auto-extend cannot fire]
  G -->|yes| I[Safety net restored]
  H --> I

Metrics and signals to monitor

Signal	Why it matters	Warning sign
dmeventd process presence	The entire event-driven safety net depends on this one process	`pgrep -x dmeventd` empty on any host with thin pools, RAID, or snapshots
seg_monitor per LV	Daemon running does not mean your specific pool is registered	Any thin pool, RAID, or snapshot LV showing “not monitored”
thin_pool_autoextend_threshold	Default 100 silently disables auto-extend	Value of 100 (or unset) on a host relying on auto-extend
Thin pool data_percent	Countdown to write failure; auto-extend is the mitigation you just lost	Above 80% with monitoring offline
Thin pool metadata_percent	Metadata exhaustion is harder to recover than data exhaustion	Above 75% with monitoring offline
VG free space	Auto-extend consumes VG extents; no free space means the extension fails silently	VG free below what one or two extension cycles would consume
Mirror/RAID copy_percent and leg health	Without dmeventd, leg failures produce no LVM-layer signal	copy_percent < 100 or any dead leg on a host where dmeventd is down
dmeventd syslog output	Shows auto-extend attempts, successes, failures, and threshold warnings	Failed extension attempts, or repeated threshold warnings with no extension

Fixes

Restore the daemon

# Enable and start the monitoring service
systemctl enable --now lvm2-monitor.service

# Confirm the daemon is actually running afterwards
pgrep -x dmeventd

Starting lvm2-monitor.service is safe. It registers eligible LVs with dmeventd and exits; it does not suspend or reload device tables. If the service fails to start, check journalctl -u lvm2-monitor.service -b. A common cause on minimal installations is a dependency failure on dm-event.socket. On older RHEL 7 systems, the service could fail when exported volume groups were present. If you are on legacy RHEL 7, check your lvm2 package version before assuming this is not your bug.

Register volumes that show “not monitored”

# Enable monitoring on a specific pool
lvchange --monitor y <vg>/<thinpool>

# Verify
lvs -o lv_name,seg_monitor <vg>/<thinpool>

This is safe and takes effect immediately. Do this for any thin pool, RAID volume, or snapshot that showed “not monitored” in your checks.

Fix the auto-extend configuration

In /etc/lvm/lvm.conf:

thin_pool_autoextend_threshold = 80
thin_pool_autoextend_percent = 20

With these values, dmeventd extends the pool by 20% of its current size when usage crosses 80%. The threshold’s minimum effective value is 50. Configuration changes in lvm.conf are read by dmeventd for subsequent events; restarting the service after editing is the conservative way to ensure the new policy is in effect.

Give auto-extend room to work

Auto-extend is only as good as the VG headroom behind it. Each extension consumes current_pool_size x autoextend_percent worth of free extents. Keep enough VG free space for at least two auto-extend cycles plus one manual extension. If the VG is already full, restoring dmeventd changes nothing. You need vgextend with a new PV first, which is a separate operation with its own risks. See the related guide on extending when the VG is full.

If you cannot restore dmeventd right now

On hosts where the daemon genuinely cannot run (some container-based LVM deployments cannot reach the host’s dmeventd), you are back to polling. That means tight external alerting on data_percent, metadata_percent, snapshot usage, and RAID sync state, collected frequently and trended, because nothing on the host will react for you. Treat every saturation threshold as if auto-extend does not exist, because it does not.

Prevention

Alert on the daemon, not just the data. A process-existence check for dmeventd on every host with thin pools, RAID, or snapshots costs nothing and catches this entire failure class.
Alert on seg_monitor drift. A pool that flips from “monitored” to “not monitored” (after LV recreation, restore, or a service failure at boot) should get someone’s attention before the pool fills.
Audit the threshold. Include thin_pool_autoextend_threshold in configuration management. The default of 100 is a loaded foot-gun; every thin pool host should have an explicit value below 100, and your config management should flag hosts that do not.
Watch the VG headroom behind the policy. Auto-extend readiness is a three-part condition: daemon running, LV monitored, VG has free extents. Alert on the conjunction, not on any single part.
Test the safety net once. Teams configure auto-extend and never verify it fires. On a non-production pool, grow usage past the threshold and confirm the extension happens and appears in syslog. The first real test should not be a production incident.
Trend the saturations the daemon protects. Even with dmeventd healthy, data_percent and metadata_percent trends tell you whether auto-extend is keeping up or merely delaying exhaustion. A pool that needs an extension every week has a capacity problem, not a monitoring problem.

How Netdata helps

Daemon liveness as a first-class signal. Netdata’s process monitoring tracks dmeventd presence and resource usage per-second, so a dead daemon raises an alert in seconds instead of being discovered during the next pool exhaustion.
Thin pool data and metadata percentages, trended. Continuous collection of pool usage gives you the runway calculation that tells you how long your unprotected window has been and how long you have left.
Correlation across the readiness chain. dmeventd down + pool at 85% + VG free near zero is a very different page than dmeventd down + pool at 40%. Seeing process state, pool saturation, and VG headroom on one dashboard makes that severity call immediate.
Historical context for the silent period. Because LVM’s own tools show only current state, retained per-second history is how you answer “when did the daemon die, and how much did the pool grow while nobody was watching.”
Kernel-level corroboration. D-state process counts and block device errors on dm devices sit next to the LVM signals, so when an unprotected pool does fill, you see the hang forming rather than just the aftermath.