LVM dm-N device numbers change after reboot: use /dev/mapper, not /dev/dm-N

After a reboot, you run lsblk or df -h and notice your logical volumes have different /dev/dm-N numbers. What was dm-0 before the reboot is now dm-2, and what was dm-1 is now dm-0. If anything on the system references /dev/dm-0 directly, it may now point at the wrong volume.

This is not a bug and not a sign of corruption. Device-mapper minor numbers are assigned dynamically at activation time based on the order devices are discovered and activated. The numbers are not stored persistently anywhere. The device-mapper tables that back logical volumes exist only in kernel memory and are torn down on every shutdown, then recreated on every boot.

The fix: never reference /dev/dm-N in any configuration file, script, or monitoring rule. Use /dev/mapper/VG-LV or /dev/VG/LV instead. These symlinks are recreated from persistent LVM metadata on every activation and remain stable across reboots regardless of which dm-N minor number the kernel assigned.

What it is and why it matters

Every active LVM logical volume has a kernel device-mapper device at /dev/dm-N, where N is a minor number allocated by the kernel when the device-mapper table for that LV is loaded. This happens during activation: at boot via vgchange -ay triggered by systemd units and udev rules, or manually via lvchange -ay.

The allocation order depends on factors that can vary between boots:

• PV scan order: LVM scans block devices for PV metadata. Adding or removing a disk changes which PVs are discovered first.
• VG processing order: Multiple VGs are activated sequentially, and the order can shift.
• LV activation order within a VG: LVs activate in dependency order, which can vary.
• Parallel systemd units: systemd may process crypttab entries or other dm-dependent units in parallel, causing dm-N assignments to land in nondeterministic order.
• Slow storage links: iSCSI sessions reconnecting slowly or multipath paths failing over delays PV discovery, shifting the sequence.

The /dev/mapper/VG-LV and /dev/VG/LV symlinks are created by LVM’s activation code from the VG and LV names stored in metadata on the PVs. That metadata is persistent. The symlink names do not depend on the kernel’s minor number assignment, so they are identical on every boot.

flowchart TD
    A["Boot: kernel starts"] --> B["LVM scans block devices\nreads PV metadata"]
    B --> C["Activation: vgchange -ay\nloads dm tables into kernel"]
    C --> D["Kernel assigns dm-N\nminor numbers by activation order"]
    C --> E["LVM creates symlinks\nfrom VG/LV names in metadata"]
    D --> F["dm-0 = first activated\ndm-1 = second activated\norder varies per boot"]
    E --> G["/dev/mapper/VG-LV\n/dev/VG/LV\nstable every boot"]
    F --> H["Symlink target updates\nto current dm-N"]
    G --> H

How device-mapper numbering works

Device-mapper is a kernel subsystem that implements virtual block devices by mapping I/O requests to underlying physical extents. LVM uses device-mapper to implement logical volumes. Each active LV gets:

• A mapping table loaded into kernel memory describing which physical extents on which PVs back each LV extent
• A /dev/dm-N device node created by udev when the table is registered with the kernel

The N in dm-N is a minor number, allocated sequentially as dm devices are registered. The first dm device registered during boot gets dm-0, the second gets dm-1, and so on. The allocation is first-come, first-served based on registration order.

Device-mapper tables exist only in kernel memory. They are not written to disk. On shutdown, all tables are removed. On the next boot, LVM reads metadata from PVs, reconstructs the tables, and registers them with device-mapper. The registration order determines the minor number, and because that order depends on device discovery timing, the dm-N assignment is not deterministic.

The /dev/mapper/VG-LV symlink points to the current /dev/dm-N node. When the LV activates and gets assigned, say, dm-3 on this boot, /dev/mapper/vg0-data points to /dev/dm-3. On the next boot, if the same LV gets dm-1, the symlink is recreated to point to /dev/dm-1. The symlink name is stable. The underlying target changes transparently.

The lvm(8) man page recommends /dev/VG/LV for scripts and configuration files over /dev/mapper/VG-LV, noting that /dev/mapper node names are intended for internal use and their precise format may change between releases and distributions. In practice, both formats are stable across reboots on all major distributions. The Red Hat persistent naming documentation for RHEL 8 recommends either filesystem UUIDs or /dev/mapper/ paths and explicitly warns against using /dev/dm-* or /dev/sd* names.

Where this causes problems in production

Most operators never think about dm-N numbers until something breaks after a routine reboot. The common failure points:

fstab entries using /dev/dm-N. If /etc/fstab references /dev/dm-0 instead of /dev/mapper/VG-LV or a filesystem UUID, the mount may succeed by accident if the same LV happens to get dm-0 again, or fail if it does not. Boot may hang waiting for a device that no longer exists under that name, or drop to emergency mode.

Backup tools referencing dm-N paths. Backup agents that identify volumes by block device path can misidentify volumes after renumbering. Some agents interpret the changed device path as a new volume and trigger a full backup instead of an incremental one. Configure agents to use mount points or /dev/mapper/ names instead of raw device paths.

Monitoring and alerting configurations. Disk I/O monitoring rules, log collectors, and alerting thresholds that reference /dev/dm-0 break or silently report on the wrong device after a reboot. This is the insidious case: monitoring appears to function because data is still collected, but it is associated with the wrong volume. Capacity trends reset, latency baselines shift, and anomaly detection loses its reference frame.

Operator confusion from df output. df -h may show /dev/dm-N rather than the /dev/mapper/ name on some systems, because the output reflects the device node the kernel reports rather than the symlink used at mount time. Behavior varies by util-linux version and how the filesystem was mounted. This is harmless in df output, but operators often copy the dm-N path from df into fstab or monitoring configs where it will break.

How to audit and fix dm-N references

Check every configuration location that might reference dm-N paths directly.

# Check fstab for dm-N references
grep '/dev/dm-' /etc/fstab

# Search common config directories for dm-N references
grep -rn '/dev/dm-' /etc/ 2>/dev/null

# List current dm devices with their stable names
dmsetup ls

# Show symlinks and their current dm-N targets
ls -l /dev/mapper/ | grep -v control

For each dm-N reference found, replace it with one of these stable alternatives, in order of preference:

1. Filesystem UUID (from blkid): completely independent of device naming. Works for filesystem mounts and swap, but not where a raw block device with no filesystem signature is required.
2. /dev/VG/LV: recommended by lvm(8) for scripts and configuration. Stable across reboots and LVM version upgrades.
3. /dev/mapper/VG-LV: also stable across reboots. Widely used and supported across distributions.

Example fstab correction:

# Fragile: breaks when dm-N number changes after reboot
/dev/dm-0  /var/lib/postgresql  ext4  defaults  0 2

# Stable: symlink recreated from LVM metadata on every boot
/dev/mapper/vg0-pgdata  /var/lib/postgresql  ext4  defaults  0 2

After editing fstab, tell systemd to regenerate its mount units, then validate before rebooting:

# systemd generates mount units from fstab; reload after edits
systemctl daemon-reload

# Validate fstab mount definitions
findmnt --verify --verbose

Do not reboot to “test” an fstab change without running findmnt --verify first. A bad entry can drop the box to emergency mode.

Why not pin dm-N numbers

Older LVM documentation from the RHEL 6 era described a --persistent y --minor <minor> option on lvcreate and lvchange that could request a specific minor number for an LV. The options still exist (lvchange --persistent y --minor N is documented in current lvchange(8)); lvcreate requires -My to accept --minor. This approach is effectively obsolete on modern systems. The kernel allocates device-mapper minor numbers dynamically at activation time, and current LVM documentation does not recommend pinning minor numbers. Do not rely on this mechanism. The correct approach is to use naming that does not depend on minor numbers at all.

Normal vs. abnormal: what to actually worry about

The critical distinction: minor number changes between boots are expected and harmless as long as nothing references the numbers directly. Minor number changes or device disappearance during runtime indicate an actual problem such as LV deactivation, PV loss, or unexpected manual intervention.

How Netdata helps

• Disk I/O metrics are collected per-second from /proc/diskstats. After a dm-N renumbering event, these metrics confirm the LV is receiving I/O under its new minor number, distinguishing benign renumbering from a failed activation.
• LVM health signals such as VG free space, thin pool data and metadata usage, PV accessibility, and LV health flags are collected using stable VG and LV name references. A dm-N shift does not trigger false capacity or health alerts.
• Per-second collection minimizes the blind window after reboot. If an LV fails to activate, which is the real risk that renumbering can mask, the gap in metrics is short enough to detect quickly.

Related guides

• LVM cannot extend a logical volume: adding a PV when the VG is full
• LVM Couldn’t find device with uuid: a physical volume has gone missing
• LVM filesystem full while the volume group has space: the resize step everyone forgets
• LVM Found duplicate PV: multipath devices and the lvm.conf filter
• How LVM actually works in production: a mental model for operators
• LVM Insufficient free extents: the volume group is out of space
• LVM logical volume not active: lvchange -ay and why activation failed
• LVM logical volume partial (p) flag: which LVs the missing disk took down
• LVM mirror resync storm: multiple rebuilds saturating disk I/O
• LVM monitoring checklist: the signals every production volume manager needs
• LVM monitoring maturity model: from survival to expert
• LVM RAID or mirror degraded: a leg is dead and you are one failure from data loss