You run lvs and see a lowercase p in the ninth character of lv_attr. That means a physical volume (PV) backing extents in this logical volume has disappeared from the system. The VG is now in partial mode, and some LVs may be silently degraded or completely inaccessible.
The p flag is direct evidence of device loss: a disk failed, a SAN LUN was unpresented, a multipath device lost all paths, or a cloud volume was detached. The question is how bad the damage is and which volumes took the hit.
What this means
Position 9 of lv_attr is the health field. A p there means one or more PVs backing this LV’s extents are missing. The kernel device-mapper layer still has the LV’s mapping table loaded, but some mappings now point to a device that no longer exists. What happens next depends on the LV type:
- Linear and striped LVs: Any I/O to extents on the missing PV fails. If filesystem metadata (superblock, journal, inode tables) lived on those extents, the filesystem may be unrecoverable. The data on the missing extents is gone unless a backup exists.
- RAID LVs (RAID1/4/5/6/10, mirror): The array degrades but continues operating. One more failure on a single-fault-tolerant array (RAID1, RAID5) means total data loss.
- Thin pools: If the missing PV hosted the thin pool’s data LV or metadata LV, all thin volumes in that pool are affected.
At the VG level, vgs shows p in position 4 of vg_attr. At the PV level, the missing device shows as [unknown] in pvs output with m in its attributes.
flowchart TD
A["PV disappears from system"] --> B["VG goes partial
(p in vg_attr pos 4)"]
B --> C["Affected LVs show p
in lv_attr pos 9"]
C --> D{"LV type?"}
D -->|"Linear / Striped"| E["I/O errors on
missing extents"]
D -->|"RAID / Mirror"| F["Array degraded,
still operating"]
D -->|"Thin pool data/meta
on missing PV"| G["All thin LVs
in pool affected"]
E --> H["Assess data loss,
restore from backup"]
F --> I["Replace PV,
rebuild with lvconvert"]
G --> J["Pool may be
unrecoverable"]Common causes
| Cause | What it looks like | First thing to check |
|---|---|---|
| Disk hardware failure | dmesg shows I/O errors for a specific sd/nvme device; PV shows as [unknown] | dmesg | grep -i error | tail -50 |
| SAN LUN unpresented or zoned away | PV disappears without kernel errors; multipath device gone | multipath -ll and SAN management console |
| Multipath all paths down | PV missing but underlying paths may still exist in /sys/class/block/ | multipath -ll, check multipathd |
| Cable or HBA failure | Link errors in dmesg; device drops from /dev/ | dmesg, HBA controller status |
| Accidental hot-unplug or cloud volume detach | PV missing after maintenance window or cloud API action | Cloud console, change management records |
| Multipath metadata inconsistency (false positive) | PV shows missing but device is healthy and accessible | vgchange --refresh to clear stale flag |
Quick checks
These commands are read-only and do not modify LVM state.
# Check for missing PVs (look for [unknown] or 'm' in pv_attr)
pvs -o pv_name,vg_name,pv_attr,pv_size,pv_free
# Check VG partial status (look for 'p' in position 4 of vg_attr)
vgs -o vg_name,vg_attr,vg_missing_pv_count
# List all LVs with their health attribute (position 9 of lv_attr)
lvs -o lv_name,vg_name,lv_attr,lv_active
# Filter for any LV with a non-healthy flag in position 9 (p, r, D, F, m, X, s)
lvs -o lv_name,vg_name,lv_attr | awk 'NR>1 && substr($3,9,1) != "-"'
# Map PV segments to LVs - the key diagnostic command
pvs --segments -o pv_name,lv_name,seg_start_pe,seg_size_pe
# Check kernel logs for device errors
dmesg | grep -i 'I/O error\|offline\|not ready\|device not found' | tail -50
# Verify which PV devices actually exist as block devices
for pv in $(pvs --noheadings -o pv_name 2>/dev/null); do
[ -b "$pv" ] && echo "$pv: OK" || echo "$pv: MISSING"
done
# Check multipath status if applicable
multipath -ll 2>/dev/null
# Check dm device state (works when LVM tools hang on a missing device)
dmsetup info -c -o name,attr,suspended
Diagnostic walkthrough
Confirm the partial state. In the
vgsoutput above, look forpin position 4 ofvg_attr. Inpvsoutput, look for any PV showing as[unknown]or withmin its attributes. Thevg_missing_pv_countcolumn gives you the count directly.Identify the missing PV. The
[unknown]entry tells you which PV UUID has no backing device. Cross-reference withdmesgfor kernel-level errors. Checkmultipath -llin multipath environments.Map extents to LVs. This is the critical step. In the
pvs --segmentsoutput, find the[unknown]device. Every LV listed alongside it had extents on that missing PV. This is how you answer “which LVs did the missing disk take down.”Classify each affected LV. Check segment type:
lvs -o lv_name,seg_typeshowslinear,striped,raid1,raid5, and so on. For thin pool internal LVs (tdata, tmeta), uselvs -a -o lv_name,lv_attr,seg_typeto see all sub-LVs.Check application impact. Determine whether affected LVs are mounted or have open file descriptors. Look for D-state processes (
ps aux | awk '$8 ~ /D/'). Check application logs for I/O errors.Assess RAID health. For degraded RAID LVs, examine dm status:
# dm device names use doubled hyphens: vg "my-vg" lv "data" -> my--vg-data dmsetup ls | grep <vgname> dmsetup status <dm-device-name>In the status output, health characters after the device count:
A= alive and in-sync,a= alive but not in-sync (resyncing),D= dead. Count surviving healthy legs to assess remaining redundancy.
Metrics and signals to monitor
| Signal | Why it matters | Warning sign |
|---|---|---|
| PV accessibility | Missing PV is the root cause of the partial flag | Any PV showing [unknown] or m in pv_attr |
| LV health (position 9) | The p flag itself; also watch for r (refresh needed) on returning devices | Any non-- value in position 9 |
| VG partial flag (position 4) | VG-level confirmation of device loss | p in vg_attr position 4 |
| RAID sync status | Tells you if a degraded array is rebuilding or stalled | copy_percent stuck, any device showing D |
| D-state process count | Confirms application-level I/O impact | Growing count of processes in D state on dm devices |
| LVM command execution time | Slow commands indicate the missing device is blocking device scans | lvs taking more than 5 seconds |
Fixes
The PV is permanently gone (disk failure, LUN removed)
For linear or striped LVs that had extents on the missing PV, the data on those extents is lost. If filesystem metadata survived on the remaining PVs, you may be able to activate the partial LV to recover whatever data is accessible.
# Preview what vgreduce --removemissing will do (safe, no changes)
vgreduce --removemissing --test <vgname>
# Activate a partial LV for data recovery (read-only recommended)
lvchange -ay --partial -pr <vgname>/<lvname>
The --partial flag is required. Without it, LVM refuses to activate any LV with missing backing devices. Use -pr (read-only) to avoid further corruption during recovery.
Once you have assessed the damage and recovered what you can, remove the missing PV from VG metadata:
# WARNING: This permanently removes all linear/striped LVs that had extents
# on the missing PV. Run --test first and review the output carefully.
vgreduce --removemissing --force <vgname>
This is destructive for linear and striped LVs. Every such LV that had any extent on the missing PV will be removed from VG metadata. Do not run this until you have confirmed which LVs are affected and assessed recoverability.
The PV is a degraded RAID leg
For RAID LVs, the array continues operating on remaining legs. The recovery path is to replace the failed device and rebuild.
# Check current RAID health
lvs -o lv_name,lv_attr,copy_percent,raid_sync_action
# Replace the failed leg (requires free extents in the VG)
lvconvert --repair <vgname>/<lvname>
lvconvert --repair allocates a new leg from free extents in the VG and initiates a resync. If the VG has no free space, add a replacement PV first: pvcreate on the new device, then vgextend <vgname> <newdevice>.
The PV reappeared (transient failure)
If the missing PV comes back (SAN path restored, cable reseated, multipath recovered), LVM may still show the partial flag. Once the underlying device is confirmed healthy and accessible:
# Refresh the VG to re-evaluate PV presence
vgchange --refresh <vgname>
# Or refresh individual LVs
lvchange --refresh <vgname>/<lvname>
For the false-positive scenario where LVM reports PVs as missing even though the devices are healthy and available (known to happen after multipath path changes), vgchange --refresh clears the stale missing flag.
The missing PV hosted thin pool infrastructure
If the missing PV backed the thin pool’s data LV or metadata LV, all thin volumes in that pool depend on those internal LVs.
- Data LV on the missing PV: The pool’s physical storage has a hole. Thin volumes with extents mapped to the missing blocks will return I/O errors.
- Metadata LV on the missing PV: The block mapping table for all thin volumes is gone. Recovery is unlikely without a backup.
Assess with dmsetup status --target thin-pool to check the pool’s current state. Do not attempt to extend or repair pool metadata while it may be corrupted.
Prevention
- Map LVs to physical topology before an incident. Run
pvs --segmentsperiodically and store the output. During an incident, you need to know which services are affected immediately, not after running diagnostics. - Use RAID for important data. Linear and striped LVs provide no redundancy. A single PV failure can destroy a filesystem if its metadata was on the lost extents. RAID1 or RAID5 costs capacity but survives single-PV loss.
- Do not co-locate thin pool data and metadata on a single PV. If both are on the same device, one failure takes down every thin volume in the pool.
- Monitor PV accessibility continuously. A PV does not go missing for benign reasons after boot. Any missing PV after the system has been up for more than 10 minutes requires investigation.
- Run
vgckperiodically. Metadata consistency checks catch corruption before it compounds during an incident. - Keep
/etc/lvm/archive/backed up. These files are the recovery lifeline for metadata corruption. If/etcis on an LVM volume, ensure backups exist on separate storage.
How Netdata helps
- Netdata collects disk I/O metrics from
/proc/diskstatsand dm device counters per second, so device-level failures surface immediately alongside LVM state changes.
- Netdata’s LVM collector surfaces LV attributes, alerting on degraded or partial states without custom scripts.
- D-state process monitoring detects application I/O impact in real time, confirming whether a partial LV is causing a service outage or degrading silently.
- Correlating device-mapper status with VG free space and thin pool utilization gives a complete picture of recovery options: whether there is room for
lvconvert --repair, whether the pool can be extended, or whether data restoration is the only path.
Related guides
- LVM cannot extend a logical volume: adding a PV when the VG is full
- LVM filesystem full while the volume group has space: the resize step everyone forgets
- How LVM actually works in production: a mental model for operators
- LVM Insufficient free extents: the volume group is out of space
- LVM monitoring checklist: the signals every production volume manager needs
- LVM monitoring maturity model: from survival to expert
- LVM reached low water mark for data device: the thin pool warning before the freeze
- LVM thin pool space not reclaimed: discard, TRIM, and fstrim
- LVM snapshot COW usage climbing: extend or remove before it overflows
- LVM snapshot invalid: the COW exception store filled and the snapshot is gone
- LVM thin pool auto-extend not working: threshold 100 means disabled
- LVM thin pool out of data space: every thin volume freezes at once






