vSphere DRS not balancing: affinity rules and reservations blocking placement
DRS is enabled and Fully Automated, yet one host sits at 90% CPU while another idles at 30%. Vmotions are not happening, the recommendations queue is empty or full of unapplied entries, and the DRS score is poor. In most cases DRS is doing exactly what it was told: it cannot find a migration that satisfies every constraint.
DRS gates every placement decision behind a chain of compatibility and capacity checks before it compares hosts by load. A single VM-Host anti-affinity rule, a reservation that fully commits a host’s CPU or memory, or a missing vMotion network can each silently zero out the set of legal destinations. When every candidate fails one check, DRS emits no recommendation and the cluster drifts. Cluster averages hide this: a cluster “averaging 60% CPU” can mask one host at 90% and another at 30%. The per-host skew that DRS is supposed to fix is invisible in aggregate, and so is the constraint that prevents the fix.
What this means
DRS evaluates VM placement every 5 minutes by default and emits migration recommendations based on a cost-benefit tradeoff between imbalance improvement and vMotion cost. The migration threshold slider (levels 1 to 5, default 3) filters which priority recommendations are generated or applied. Before DRS compares hosts by load, it filters the destination set by compatibility. A host only becomes a candidate if:
- It is connected and not in maintenance mode.
- The VM’s storage is accessible from that host (shared datastore, accessible VMX and VMDK).
- CPU compatibility (EVC baseline, vendor, feature set) permits the migration.
- A vMotion network exists with adequate bandwidth.
- All VM-VM and VM-Host affinity and anti-affinity rules are satisfied by the placement.
- The host has enough unreserved CPU and memory to satisfy the VM’s reservation.
- The host has enough free physical memory to receive the VM without breaching memory pressure targets.
If no host outside the current one passes all of those checks, DRS emits nothing. The VM stays put, the imbalance persists, and the cluster’s DRS score stays poor. The symptom looks like “DRS is not running.” The reality is “DRS ran and found zero legal moves.”
flowchart TD
A[DRS scan every 5 min] --> B{Candidate hosts pass compat checks?}
B -- No --> C[No recommendation emitted]
B -- Yes --> D{Migration priority >= threshold?}
D -- No --> E[Recommendation suppressed]
D -- Yes --> F{Automation level?}
F -- Manual --> G[Recommendation queued, not applied]
F -- Fully Automated --> H[vMotion initiated]
C --> I[Imbalance persists, DRS score stays poor]
E --> I
G --> ICommon causes
| Cause | What it looks like | First thing to check |
|---|---|---|
| VM-Host affinity or anti-affinity rule | DRS never moves a specific VM, or a group of VMs is pinned to one host | Cluster > Configure > VM/Host Rules and VM/Host Groups |
| VM-VM affinity or anti-affinity rule | VMs that should separate are stuck together, or VMs that should co-locate cannot migrate independently | Same rules page; check “Must” vs “Should” (mandatory vs preferential) |
| Reservation fully commits host capacity | “No other hosts have sufficient CPU or memory resources” fault in DRS recommendation; one host is the only legal landing zone | Cluster > Configure > Resource Allocation |
| Memory overcommit on target hosts | Hosts look balanced by CPU but DRS will not move a memory-heavy VM because no target has free physical RAM | esxtop memory view (MCTLSZ, SWCUR) on candidate hosts |
| DRS in manual mode | Recommendations accumulate in the cluster Monitor tab but no vMotion runs | Cluster > Configure > vSphere DRS > Automation Level |
| Per-VM DRS override | Cluster is Fully Automated but specific VMs never move | VM > Configure > vSphere DRS > Automation Level |
| Migration threshold too conservative | Cluster is roughly balanced but small improvements never happen; slider at 1 or 2 | Cluster > Configure > vSphere DRS > Migration Threshold |
| vMotion prerequisites unmet | No DRS-triggered vMotions happen at all; manual vMotion fails | vMotion VMkernel adapter enabled, shared storage accessible, EVC baseline satisfied |
| Compatibility check failure | “Current host is incompatible” or transient incompatibility blocks migration | CompatCheckTransientFailureTimeSeconds advanced option (see below) |
| Host HA agent errors | Affected host shows agent unreachable, isolated, or partitioned; DRS will not place VMs on it | Cluster > Monitor > vSphere HA; host connection state |
Quick checks
# Cluster DRS configuration and current recommendations
Get-Cluster | Select Name, DrsEnabled, DrsAutomationLevel
Get-DrsRecommendation -Cluster <cluster>
# Recent DRS migrations in the last hour
Get-VIEvent -Entity (Get-Cluster <cluster>) -Start (Get-Date).AddHours(-1) |
Where-Object { $_ -is [VMware.Vim.DrsVmMigratedEvent] }
# Host connection and maintenance state
Get-VMHost | Select Name, ConnectionState,
@{N='InMaintenance';E={$_.ExtensionData.Runtime.InMaintenanceMode}}
# Failed vMotion events in the last 24 hours
Get-VIEvent -Entity (Get-Cluster <cluster>) -Start (Get-Date).AddHours(-24) |
Where-Object { $_ -is [VMware.Vim.VmFailedMigrateEvent] }
Note: Get-VIEvent retrieves all matching events client-side and can be slow on large vCenter inventories. Scope it to a cluster with -Entity and a time window.
# On the busy host: confirm CPU contention (source of the problem)
# On candidate destination hosts: check for memory reclamation (why DRS rejects them)
esxtop
# Press 'c' for CPU view: %RDY, %CSTP, %MLMTD per VM world
# Press 'm' for memory view: MCTLSZ (balloon target), SWCUR (swap current)
For per-VM DRS overrides and reservation headroom, use the UI paths in the table above. Per-VM overrides are easy to miss because the cluster-level setting looks correct.
How to diagnose it
- Confirm DRS is actually enabled and automated. Open Cluster > Configure > vSphere DRS. Confirm DRS is enabled, the Automation Level is Fully Automated (or Partially Automated), and the Migration Threshold is at 3 or higher. A cluster in Manual mode still computes recommendations but never applies them. This is the single most common cause.
- Check the recommendations queue. Cluster > Monitor > vSphere DRS > Recommendations. If the queue is empty, DRS found no legal moves. If it is full and nothing is running, the cluster is in Manual mode. Each recommendation carries a fault reason when inspected through
Get-DrsRecommendation. - Inventory affinity and anti-affinity rules. Cluster > Configure > VM/Host Rules. For each rule, note whether it is mandatory (“Must”) or preferential (“Should”). Mandatory rules are hard gates. A “Must run on hosts in group X” rule with only one host in group X pins those VMs regardless of load. A VM-VM anti-affinity rule with more VMs than hosts is mathematically unsatisfiable.
- Check per-VM DRS overrides. A VM set to Manual or Disabled at the VM level overrides the cluster Fully Automated setting. Open VM > Configure > vSphere DRS for suspect VMs. Common sources: a VM pinned during a previous incident, or a template-derived override that was never cleared.
- Compare reservation capacity to host capacity. Cluster > Configure > Resource Allocation shows total CPU (MHz) and memory reserved capacity for the cluster. If total reserved CPU or memory approaches total cluster capacity, every host is effectively fully committed and no migration will find unreserved headroom. Reservations that fully commit the capacity of the cluster or of individual hosts can prevent DRS from migrating VMs between hosts.
- Check memory headroom on candidate hosts. DRS will not move a VM to a host that lacks the free physical memory to receive it, regardless of CPU balance. Run esxtop on candidate destination hosts and check MCTLSZ and SWCUR at the host level. If candidates are ballooning or swapping, they are not legal destinations for a memory-heavy VM.
- Verify vMotion prerequisites. A manual vMotion of any VM in the cluster should succeed. If it fails, the error identifies the blocker: no vMotion VMkernel adapter, no shared storage accessibility, EVC mismatch, or CPU vendor mismatch. DRS uses the same vMotion path, so a manual failure explains DRS silence.
- Check host connection and HA agent state. DRS will not place VMs on disconnected, notResponding, or maintenance hosts, nor on hosts with HA agent errors (agent unreachable, isolated, partitioned). Cluster > Monitor > vSphere HA lists host health. HA does not consult DRS affinity rules during power-on, so VMs placed by HA may later violate rules and DRS will migrate them after the fact.
- Check CompatCheckTransientFailureTimeSeconds. This advanced option controls whether DRS will move a VM off a host it is transiently incompatible with. In vSphere 7.0 U1 the default was 600 seconds. In vSphere 8.0 U3 the default changed to -1, which disallows DRS from moving VMs out due to incompatibility with their current host. If you are on 8.0 U3 or later and see “current host incompatible” faults, set a positive value to restore the prior behavior.
Metrics and signals to monitor
| Signal | Why it matters | Warning sign |
|---|---|---|
| DRS balance score | Tracks cluster balance; sustained poor score means DRS cannot find legal moves | Score stays poor despite Fully Automated mode |
| Per-host CPU utilization | Cluster averages hide skew; one host at 90% while another idles at 30% is invisible in aggregate | Per-host spread greater than 20 percentage points sustained |
| Per-host memory consumed and active | Memory headroom on candidate destinations gates DRS placement of memory-heavy VMs | Candidate destinations near physical RAM |
| Memory balloon (mem.vmmemctl) and host swap | Ballooning or swap on a destination makes it a poor or illegal placement target | MCTLSZ greater than 0 or SWCUR greater than 0 on candidate hosts |
| CPU ready time (%RDY) | Reveals VMs starved on saturated hosts while the cluster average looks fine | Sustained above 5% on VMs on the busiest host |
| vMotion failure events | Failed migrations indicate infrastructure issues (CPU compatibility, vMotion network, storage accessibility) | VmFailedMigrateEvent count rising |
| ESXi host connection state | Disconnected, notResponding, or maintenance hosts are excluded from DRS placement | Any host not “connected” in a production cluster |
| HA cluster health and host agent state | HA agent errors generate compatibility failures that block DRS | Host marked isolated, partitioned, or agent unreachable |
| DRS migration rate | Sustained high rate indicates thrashing; near-zero with poor balance indicates constraints | Rate outside the expected band for the workload |
Fixes
Affinity and anti-affinity rules. Review every rule and convert mandatory (“Must”) rules to preferential (“Should”) where the workload tolerates it. Preferential rules generate fault reasons instead of blocking placement. Check VM-Host group membership: a rule that pins VMs to a one-host group is a hard pin. Check VM-VM anti-affinity rules against host count: anti-affinity across N VMs requires at least N hosts. Removing or relaxing the rule is usually preferable to fighting the scheduler.
Reservations. Compare reserved CPU (MHz) and memory across the cluster against physical capacity. Reduce reservations on VMs that do not need hard guarantees and use shares instead, which respond to actual demand. If reservations are required for licensing or SLA reasons, size the cluster so total reservations leave real headroom on every host, not just in aggregate.
Memory overcommit on candidate hosts. If candidate destinations are ballooning or swapping, DRS will not place memory-heavy VMs on them. Either reduce memory pressure on those hosts (right-size VMs, reduce overcommit) or add memory. The memory reclamation cascade is documented in the vSphere monitoring playbook: ballooning is the first tier, followed by compression, then host-level swap to .vswp files.
DRS automation level. If the cluster is in Manual mode because someone was cautious during a change, set it back to Fully Automated. If recommendations have been accumulating, apply or dismiss them so the queue is not masking the real state.
Per-VM DRS overrides. Find VMs with Manual or Disabled overrides and reset them to the cluster default unless there is a documented reason.
Migration threshold. If the slider is at 1 or 2, only the highest-priority recommendations are generated. Move to 3 (default) for general balancing. Going to 4 or 5 increases vMotion churn and is rarely justified.
vMotion prerequisites. Confirm each host has a VMkernel adapter tagged for vMotion, the vMotion network has adequate bandwidth (10GbE is the practical floor for production), all hosts see the VM’s datastores, and the EVC baseline matches across the cluster. A manual test vMotion should succeed before expecting DRS to use the path.
Compatibility transient option. On vSphere 8.0 U3 and later, if “current host incompatible” faults block migrations that previously worked, set CompatCheckTransientFailureTimeSeconds to a positive value (the prior default was 600) via the cluster’s advanced options. This restores the behavior where DRS tolerates transient incompatibility for a bounded window.
Prevention
- Audit rules and reservations quarterly. Affinity rules and reservations accumulate from incidents and one-off changes. Review them against current host count and cluster capacity.
- Monitor per-host, not cluster-average. Cluster CPU and memory averages hide the exact skew DRS is supposed to fix. Alert on per-host spread, not on the aggregate.
- Track DRS score over time. A steadily declining DRS score with no topology change usually means a new constraint was added: a rule, a reservation, or a per-VM override.
- Gate changes that add reservations or rules. Treat a new mandatory VM-Host rule or a large reservation like a capacity change. Verify DRS still has legal moves after it is applied.
- Verify vMotion health continuously. A periodic test vMotion across host pairs catches EVC drift, network changes, and storage accessibility issues before DRS needs the path.
How Netdata helps
- Netdata collects per-host CPU utilization, memory consumed, and memory active metrics at high resolution, making per-host skew visible without waiting for vCenter’s 5-minute rollups. The cluster average can look fine while one host saturates.
- The vSphere integration surfaces CPU ready time, co-stop, and max-limited per VM, which reveal whether VMs on the busiest host are actually being starved even when cluster averages look acceptable.
- Memory balloon and host swap metrics identify candidate destination hosts that DRS will reject for memory placement reasons, before you spend time wondering why a memory-heavy VM will not migrate.
- DRS migration events and vMotion failure events correlate with host connection state and HA agent health, so a migration that stopped working can be traced to a host that became unreachable or partitioned.
- Anomaly detection on per-host utilization spread flags the moment a cluster starts to skew, which is often the first visible symptom of a constraint blocking DRS.
Related guides
- vSphere active vs consumed vs granted memory: why the percentage lies
- vSphere CPU co-stop high (%CSTP): the SMP vCPU co-scheduling penalty
- vSphere CPU limit hit (%MLMTD): the forgotten MHz cap that silently throttles a VM
- vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle
- vSphere datastore full: ‘No space left on device’, paused VMs, and power-on failures
- vSphere datastore IOPS and throughput: spotting storage saturation before latency bites
- vSphere datastore latency high: reading GAVG, DAVG, and KAVG
- vSphere dropped packets (%DRPRX/%DRPTX): ring buffers, CPU, and uplink backpressure
- vSphere storage latency cliff: the ’everything is slow’ incident that hits every VM at once
- vSphere HA host isolation and split-brain: when isolation response goes wrong
- vSphere HA ‘Insufficient resources to satisfy configured failover level’: admission control
- vSphere ESXi hardware health: ECC errors, fan failure, and thermal throttling






