vSphere CPU ready time high (%RDY): VMs starved while the guest looks idle

A database VM takes twice as long to run its nightly batch. Application latency pings fire. You SSH into the guest, run top, and CPU utilization sits at 25%. Memory is fine. Disk I/O looks normal. Nothing inside the VM explains the slowdown.

This is the classic signature of CPU ready time in vSphere. The guest OS has no visibility into hypervisor scheduling decisions. When the ESXi CPU scheduler cannot find a free physical CPU for a runnable vCPU, the vCPU waits in the READY state. The guest never learns it was descheduled, so from inside the VM everything looks idle while the hypervisor sees a starved VM.

What this means

CPU ready time (%RDY in esxtop, cpu.ready.summation in vCenter) measures the time a vCPU was runnable (it had work to do) but waited for a physical CPU. It is the direct tax of CPU overcommitment.

The guest OS cannot see this. The guest CPU utilization counter measures work completed, not work demanded. A VM reporting 25% guest CPU with 15% ready time is running at roughly 85% of its requested speed, yet the guest reports a comfortable 25%.

Per-vCPU conversion (vCenter and PowerCLI)

The cpu.ready.summation counter is in milliseconds, summed across all vCPUs of the VM. Convert to a per-vCPU percentage:

per_vcpu_rdy_pct = (ready_ms / (interval_ms * vCPUs)) * 100

For the realtime interval (20 seconds = 20000 ms):

per_vcpu_rdy_pct = ready_ms / (20000 * vCPUs) * 100
             = ready_ms / (200 * vCPUs)

The interval changes with the statistics rollup level. 500 ms ready in a realtime (20s) interval is a different percentage than 500 ms in a 5-minute or daily rollup. Confirm which interval you are reading before applying the formula.

The aggregate trap

vCenter’s cpu.ready.summation (and the value returned by Get-Stat -Stat cpu.ready.summation) sums ready time across all vCPUs. A 4-vCPU VM reporting 8000 ms ready in a 20s interval is not 8000 / 20000 = 40% per vCPU; it is 8000 / (20000 * 4) = 10% per vCPU. Skipping this division is the most common operator mistake.

esxtop’s default per-world %RDY is already a per-vCPU percentage (one world per vCPU, vmmX:vmname). The trap applies to vCenter charts, PowerCLI Get-Stat, and any VM-grouped esxtop view that rolls worlds up into a single line.

Per-vCPU thresholds

Per-vCPU %RDYSeverityAction
< 2%NormalRoutine operation. Target for latency-sensitive workloads (databases, VDI, real-time).
2 to 5%Early contentionInvestigate VM placement and sizing.
5 to 10%Performance impactLatency-sensitive workloads degraded. Right-size or rebalance.
> 10%Production incidentApplication-visible latency and timeouts likely. Combine with host CPU > 85% to confirm overcommitment.

False positives

  • Brief spikes during vMotion (1-2 minutes) are expected.
  • Mass VM boot events cause transient spikes that resolve in minutes.
  • Idle servers do not produce false positives because ready time is near-zero when nothing is queued.

Rolled-up data hides spikes

At 5-minute and 30-minute rollups, averaging masks short spikes. A VM at 0% ready for 4.5 minutes and 50% for 30 seconds reports roughly 5% at the 5-minute rollup. Realtime data (20s interval, retained for 1 hour in vCenter) is the only reliable source for spike analysis. After that window, the detail is gone.

flowchart TD
    A["High %RDY on VM"] --> B{"Host CPU utilization?"}
    B -->|"Above 85%"| C["Genuine overcommitment"]
    B -->|"60 to 80%"| D{"%CSTP above 3%?"}
    B -->|"Below 60%"| E{"%MLMTD non-zero?"}
    D -->|"Yes"| F["vCPU oversizing"]
    D -->|"No"| G["NUMA imbalance"]
    E -->|"Yes"| H["CPU limit throttling"]
    E -->|"No"| I["Scheduler anomaly or affinity"]
    C --> J["Add capacity or migrate VMs"]
    F --> K["Reduce vCPU count"]
    G --> L["Check NUMA locality"]
    H --> M["Remove CPU limit"]

Common causes

CauseWhat it looks likeFirst thing to check
Host CPU overcommitmentHigh %RDY on multiple VMs. Host CPU above 85%. All VMs on the host affected.esxtop PCPU %USED section
vCPU oversizingHigh %RDY on specific large VMs. Host CPU moderate (60-80%). %CSTP elevated on the same VMs.%CSTP column in esxtop
NUMA imbalanceHigh %RDY on VMs spanning NUMA nodes. One NUMA node saturated, the other idle.esxtop, press ’m’, N%L column
CPU limit misconfigurationVM is slow but host CPU is low and %RDY looks fine. %MLMTD is non-zero.%MLMTD column in esxtop
Co-scheduling overhead%RDY and %CSTP both elevated on multi-vCPU VMs. Worse on VMs with more vCPUs.%CSTP vs vCPU count

Quick checks

Run esxtop on the ESXi host (SSH or DCUI) or use PowerCLI against vCenter.

# CPU scheduler view. %RDY per world is already per-vCPU.
# Press 'c' for CPU view, then look at %RDY, %CSTP, %MLMTD per world.
esxtop

# Collect cpu.ready.summation via PowerCLI (realtime, 20s interval).
# This value is an aggregate in ms across all vCPUs. Divide by vCPU count.
Get-Stat -Entity (Get-VM "myvm") -Stat cpu.ready.summation -Realtime
# per-vCPU %RDY = value_ms / (20000 * vCPU_count) * 100

# Host CPU utilization. OverallCpuUsage is in MHz.
# Note: NumCpuCores counts physical cores; on HT-enabled hosts this
# understates logical capacity. Substitute NumCpuThreads where available.
Get-VMHost |
  Select Name, @{N='CpuPct';E={[math]::Round(
    $_.Summary.QuickStats.OverallCpuUsage /
    ($_.Summary.Hardware.CpuMhz * $_.Summary.Hardware.NumCpuCores) * 100, 2)}}

# NUMA locality. Press 'm' for memory, 'f' to toggle NUMA fields.
# N%L is NUMA locality. Below 80% means significant cross-node access.
esxtop

How to diagnose it

  1. Confirm the symptom is ready time, not in-guest CPU saturation. Check guest-internal CPU first. If the guest shows near 100% CPU, the problem is inside the guest. Ready time only applies when the guest looks idle or under-utilized while the application is slow.

  2. Convert the counter correctly. Take cpu.ready.summation in milliseconds and apply (ready_ms / (interval_ms * vCPUs)) * 100. If reading esxtop per-world %RDY, the value is already per-vCPU.

  3. Correlate with host CPU utilization. This correlation is the key diagnostic step.

    • High ready (> 5% per vCPU) plus high host CPU (> 85%): genuine overcommitment. The host is out of physical cycles.
    • High ready plus moderate host CPU (60-80%): vCPU oversizing or NUMA imbalance. The host has capacity but the scheduler cannot find enough simultaneously-free pCPUs for the specific VM.
    • High ready plus low host CPU (< 60%): scheduler anomaly, CPU affinity misconfiguration, or a hidden CPU limit.

  4. Check co-stop (%CSTP) for multi-vCPU VMs. Co-stop measures the time vCPUs wait for sibling vCPUs to be co-scheduled. %CSTP above 3% means the VM likely has more vCPUs than it needs. This is the definitive oversizing spiral signal: an admin sees a slow VM, adds vCPUs, and the problem worsens because the scheduler must find even more simultaneously-free pCPUs.

  5. Check for CPU limits (%MLMTD). A VM with a configured CPU limit shows low ready time but high max-limited time. The limit is an artificial MHz ceiling. Any non-zero %MLMTD on a slow VM means a configured limit is the culprit. Limits are inherited from templates and resource pools; the “unlimited” default is -1 in the API, not 0.

  6. Check NUMA locality. A VM whose vCPUs and memory span NUMA nodes suffers cross-node memory access latency and harder scheduling. Check N%L in esxtop memory view. Locality below 80% for a latency-sensitive workload is a configuration problem. Hot-add of CPU or memory can break vNUMA alignment.

  7. Rule out transient causes. Check for recent vMotion migrations, mass boot events, or DRS storms after a host reconnect or vCenter restart. Sustained ready time for more than 5 minutes is not transient.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
CPU ready time (cpu.ready.summation)Primary contention signal. Invisible to the guest OS.Per-vCPU above 5% sustained, or above 2% for latency-sensitive workloads.
Co-stop (cpu.costop.summation)Co-scheduling overhead for multi-vCPU VMs. The oversizing indicator.Above 3% sustained. Should be near zero when sized correctly.
Max-limited (cpu.maxlimited.summation)Artificial CPU limit throttling. Invisible to ready time.Any non-zero value where the VM owner reports slowness.
Host CPU utilization (cpu.usage.average)Overall compute pressure. Distinguishes host-wide from VM-specific contention.Sustained above 85% combined with ready time above 5%.
NUMA locality (numa.local / numa.remote)Cross-node memory access penalty.Locality below 80% for latency-sensitive workloads.
Guest CPU utilizationIn-guest CPU usage. Low guest CPU plus high ready time is the invisible starvation signature.Guest below 30% while the application is slow.

Fixes

Host CPU overcommitment

When host CPU is above 85% and ready time is elevated across multiple VMs, the host is out of physical capacity.

  • Increase DRS aggressiveness or move it to fully automated to balance VMs across hosts with headroom.
  • vMotion the most impacted VMs to less-loaded hosts. Expect a brief stun (typically under 1 second, longer for memory-dirty workloads).
  • Add host capacity to the cluster or reduce VM density on the saturated host.
  • Find runaway VMs consuming disproportionate CPU.

vCPU oversizing

When host CPU is moderate (60-80%), co-stop is elevated, and large VMs show worse ready time than small VMs on the same host, the problem is VM size, not host capacity.

  • Check in-guest CPU utilization. A 16-vCPU VM at 20% guest CPU needs roughly 3-4 vCPUs.
  • Reduce vCPU count to match actual utilization. This requires a VM power cycle.
  • Fewer vCPUs means the scheduler needs fewer simultaneously-free pCPUs, directly reducing both ready time and co-stop.
  • Applications that spawn worker threads based on detected vCPU count may need reconfiguration after reduction.

NUMA imbalance

When a VM’s vCPU count exceeds the cores per NUMA node, it spans nodes by definition. This causes harder scheduling and remote memory access.

  • Size VMs to fit within a single NUMA node when possible.
  • Verify CPU Hot Add is not breaking vNUMA alignment.
  • Check Cores per Socket against physical topology.

Large database VMs that genuinely need more cores than one NUMA node provides will always span nodes. Accept the penalty or split the workload across smaller VMs.

CPU limit misconfiguration

When host CPU is low, ready time looks fine, but %MLMTD is non-zero.

  • Remove the CPU limit (set to Unlimited) in VM settings.
  • Check resource pool limits that cascade to child VMs.
  • Check VM templates for inherited limits that propagate at deployment time.

Removing limits on a shared host increases noisy-neighbor risk. Use shares instead of limits for proportional priority control.

Prevention

  • Monitor per-vCPU ready time, not the aggregate. Alert on the converted per-vCPU value, not the raw summation counter or a VM-grouped esxtop line.
  • Right-size at provisioning. Start with fewer vCPUs and add only when in-guest utilization justifies it. Reducing vCPUs requires a power cycle; adding does not.
  • Monitor co-stop alongside ready time. Co-stop above 3% is the definitive oversizing signal. Catch it before someone adds more vCPUs to a slow VM.
  • Scan for CPU limits during VM audits. Limits inherit from templates and resource pools and are invisible to the guest. Check for non-zero %MLMTD across the inventory.
  • Keep latency-sensitive VMs on hosts below 60-70% CPU. The ready time curve is non-linear. Between 80-90% utilization, ready time climbs sharply.
  • Run DRS fully automated for clusters with variable workloads. Manual DRS lets imbalance persist until an operator intervenes.

How Netdata helps

  • Netdata’s vSphere collector ingests per-VM CPU scheduler counters and presents ready time, co-stop, max-limited, and host CPU utilization together, shortening the path from “this VM is slow” to the specific cause: overcommitment, oversizing, NUMA misalignment, or a hidden CPU limit.
  • Per-vCPU conversion and default alarms on ready time and co-stop remove the manual formula step and the aggregate-misread path that triggers false escalations.
  • The max-limited counter sits alongside ready time, so CPU limit throttling does not hide behind a healthy ready signal.
  • Anomaly baselines on ready time surface slowly creeping contention before it trips a static threshold.
  • NUMA locality metrics are available alongside CPU signals for large VMs where cross-node memory access is the silent throughput penalty.