vSphere dropped packets (%DRPRX/%DRPTX): ring buffers, CPU, and uplink backpressure

%DRPRX and %DRPTX in esxtop are usually the first sign a VM is losing packets inside the host. They should be zero at steady state. When they are not, the guest retransmits, latency climbs, and for latency-sensitive workloads (IP-based storage, databases, replicated queues) the impact can be severe well before the drop rate looks alarming.

The distinction that matters: %DRPRX and %DRPTX count drops at the virtual switch port, between the vSwitch and the guest OS driver. They are not physical NIC drops. The uplink vmnic can report zero drops via esxcli network nic stats get while %DRPRX is non-zero on the VM attached to it. Treating them as the same counter is the most common diagnostic mistake.

The second model that matters: the vSwitch lives in the VMkernel, and every packet traversing it consumes pCPU cycles. Anything that raises per-packet CPU cost (NSX Distributed Firewall, high packet rates, fragmentation) or reduces the pCPU time available to the VM (CPU ready, co-stop, limits) can cause the guest driver to drain its receive ring too slowly. The ring fills, and packets are dropped at the vSwitch port. The fix is not always a larger ring buffer.

What this means

%DRPRX is the percentage of received packets dropped at the vSwitch port for that VM. %DRPTX is the same for transmitted packets. In esxtop, press n for the network view and read the %DRPRX and %DRPTX columns per port. The underlying counters are net.droppedRx.summation and net.droppedTx.summation.

Non-zero values mean one of the following is happening:

  • The vmxnet3 ring buffer in the guest cannot be drained fast enough (CPU starvation or an undersized ring).
  • A traffic shaping policy on the port group is intentionally dropping excess traffic.
  • Port security or MAC filtering rules are discarding packets.
  • NSX Distributed Firewall processing is consuming enough pCPU that the VM cannot keep up.

Physical uplink saturation produces a different counter. When the vmnic itself runs out of receive or transmit ring capacity, the drops appear in esxcli network nic stats get -n vmnicX as rx_dropped and tx_dropped, not in %DRPRX and %DRPTX. Both can happen at once, but they have different root causes and different fixes.

flowchart TD
    A[Guest OS app] -->|packets in/out| B[vmxnet3 ring buffer]
    B -->|overflow if CPU slow
or ring too small| C["%DRPRX rises"]
    B --> D[vSwitch port]
    D -->|traffic shaping
or port security| E["%DRPTX or %DRPRX"]
    D --> F[NSX DFW kernel path]
    F -->|per-packet CPU cost| G[raises VM CPU demand]
    D --> H[Physical uplink vmnic]
    H -->|pNIC ring overflow| I[rx_dropped and tx_dropped
separate counters]

A small number of drops during a burst, a vMotion, or a backup window may be normal. TCP retransmission absorbs minor loss. Sustained non-zero %DRPRX or %DRPTX outside configured traffic shaping is a real problem.

Common causes

CauseWhat it looks likeFirst thing to check
vmxnet3 ring buffer exhaustion%DRPRX rises under burst traffic, pNIC drops are zero, guest TCP retransmits climbvsish rxSummary on the port: “running out of buffers”
CPU starvation (ready, co-stop, or limit)%DRPRX tracks %RDY or %CSTP on the same VM, guest CPU looks idleesxtop CPU view, %RDY and %MLMTD columns
NSX Distributed Firewall overheadDrops appear only on hosts or VMs behind DFW, per-packet CPU cost visible as system timeCompare drop rate with DFW disabled on a test VM
Traffic shaping enforcement%DRPTX rises exactly when the shaping average, peak, or burst limit is exceededPort group traffic shaping policy
Physical uplink saturationpNIC rx_dropped or tx_dropped non-zero, %DRPRX may also rise if backpressure reaches the vSwitchesxcli network nic stats get, uplink MbRX/s and MbTX/s
SR-IOV passthrough%DRPRX and %DRPTX always zero despite packet loss reported by the applicationConfirm SR-IOV is in use, then diagnose from the guest

Quick checks

All read-only. Run from the ESXi shell unless noted.

# Network view: locate the VM with non-zero %DRPRX or %DRPTX
esxtop
# press 'n', then read the %DRPRX and %DRPTX columns per port

# Physical NIC drop counters (separate from vSwitch drops)
esxcli network nic stats get -n vmnic0

# Current ring sizes on the uplink
esxcli network nic ring current get -n vmnic0
<!-- TODO: verify whether 'preset get' exists across ESXi 6.7/7.0/8.0 -->
# Hardware-maximum ring sizes supported by the NIC
esxcli network nic ring preset get -n vmnic0

# VMXNET3 receive ring summary from the port's vmxnet3 layer
# Substitute the SwitchName and PortNumber shown in the esxtop 'n' view
vsish -e get /net/portsets/<SwitchName>/ports/<PortNumber>/vmxnet3/rxSummary

On the guest (Linux):

# Driver-level drop, miss, and discard counters
ethtool -S eth0 | grep -iE "drop|miss|discard"

# Current and maximum ring sizes
ethtool -g eth0

# TCP retransmissions: the application-visible consequence of drops
ss -ti
nstat -az TcpRetransSegs

How to diagnose it

  1. Confirm the drop location. Open esxtop, press n, and identify the port with non-zero %DRPRX or %DRPTX. Then run esxcli network nic stats get -n <vmnic> for the uplink that port uses. If the pNIC also shows rx_dropped or tx_dropped, the problem is at or below the physical layer. If the pNIC is clean, the drops are between the vSwitch and the guest.

  2. Check whether SR-IOV is bypassing the vSwitch. If the VM uses an SR-IOV virtual function, traffic never traverses the vSwitch. %DRPRX and %DRPTX will always read zero regardless of actual loss. Diagnose from the guest with ethtool -S or from the physical NIC hardware counters.

  3. Inspect the vmxnet3 ring buffer state. From the ESXi shell, run vsish on the port identified in step 1 and read the rxSummary. Non-zero values for “running out of buffers” or the count of times the first ring was full confirm the guest is not draining the ring fast enough. These counters reset on vMotion or power cycle, so check them immediately after observing %DRPRX.

  4. Correlate with CPU. Switch esxtop to the CPU view (press c) and check %RDY, %CSTP, and %MLMTD for the same VM. If any of these are elevated, the VM is not getting enough pCPU time to process packets, and the ring fills as a consequence. Enlarging the ring only delays the drops in this pattern.

  5. Check for NSX Distributed Firewall. If DFW is enabled on the host or the VM’s segment, every packet is processed in the VMkernel. Under high throughput this can consume a full pCPU core on firewall processing alone. If the environment permits, compare drop rates with DFW rules disabled on a test VM.

  6. Check traffic shaping. On the port group (vSS) or distributed port group (vDS), inspect the traffic shaping policy. If average bandwidth, peak bandwidth, or burst size is configured and the workload exceeds it, drops are expected behaviour, not a fault.

  7. Check uplink utilization. In the esxtop network view, look at MbRX/s and MbTX/s for the vmnic. Sustained utilization near link speed produces pNIC-level drops and can backpressure into vSwitch-level drops. NIC teaming does not aggregate bandwidth for a single flow, so one heavy stream can saturate one uplink while teammates sit idle.

Metrics and signals to monitor

SignalWhy it mattersWarning sign
%DRPRX and %DRPTX (esxtop)Direct vSwitch port drop rateAny sustained non-zero value outside traffic shaping
net.droppedRx and net.droppedTx per vNICSame data, collectible via API for alertingSustained non-zero
pNIC rx_dropped and tx_droppedPhysical layer drops, separate root causeNon-zero while vSwitch drops are zero
CPU %RDY per VMA starved VM cannot drain its receive ringGreater than 5% sustained alongside %DRPRX
CPU %CSTP per VMMulti-vCPU co-scheduling penalty delays packet processingGreater than 3% sustained on multi-vCPU VMs
CPU %MLMTD per VMAn artificial MHz limit throttling the VMAny non-zero value
Uplink MbRX/s and MbTX/sSaturation backpressure into the vSwitchGreater than 80% of link speed sustained
Guest TCP retransmitsApplication-visible consequence of dropsRising rate correlated with %DRPRX
vmxnet3 rxSummary buffer exhaustion countersDirect evidence of ring overflowNon-zero

Fixes

Ring buffer exhaustion

Enlarge the vmxnet3 ring in the guest. On Linux:

# Check current and maximum ring sizes
ethtool -g eth0

# Set the RX ring toward its maximum (commonly 4096 for vmxnet3)
ethtool -G eth0 rx 4096

On Windows VMs with VMXNET3, ring sizes are controlled by advanced adapter properties: Small Rx Buffers and Rx Ring #1 Size. Defaults are Small Rx Buffers 512 (maximum 8192) and Rx Ring #1 Size 1024 (maximum 4096). Raise both toward their maximums for high-throughput VMs.

Guest-side ring changes take effect immediately but do not persist across reboots without a udev rule, a NetworkManager dispatcher script, or an equivalent persistent configuration.

For the physical uplink, when pNIC drops are present:

# Check current ring and the hardware-enforced preset maximum
esxcli network nic ring current get -n vmnic0
<!-- TODO: verify whether 'preset get' exists across ESXi 6.7/7.0/8.0 -->
esxcli network nic ring preset get -n vmnic0

# WARNING: increasing RX and TX ring on a live uplink can cause a brief
# traffic disruption on that link. Schedule during a maintenance window
# or move VMs off the uplink first.
esxcli network nic ring current set -n vmnic0 -r <value> -t <value>

Host-side ring settings may not persist across host reboots or driver upgrades. Verify persistence for your ESXi version and driver, and re-check after any nmlx5 or similar driver update.

CPU starvation

If %RDY or %CSTP is elevated, the root cause is scheduling, not the ring. Enlarging the buffer only delays the drops. Options:

  • Reduce vCPU count if the VM is oversized. Co-stop falls, and the scheduler finds placement more easily.
  • vMotion the VM to a less contended host.
  • Remove forgotten CPU limits, which are visible as non-zero %MLMTD.
  • Reserve CPU for latency-sensitive VMs when the cluster has the capacity.

See the CPU ready time and co-stop guides for the full treatment of those signals.

NSX Distributed Firewall overhead

DFW processes every packet in kernel. For high-throughput VMs, the per-packet CPU cost can dominate. Mitigations:

  • Review the DFW rule set for rules that can be consolidated or pushed to the edge firewall.
  • Use exclusion lists for traffic that does not require DFW inspection, where policy allows.
  • Ensure the VM has enough pCPU headroom to absorb the firewall processing cost.
  • Confirm the relationship by correlating DFW-enabled hosts with %DRPRX before changing policy.

Traffic shaping

If shaping is dropping traffic, the drops are policy, not fault. Decide whether the policy is correct:

  • If the average, peak, or burst values are too low for the workload, raise them.
  • If the policy is intentional, the drops are expected and should alert only if they indicate the workload has outgrown its allocation.
  • Add uplinks to the team where the switch and configuration allow.
  • Separate traffic classes onto dedicated uplinks, or use Network I/O Control to guarantee bandwidth for latency-sensitive traffic.
  • Load Based Teaming and source or ID hashing do not split a single flow across uplinks. A single heavy flow can saturate one uplink while others remain idle.

Prevention

  • Baseline drop rates per VM. A VM that shows zero drops at normal load and then shows sustained drops is the earliest signal of a regression. Trend %DRPRX and %DRPTX, not just absolute counters.
  • Size rings for burst, not average. Ring buffers absorb microbursts that are invisible in five-minute averaged metrics. Default rings are often too small for high-throughput VMs.
  • Track CPU contention alongside network drops. %DRPRX without %RDY context leads to wrong fixes: bigger rings when the VM is actually starved.
  • Flag SR-IOV VMs explicitly. Their drops are invisible to esxtop. Monitor them from the guest with ethtool, not from the hypervisor.
  • Verify ring settings after ESXi and driver upgrades. Driver updates can reset custom ring buffer parameters to defaults.

How Netdata helps

  • Netdata collects net.droppedRx and net.droppedTx per second per interface, so vSwitch-port-level drop rates are visible without an interactive esxtop session and without waiting for vCenter’s rolled-up statistics.
  • Per-second resolution catches microbursts that five-minute averages hide. A 30-second drop spike during a backup window is invisible in vCenter charts but obvious at one-second granularity.
  • ML anomaly detection flags a sustained non-zero drop rate on an interface that is normally clean, even when the absolute rate sits below a static threshold.
  • Correlating dropped packets with CPU ready time, co-stop, and per-VM CPU utilization on one dashboard separates ring exhaustion from CPU starvation, which is the central diagnostic question for this symptom.
  • Guest-side TCP retransmission counters, collected by the Netdata guest agent, confirm that drops are causing real impact rather than counter noise.
  • For SR-IOV VMs where hypervisor-side drops read zero, the guest agent’s per-interface drop and retransmit metrics are often the only signal that something is wrong.