Fluentd input emit_records stuck at zero: enable_input_metrics on older versions

You open the monitor_agent API (or a dashboard built on top of it) and every input plugin reports emit_records: 0. First read: ingestion has stopped. Then you notice output plugins are delivering records, buffers are cycling, and downstream log storage is receiving fresh data. The pipeline is fine. The metric is blind.

On Fluentd versions before v1.19.0, input plugin metrics are disabled by default. Unless you set enable_input_metrics true in the <system> block, every input plugin’s emit_records counter stays at 0 forever, no matter how much data flows through it. This is a common false alarm, and it also cuts the other way: teams running real ingestion stalls on older versions have no input-side visibility at all because they never turned the metrics on.

The fix is a one-line config change or an upgrade. The harder part is confirming which situation you are in before you touch anything, because an actual input stall and a blind metric look identical in a dashboard that only plots input emit_records.

What this means

Fluentd’s monitor_agent (GET /api/plugins.json) exposes per-plugin counters. For input plugins, the key field is emit_records, a cumulative counter of records ingested. The version behavior is:

• Fluentd < v1.14.0: input metrics do not exist. The counter is not available at all.
• Fluentd v1.14.0 through v1.18.x: enable_input_metrics was introduced but defaults to false. Input emit_records is always 0 unless you opt in.
• Fluentd v1.19.0 (July 2025) and later: input metrics are on by default. The old --enable-input-metrics CLI flag is deprecated, and a new --disable-input-metrics flag exists for opting out.

Why this matters beyond the false alarm: the most important pipeline health check is comparing input rate against output rate. A sustained gap between them means data loss or unbounded buffer growth. Without input metrics, that check is impossible, and failures like silent data loss via overflow_action: throw_exception become much harder to detect. See how Fluentd actually works in production for the full pipeline model.

The diagnostic question is always the same: is the counter zero because nothing is arriving, or because nothing is being counted?

flowchart TD
  A["input emit_records = 0"] --> B{"output emit_records incrementing?"}
  B -- yes --> C{"Fluentd version?"}
  C -- "< v1.19.0" --> D{"enable_input_metrics true set?"}
  D -- no --> E["Blind metric: enable input metrics"]
  D -- yes --> F["Metrics on but counter flat: investigate input plugin"]
  C -- "v1.19.0+" --> G{"Opt-out set? (enable_input_metrics false / --disable-input-metrics)"}
  G -- yes --> E
  G -- no --> F
  B -- no --> H["Real pipeline stall: check source, in_tail, pos_file"]

Common causes

The first three causes are instrumentation gaps. The fourth is a real incident. The fastest way to separate them is the output side: if output emit_records is climbing, records are entering the pipeline from somewhere, and the input counter is lying to you.

Quick checks

All of these are read-only.

# 1. Check the Fluentd version (adjust for your package: fluentd, td-agent)
fluentd --version

# 2. Check whether input metrics are enabled in the config
grep -n "enable_input_metrics" /etc/fluent/fluentd.conf /etc/td-agent/td-agent.conf 2>/dev/null

# 3. Check whether the CLI flag is in use (deprecated on v1.19.0+, still functional)
ps -o args= -C fluentd | grep -o "input-metrics"

# 4. Sum input emit_records across all input plugins
curl -s http://localhost:24220/api/plugins.json | \
  jq '[.plugins[] | select(.plugin_category=="input") | .emit_records // 0] | add'

# 5. Sum output emit_records across all output plugins
curl -s http://localhost:24220/api/plugins.json | \
  jq '[.plugins[] | select(.plugin_category=="output") | .emit_records // 0] | add'

# 6. Confirm log sources are actively growing (adjust the path to your tailed files)
ls -la /var/log/containers/*.log 2>/dev/null | head -5

Run checks 4 and 5 twice, a minute apart. If the output sum increases and the input sum stays at 0, you have a blind metric, not a stall.

For per-plugin detail (for example, to see which input should be producing data):

curl -s http://localhost:24220/api/plugins.json | \
  jq '.plugins[] | select(.plugin_category=="input") | {id: .plugin_id, type: .type, records: .emit_records}'

In multi-worker mode, each worker exposes its own monitor_agent port (worker 0 on 24220, worker 1 on 24221, and so on), and in_tail must be pinned to a specific worker with <worker N>. Query the port for the worker that owns the input you care about, or sum across all worker ports.

How to diagnose it

1. Establish ground truth from the output side. Take two samples of output emit_records a minute apart. If it increments, data is flowing through the pipeline and at least one input is working. If it is also flat, stop here and treat this as a real ingestion or pipeline stall, not a metrics problem.

2. Check the version. fluentd --version. Anything below v1.19.0 with no enable_input_metrics true in <system> fully explains the zero counters. On v1.14.0 or older, the option does not exist and input metrics are simply unavailable.

3. Check the config and the command line. Grep for enable_input_metrics in the active config. On v1.19.0+, also look for an explicit enable_input_metrics false or --disable-input-metrics in the process arguments, since either one overrides the new default.

4. Verify which collection path you are reading. If your dashboards come from fluent-plugin-prometheus rather than the monitor_agent REST API, check the REST API directly with curl. As of 2026, fluent-plugin-prometheus does not ship a dedicated monitor that scrapes and exports input plugin metrics — upstream issue #105 remains open, and issue #199 was closed in July 2024 with a filter-based workaround instead of a native input-metrics monitor — so Prometheus can show nothing for inputs even on v1.19.0+ where the core agent collects them.

5. Rule out a real stall on the suspected input. If metrics are enabled (or the version is v1.19.0+) and a specific input’s counter is still flat while others move, that input has a genuine problem: a source that stopped writing, an in_tail rotation miss, a pos_file desync, or a port that stopped receiving. Check tailed file growth, pos_file modification time, and on v1.19.0+ the tracked_file_count field for tail inputs. The playbook for a real stall is different: see Fluentd input emit_records dropped to zero: ingestion has stopped.

Metrics and signals to monitor

Fixes

Enable input metrics in config (any version v1.14.0 through v1.18.x)

Add the flag to the <system> block:

<system>
  enable_input_metrics true
</system>

Then reload or restart. A SIGHUP reload can apply this, but partial reloads are a known failure mode in Fluentd, so verify after reloading that the input counters actually start incrementing, and check the Fluentd log for reload errors. See Fluentd config reload failed: SIGHUP that partially applies if the counters do not move after a reload. A full restart is the reliable path; with file-backed buffers the restart is safe for queued data, but memory-backed buffers lose unflushed chunks on any restart, so plan accordingly on production nodes.

The overhead of input metrics is negligible. The benchmark in the upstream change that flipped the default (PR #4966) showed 65.9s versus 67.8s for reading a 10GB file via in_tail with input metrics off and on, respectively.

Use the CLI flag (older versions, config you cannot easily change)

# Deprecated on v1.19.0+ but the only option short of a config change on older versions
fluentd --enable-input-metrics ...

This works on v1.14.0 through v1.18.x. Prefer the config option for anything permanent; command-line flags drift away from config management and get lost during package upgrades or systemd unit regeneration.

Upgrade to v1.19.0 or later

On v1.19.0+, input metrics are collected by default with no configuration. This is the durable fix, and it also gets you tracked_file_count for in_tail, which makes real input stalls much easier to catch. Two upgrade notes:

• If your config templates explicitly set enable_input_metrics false anywhere, they will override the new default. Audit for it before upgrading.
• The official v1.19+ Docker image changed the default UID, which can cause permission errors on mounted volumes (buffer directories, pos_files). The v1.19 image switched the default from the old Alpine-based UID 100 to a Debian-based fluent system user (USER fluent), and the image README documents permission errors on mounted volumes; check id -u inside the container and chown mounted dirs accordingly. Test the upgrade on one node before rolling it out.

If you rely on fluent-plugin-prometheus for scraping, upgrading Fluentd alone does not make input metrics appear in Prometheus. Until the plugin gains an input metrics monitor, scrape the monitor_agent REST API (/api/plugins.json) directly for input-side data, or use an agent that reads that endpoint.

Prevention

• Bake enable_input_metrics true into every config template for any Fluentd version below v1.19.0. Treat it as required, the same way you treat the monitor_agent source itself.
• Gate your input/output divergence alerts on metric availability. An alert on “input rate dropped to zero” that fires because the counter was never enabled trains everyone to ignore the alert that matters. After enabling, let the counter accumulate a baseline before turning the alert on.
• Record the version-dependent behavior in your runbooks. Mixed-version fleets are normal during upgrades. A runbook that says “input emit_records of 0 means ingestion stopped” is wrong for every node below v1.19.0 without the flag.
• Verify after every config change and upgrade. Two curl calls a minute apart against /api/plugins.json confirm input counters are incrementing. Put this in the post-change checklist.
• Plan the v1.19.0+ upgrade so the flag becomes unnecessary, and audit templates for explicit opt-outs first.

How Netdata helps

• Netdata collects Fluentd plugin metrics from the monitor_agent endpoint, so input and output emit_records land on the same timeline. The divergence check this article is about becomes a visual comparison instead of two curl calls.
• Per-plugin breakdowns let you spot the difference between “all inputs at zero” (instrumentation gap) and “one input at zero” (real stall on that source).
• Correlating input rate with buffer_queue_length and buffer_available_buffer_space_ratios catches the case where blind input metrics hid the early stages of backpressure.
• tracked_file_count, rotated_file_count, and throttled_log_count for in_tail (version-dependent) sit next to the emit counters, so rotation-related ingestion gaps are visible in the same view.
• Historical retention means you can confirm whether a counter was ever nonzero, which immediately separates “never enabled” from “recently stalled”.

Related guides

• Fluentd input emit_records dropped to zero: ingestion has stopped
• How Fluentd actually works in production: a mental model for operators
• Fluentd config reload failed: SIGHUP that partially applies
• Fluentd emit_error_count: the number-one under-monitored data-loss signal
• Fluentd buffer queue length growing: the output cannot keep pace with the input
• Fluentd buffer available space low: computing time-to-overflow before it fires
• Fluentd BufferOverflowError: buffer space has too many data
• Fluentd failed to flush the buffer: the output cannot deliver and retries begin
• Fluentd drop_oldest_chunk_count incrementing: confirmed buffer data loss
• Fluentd broken pipe / connection reset: dropped output connections and LB timeouts
• Fluentd memory vs file buffer: why the default buffer loses data on restart
• Fluentd CrashLoopBackOff: rapid restart cycling in Kubernetes