Logstash address already in use: input port conflicts on Beats, TCP, and HTTP

Logstash logs Address already in use (a Java BindException) when a listening input plugin, typically the Beats input on 5044, a tcp, http, or syslog input, tries to bind a port that is already taken. The bind fails, the input cannot start, and the pipeline that owns it stops ingesting.

The blast radius is the problem. In a multi-pipeline deployment the JVM usually stays alive and the other pipelines keep running. Process-level checks stay green, the monitoring API answers, and one pipeline silently stops ingesting. Upstream Filebeat agents queue, syslog senders drop, and nobody notices until someone asks where a stream of logs went.

There are three root causes: a previous Logstash instance still holding the socket, another service already bound to the port, or the same port declared twice inside your own configuration. The diagnosis is the same in all three: find the holder, then either stop it or move your listener.

What this means

When a Logstash input plugin starts, it opens a server socket on its configured port. The kernel refuses the bind if that port is already in LISTEN (or, for UDP inputs, already bound) by any process on the same address. The Java runtime surfaces this as java.net.BindException: Address already in use, which Logstash logs as a plugin error. The input retries, but the port does not free itself, so the retry loop never succeeds.

Version-dependent behavior matters here:

Logstash 8.x: an input that fails to bind retries while the pipeline can still report healthy. The pipeline looks fine in the API while receiving nothing. This is the “living dead” variant specific to port conflicts.
Newer Logstash versions with updated input plugins: port-binding failures are propagated to the pipeline health report, so the pipeline shows as unhealthy instead of silently retrying. If you are on 8.x, do not trust a green health report for listening inputs.

There is also a special case: the monitoring API itself. Logstash’s API binds to 9600 by default and falls back across 9600-9700. If the entire range is occupied, the whole process fails to start, not just one pipeline. That failure mode is covered in Logstash API unreachable on port 9600.

Common causes

Cause	What it looks like	First thing to check
Previous Logstash instance still running	BindException on the same port right after a restart or redeploy; an old JVM process survives	`ss -tlnp` on the port; `pgrep -f org.logstash.Logstash` returns more than one PID
Another service on the port	Conflict appears with no Logstash restart; rsyslogd, another agent, or a stray app holds 5044/514/8080	`ss -tlnp` shows a non-Java process owning the port
Same port in two pipelines	Second pipeline fails at startup or after a config reload; first pipeline works	`grep -rn "port =>" /etc/logstash/conf.d/` shows the port declared twice
Duplicate config file in conf.d	Conflict appears after someone made a `.backup` or `.bak` copy of a pipeline config inside `conf.d/`	Look for backup copies in the directory Logstash loads
Docker port mapping collision	Conflict only in containerized runs; two containers or a host process publish the same host port	`docker ps` port mappings on the host

Quick checks

All read-only and safe to run during an incident.

# 1. Find what holds the port (example: Beats 5044)
ss -tlnp | grep -w 5044

# 2. Count Logstash JVM processes. More than one is your answer.
pgrep -af org.logstash.Logstash

# 3. Confirm the error in the log and note the pipeline name
grep -Ei 'address already in use|BindException' /var/log/logstash/logstash-plain.log | tail -n 20

# 4. List which pipelines the JVM thinks it is running
curl -sS http://127.0.0.1:9600/_node/stats/pipelines?pretty | grep -E '"[a-zA-Z_-]+":\s*\{'

# 5. Check pipeline health (on 8.x a bind failure may still report healthy)
curl -sS http://127.0.0.1:9600/_health_report?pretty

# 6. Find every port declaration in your configs, counted, so duplicates stand out
grep -rhoE 'port\s*=>\s*"?[0-9]+"?' /etc/logstash/conf.d/ /etc/logstash/pipelines.yml 2>/dev/null | sort | uniq -c | sort -rn

# 7. Check for stray config copies in the directory Logstash loads
ls -la /etc/logstash/conf.d/

# 8. Verify input activity per pipeline (zero on the failed one)
curl -sS http://127.0.0.1:9600/_node/stats/pipelines?pretty | grep -E '"in"|"out"'

How to diagnose it

Identify the port from the log. The BindException stack trace names the input plugin and usually the pipeline. Grep the Logstash log for Address already in use and note which pipeline and plugin raised it.
Find the socket holder. Run ss -tlnp | grep -w <port> (use ss -ulnp for UDP inputs such as syslog/udp). The output gives you the PID and process name. This single command resolves most incidents.
Classify the holder:
- Another java process running Logstash: an old instance survived a restart, an upgrade, or a failed service stop. Confirm with pgrep -af org.logstash.Logstash. This is also the likely cause when the conflict appears immediately after a deploy.
- A different service (rsyslogd, fluentd, a monitoring agent, an application): someone else owns that port now, or a package update enabled a listener that was not there before.
- The same Logstash process: the port is declared twice inside your own config. Move to step 4.
Hunt the duplicate declaration. Run check 6 above. Two pipelines in pipelines.yml pointing at configs that declare the same input port is the classic case: Logstash runs both configs in one JVM, the first bind wins, the second pipeline fails. Also check for backup copies inside conf.d/ (for example pipeline.conf.bak): when path.config points at a directory, Logstash concatenates the files it reads there, so a “backup” file can silently double your listeners.
If nothing holds the port but the bind still fails, check address family and interface: an input binding 0.0.0.0:5044 conflicts with a listener on :::5044 on dual-stack hosts, and an explicit host => "10.0.0.5" limits the conflict check to that address. In containers, verify the port is published (-p 5044:5044). A missing publish does not cause this error, but a wrong mapping makes the listener unreachable after a successful bind, which is easy to confuse with a bind failure.
Confirm the partial-outage scope. List pipelines from the stats API and compare against pipelines.yml. Any pipeline missing from the API, or present with zero events.in while its source is active, is affected. On 8.x the health report may not reflect the bind failure.

flowchart TD
  A[BindException in Logstash log] --> B[ss -tlnp on the port]
  B --> C{Who holds it?}
  C -->|Old Logstash JVM| D[Stop the stale instance]
  C -->|Other service| E[Stop it or move your input port]
  C -->|Same Logstash JVM| F[Find duplicate port in pipelines.yml or conf.d]
  F --> G[Give each pipeline a distinct listener port]
  B -->|Nothing holds it| H[Check host/address family and container port mapping]

Metrics and signals to monitor

Signal	Why it matters	Warning sign
Per-pipeline `events.in`	A pipeline whose input never bound shows zero inflow	`events.in` flat at zero while the source is sending
Pipeline presence in `/_node/stats/pipelines`	A pipeline that failed to start may be absent entirely	Expected pipeline ID missing from the API response
Health report pipeline status	On versions with bind-failure propagation, the failure surfaces here	Pipeline reported unhealthy for a listening input
`reloads.failures`	A reload that introduces a duplicate port fails and leaves the old config running	Any non-zero increment after a config deploy
BindException lines in the log	The input restart loop logs each failed bind attempt	Repeating “Address already in use” entries in `logstash-plain.log`
Upstream sender state (Filebeat registry, Kafka lag)	Senders back up when the listener is down	Filebeat registry not advancing, consumer lag growing on the affected stream

The key insight: on Logstash 8.x, none of the standard health signals catch this reliably. Process liveness is green, the API answers, and the health report can claim the pipeline is fine. Zero events.in per pipeline, compared against the expected pipeline list, is the only reliable detector.

Fixes

Stale Logstash instance holding the port

Stop the old instance properly: systemctl stop logstash against the unit that owns it, or a plain kill <pid> for a manually started process. Then start the intended instance. Avoid kill -9: it frees the port but skips graceful shutdown, which risks persistent queue corruption on the killed instance (see memory queue vs persistent queue failure modes). If a stop-and-restart cycle caused the conflict, find out why the old process survived: a wrapper script that does not track the real Java PID, or a service manager timeout during shutdown.

Another service owns the port

Decide who should own it. If the other service is legitimate (for example rsyslogd bound to 514), move the Logstash input to a different port and update senders, or run the syslog flow through the other service’s forwarding instead of fighting for the port. Note that binding below port 1024 also requires root or CAP_NET_BIND_SERVICE; a permission-denied bind failure looks similar in the log but is a different fix.

Same port declared in two pipelines

There is no sharing: each pipeline that opens a listening input needs its own port. Assign distinct ports per pipeline and update the senders (Filebeat output.logstash hosts in filebeat.yml, load balancer backends) accordingly. If two pipelines genuinely need the same data, merge them into one pipeline and branch with conditionals, or use pipeline-to-pipeline communication with one shared input.

Duplicate config file in conf.d

Move the backup copy out of conf.d/. Keep backups outside the directory Logstash loads, or in version control. Then reload and confirm via reloads.successes and the reappearance of the pipeline in the stats API.

Prevention

One port, one pipeline, enforced in review. Treat input port assignments like a registry: document them, review config PRs for duplicates, and lint grep-able port declarations in CI.
Keep conf.d/ clean. No editor swap files, no .bak copies. Logstash loads the directory, not a single file.
Monitor per-pipeline events.in, not just the process. This is the only signal that catches the 8.x silent-retry failure mode.
Alert on reloads.failures. A failed reload after deploying a port change leaves the old config running and masks the new conflict. See config reload failures and invisible drift.
Upgrade when you can. On versions where bind failures propagate to the health report, this class of failure becomes visible instead of silent.
Check ports before restarts in automation. A pre-start ss -tln assertion against declared ports turns a 3 a.m. partial outage into a failed deploy.

How Netdata helps

Netdata’s Logstash collector polls the monitoring API per pipeline, so a pipeline with zero events.in stands out against its peers instead of hiding in an aggregate.
Correlating per-pipeline input throughput with process liveness catches the partial outage this error creates: the JVM is up, one pipeline is dark.
Charts of reloads.failures next to pipeline presence make it obvious when a deploy introduced a conflicting listener and the reload failed.
ML anomaly detection on input rate per pipeline flags a stream that dropped to zero even when you never set an explicit threshold for it.
Because Netdata also watches the host, socket listener state and the Logstash pipeline metrics are on the same dashboard, which shortens the “who holds this port” question to one screen.