Filebeat vs Logstash: when the shipper is enough and when you need a pipeline

Filebeat monitors files or locations, starts one harvester for each file, hands those events to libbeat, and publishes them to the output you configured. Elastic documents package, container, and Kubernetes deployment models, including a DaemonSet pattern for Kubernetes that mounts /var/log/containers on every node. Linux DEB and RPM packages also ship with a systemd unit.

The constraints are just as important as the layout. Filebeat allows only one output, so any dual-write, fan-out, or destination-specific branching has to happen later. Elastic deprecated the old log input in 7.16 and turned it off in 9.0, so new file tailing work should use filestream. Elastic still supports Filebeat modules, but it recommends Elastic Agent integrations for new work.

Elastic also publishes Filebeat in two licensing forms. The default download is under the Elastic License, and there is an OSS-only build for teams that want the Apache 2.0 distribution.

Logstash runs a server-side event pipeline with inputs, filters, and outputs. Its execution model is explicit: inputs write to a central queue, pipeline workers pull batches from that queue, filters transform the events, and outputs ship them onward. Elastic ships it as tar.gz, zip, deb, rpm, and Docker images, and the current line bundles JDK 21 by default.

That design gives you more room to shape traffic. A single instance can run multiple pipelines, and pipeline-to-pipeline communication supports distributor, forked-path, and output-isolator patterns inside one process. If you turn on centralized pipeline management, Elastic marks it as a subscription feature, and local pipeline files stop applying.

Logstash follows the same packaging split as Filebeat. The default packages are under the Elastic License, and Elastic also provides an OSS-only Logstash build.

Elastic’s own comparison page says Beats have a small footprint and use fewer system resources than Logstash. Filebeat’s memory queue defaults to 3200 events, and its disk queue can preserve pending data across restarts. If you point Filebeat at Logstash, the Logstash output batches up to 2048 events by default.

Logstash exposes more sizing knobs because it does more work. Its memory queue is not configured as a single event count. The upper bound is pipeline.workers x pipeline.batch.size, with defaults of CPU core count and 125 events. On an 8-core host, that means 1,000 in-flight events per pipeline before tuning. If you need disk-backed buffering, persistent queues exist, but they are disabled by default and sized per pipeline.

Filebeat scales by repeating a simple unit: one agent per host or one pod per Kubernetes node. That stays easy to reason about when each instance reads local files and forwards to one destination.

Logstash scales in a different direction. One pipeline can accept several inputs and send to several outputs, and multiple pipelines let you give different flows their own workers, queues, and durability settings. Elastic also notes that a blocked output in one pipeline does not backpressure another pipeline when the flows are separated this way. That is valuable during migrations, during partial outages, and when one destination has stricter latency than another.

Filebeat has the shortest path into Elastic. The quick-start flow shows filebeat setup loading the recommended index template and sample dashboards, and dashboard loading is built into the product. If Filebeat modules connect directly to Elasticsearch, ingest pipelines are set up automatically.

That path gets longer when you insert Logstash. Elastic states that if your output is Logstash instead of Elasticsearch, you must load the index template, dashboards, and ingest pipelines manually. Logstash gives you a central place to parse and route logs, but it is not the shortest route to a first dashboard.

Neither Filebeat nor Logstash is a log store. If you need ad hoc search, saved investigations, or dashboards, those features come from Elasticsearch and Kibana or from another backend. Filebeat gets you there with less setup because modules package input settings, ingest pipeline definitions, fields, and sample dashboards for common log types.

Logstash has visibility of its own, but that visibility is operational rather than analytical. The monitoring APIs expose node info, JVM stats, plugin inventory, pipeline runtime stats, and hot threads. The pipeline viewer highlights topology, throughput, CPU anomalies, and plugin latency so you can find ingestion bottlenecks. None of that replaces a search UI for your application logs.

Filebeat can do more than plain forwarding. Processors handle field cleanup, JSON decoding, metadata enrichment, event dropping, and multiline assembly. The script processor runs JavaScript when the built-in processors are not enough. That is enough for edge normalization and for trimming noise before transport.

Logstash takes over when transformation becomes the main design problem. Grok parses variable text with regular expressions. Dissect tokenizes stable formats without regex. Mutate handles field renames, replacements, and conversions, and standard pipeline configuration supports conditionals plus several outputs. Add pipeline-to-pipeline communication, and Logstash becomes a routing layer rather than just a parser.

Neither product offers first-class alerting on stored log data by itself. Filebeat depends on the destination for rules and notifications. What it does offer is a direct path to Elastic assets when you keep the pipeline simple.

Logstash answers a different operational problem. Persistent queues reduce loss during restarts and downstream interruptions. Dead letter queues capture selected failures for later inspection instead of dropping them silently. The main paid-tier boundary in this comparison is centralized pipeline management, which Elastic lists as a subscription feature.