Below is the list of blog posts with the “Telemetry pipeline management” tag.

The choice here is not between two interchangeable log tools. It is a choice about where you want parsing, routing, and failure handling to live. Filebeat runs close to the source and keeps collection small. Logstash sits in the middle of the flow and takes on filtering, enrichment, and fan-out. That architectural difference matters more than a feature checklist. Pick the narrower tool when your logs have one destination and your parsing rules are modest.

Filebeat and Vector both move logs, but they solve different design problems. Filebeat is a shipper that fits neatly into Elastic-centric pipelines. Vector is a data pipeline runtime that can collect, reshape, split, and forward the same stream to several destinations before storage. The cost of choosing badly does not show up on day one. It shows up later as duplicate agents, extra relay tiers, backend-specific parsing rules, or migration work when a second destination appears.

As organizations collect more telemetry data, their pipelines grow in complexity and scale. Telemetry pipelines are dynamic, continually adjusted to improve data quality, reduce costs, and meet evolving observability requirements. At this scale, even small configuration changes can significantly affect how much data moves through your pipeline. Without clear visibility, you rely on assumptions. Did the new filtering rule actually reduce the amount of data you’re sending to the SIEM?

If you are choosing between Fluent Bit and Filebeat, the real question is where you want routing, parsing, and failure handling to live. Pick the wrong default, and you create config sprawl, brittle pipelines, and extra work every time your backend or deployment model changes. Choose Fluent Bit when the agent itself needs to behave like a small pipeline, and choose Filebeat when your log path ends inside Elastic and you want the shipper to match Elastic’s operating model.

Telemetry data is the stream of measurements that instrumented devices, applications, and services continuously emit to a central system so engineers can monitor behavior, diagnose problems, and make informed decisions in real time and over the long term. In this article, we’ll look at what telemetry data means in practice for modern software, networks, and cloud platforms: how it’s produced, what kinds of signals it carries (logs, metrics, traces, and more), and why it has become essential for observability, performance, and security at scale.

Most discussions about OpenTelemetry pipelines focus on getting data from point A to point B. Collect telemetry, maybe convert the format, forward it to a backend. That’s the minimum viable pipeline, and it’s where most tooling stops. But a pipeline that only moves data is a pipe, not a processing layer. The telemetry arriving at your observability platform or SIEM is only as useful as the context it carries. A raw log entry saying "connection from 198.

OpenTelemetry has become the de facto standard for telemetry data. Nearly 50% of surveyed cloud-native end-user companies have adopted it, and the project ranks as the second-highest-velocity initiative in the CNCF, behind only Kubernetes. The direction is clear: if your infrastructure doesn’t speak OpenTelemetry, it will increasingly be left out of the observability conversation. But adopting OpenTelemetry across an entire infrastructure is a different problem than adopting it in a greenfield application.

Choosing between Fluent Bit and Fluentd is an architecture decision, not a product shootout. Both projects live under the CNCF Fluent umbrella and share a common lineage at Treasure Data, but they target different roles in a logging pipeline. Fluent Bit is a C-based telemetry agent designed for low-overhead collection at the edge. Fluentd is a Ruby-and-C data collector built for aggregation, transformation, and multi-destination routing. The practical question is not which one is better — it’s where each one belongs in your stack, and whether you need both.

Why the federal agency breach shows that standardized telemetry formats aren’t optional anymore When CISA analyzed the federal agency breach that went undetected for three weeks, they identified a familiar pattern: EDR alerts existed but weren’t continuously reviewed. Security teams had visibility tools, but critical signals got lost in the noise. What the advisory doesn’t detail—​but every security practitioner knows—​is the infrastructure nightmare hiding behind that simple statement. Those unreviewed alerts likely came from dozens of sources, each speaking its own dialect of security telemetry.

Collecting, processing, and forwarding logs and metrics at scale. OpenTelemetry provides a common instrumentation model that makes it easier to collect telemetry data across distributed systems, and many modern applications are adopting it as a standard for generating logs and metrics. However, in practice, you still need to collect, process, and shape the data before it becomes useful. You cannot simply forward raw telemetry data downstream without risking that your observability platform becomes expensive storage instead of a means of maintaining visibility into your environment.

A practical approach to converting existing logs into modern observability. OpenTelemetry promises a vendor-neutral standard for observability, consistent telemetry, and the flexibility to change backends without rewriting everything. In practice, however, OpenTelemetry adoption often runs into a familiar obstacle: reality. Here’s a common scenario. You’re eager to improve observability, but your environment includes a mix of legacy applications, network devices, and third-party systems. Many of these were never designed for modern instrumentation, and changing them is risky, expensive, or simply not an option.

How a federal agency’s monitoring gaps turned a containable incident into a three-week nightmare In September 2025, CISA responded to a federal agency breach that security teams could have stopped in hours. Instead, threat actors roamed the network undetected for three weeks. The damage? Multiple compromised servers, web shells planted across the infrastructure, and a persistent foothold that took significant resources to remediate. The root cause wasn’t a zero-day exploit or sophisticated malware.

Some still think telemetry is a futuristic concept, but it isn’t. It’s already integral to the smooth running of everything from websites, e-commerce platforms and mobile apps to manufacturing, traffic control and much, much more. And it all begins with the humble data log. From the earliest days of computing, programmers have recorded useful information — often in a file — to help track and react to potential threats and understand what’s going on "under the hood" of their IT infrastructures.

It’s Halloween season, and while everyone else is worried about ghosts and goblins, you—the sysadmin holding the fort—know the real terror: that dusty print server in the corner that’s been running firmware from 2014. Or the Raspberry Pi someone set up to monitor the server room temperature "temporarily" three years ago. Or the CEO’s personal tablet that absolutely must connect to the internal network because "it’s just easier this way.

Organizations invest heavily in sophisticated monitoring platforms, deploy countless agents across their infrastructure, and build elaborate dashboards to track every metric imaginable. Yet amid this pursuit of comprehensive visibility, a dangerous blind spot often emerges: the observability system itself becomes unobservable. This meta-problem represents one of the most insidious risks in modern infrastructure management. When telemetry collection fails silently—​whether due to misconfiguration, infrastructure changes, or system failures—​operations teams continue making critical decisions based on incomplete or stale data, unaware that their digital nervous system has developed gaps in coverage.

The AI gold rush has arrived, and organizations worldwide are making unprecedented investments in cutting-edge accelerator hardware. GPU clusters worth millions of dollars are being deployed at breakneck speed, with companies betting their competitive futures on these silicon powerhouses. Yet beneath the excitement of acquiring the latest H100s or MI300s lies a sobering reality: the most expensive part of your AI investment isn’t the initial purchase—​it’s ensuring that hardware delivers value every single moment it’s operational.

Do you ever feel like you’re drowning in data? From endpoint logs and firewall events to database auditing and cloud metrics, the sheer amount of data is overwhelming. While telemetry data is crucial for threat detection, incident response, and compliance, it also brings a major challenge: log noise. Log noise obscures meaningful security signals. If left unchecked, you risk increased false positives, overloading security tools, higher SIEM licensing costs, and, most importantly, SOC alert fatigue.

Today, most enterprises use a security log analytics solution or SIEM (Security Information & Event Management), but analytics are only as good as the data fed into your solution. If you’re missing data sources or are failing to extract full value from the data, you won’t see the big picture. This is an issue new customers commonly mention to NXLog. That’s why one of our key goals is to provide a solid data collection layer that ensures all relevant data is collected and properly fed into the SIEM.

NXLog Agent plays a vital role in aggregating, processing, and forwarding logs to centralized platforms for analysis. Whether it’s system logs, application logs, or security audit trails, these agents are often the first line of visibility into what’s happening in your environment. In many setups, especially large-scale infrastructures, NXLog Agent relays act as crucial intermediaries, collecting logs from edge systems and forwarding them to a SIEM or log analytics platform.

People think about logs as one of the biggest chores in the IT industry. Well, that does not necessarily need to be true. If you adhere to some fundamental log management best practices, the value you could get out of them quickly outweighs the effort put into managing them. Logs can easily become the best friend of IT teams looking to keep their systems secure, meet compliance requirements, and maintain a smoothly running network.

Logs play a critical role in IT infrastructure, and choosing the right log management solution is key to effective operations. This guide explores the core principles for selecting a solution that aligns with your log collection and management needs. Given the wide range of options available, we categorize them into three main groups for clarity. End-to-end Log Management Solutions Security Information & Event Management (SIEM) Application Performance Monitoring and Observability (APM)

New CEO appointed as the company’s founder, former CEO & CTO, transitions to a dedicated CTO role, focusing on innovation in observability and telemetry pipeline management market. (LONDON, UK) – 19th December 2024 - NXLog, a leading technology provider of log management solutions, announced the appointment of Harald Reisinger as its new Chief Executive Officer. Co-founder and former CEO Botond Botyánszki will transition to the Chief Technology Officer (CTO) role.

With an ever-expanding choice of technologies on the market, navigating the range of open-source observability tools can be a challenge. Which is why, when it comes to managing complex multicloud environments and their services, standardization is crucial. Here’s where OpenTelemetry (OTel) can play a key role. Developed through the merger of OpenCensus and OpenTracing, OpenTelemetry has become the new standard for open-source telemetry. Discover what OTel is, the types of telemetry data it encompasses, its potential benefits, and how NXLog can support your OpenTelemetry ecosystem.

Data logging and event monitoring have become essential to provide security and performance monitoring of business operations. However, the vast volume of logs generated can lead to significant challenges, including high costs and inefficiencies. Many companies collect an excessive number of logs, often missing out on the most critical security-related events. The majority of these logs, known as log noise, offer little to no value to security analysts and can obstruct timely access to high-priority security events.

Back in the day, Gordon Moore made relatively accurate observations and projections about the exponential growth of transistors on semiconductors. It still amazes me, yet very few predicted the incredible growth of system interconnectedness and the vast amount of data it generates. It is estimated that 90% of all data was created in the last two years. Given that everything is connected, the need for telemetry is growing at an unprecedented rate, and so is the need to efficiently channel and manage telemetry data.

The global Security Information and Event Management (SIEM) market is big business. In 2022, it was valued at $5.2 billion, with analysts projecting that it will reach $8.5 billion dollars within five years. It’s a highly consolidated market dominated by a few major players in the information security field. They want your business, and they don’t want to lose it. As companies ship more and more data to their respective solutions and make use of more and more features, they become specialized and dependent on a vendor.

The European Union’s General Data Protection Regulation (EU GDPR) came into force on 25 May 2018. Many of us remember the influx of marketing emails around this time, with companies updating their privacy policies and asking for the consent of around 450 million Europeans to continue using their personal data. An often misunderstood participant of this compliance quest is log data—​a source potentially rich in protected personal data. So, how does the GDPR apply to an organization’s log data?