Fault tolerance | Telemetry collection | Telemetry pipeline management
Log shipper offline buffering: how to keep logs safe when the destination goes down
Offline buffering is a log shipper’s ability to store events in memory or on disk while the destination is unreachable, then forward them automatically once the connection recovers. Without it, every SIEM outage, network partition, or agent restart becomes a permanent gap in your security telemetry.
Every SIEM eventually goes down. So does the WAN link to your central collector, and eventually the host running the agent itself. None of these events should cost you log data, but whether they do depends on how your log shipper buffers when it can’t deliver data to the destination.
Telemetry collection | Fault tolerance
How to handle log rotation without losing events
Log rotation is supposed to be routine maintenance. But if your collector reads a file while another process renames, truncates, or compresses it, events can slip through the gap — and you often won’t notice until you go looking for a log that isn’t there. For a security team, that gap is a blind spot: a detection that never fired, an audit trail with a hole in it, a control you can’t prove was working.
Fault tolerance
High Availability and Fault Tolerance
Imagine trying to buy tickets for your favorite band’s concert, only to find the website down just minutes before they sell out. Or logging into the cloud to look through your cherished digital photos and discovering they’ve been lost because of a data center failure.
These scenarios are — at best — frustrating for you. But, for your customers, they can erode trust and damage your business’s reputation.
That’s why organizations invest in strategies like high availability (HA) and fault tolerance (FT).
Fault tolerance | Log forwarding
Reliable delivery of logs - can you trust TCP?
When considering your log collection strategy, a decision you have to make is which transport protocol to use to transfer logs from source to destination. The choice is often between the two most commonly used protocols, UDP (User Datagram Protocol) and TCP (Transfer Control Protocol). Which one to use depends on the type of logs you need to transfer, and whether performance or reliability is more important.
This blog post will compare these protocols, discuss why TCP is usually the preferred choice, and provide some options to further increase log delivery reliability with NXLog.