Forwarding logs with NXLog

So, you managed to read through all the compliance mandates that are required for the industry you are in. And, during the mandatory consultation you had with your company’s IT security expert and network manager you came to an agreement on which logs to collect and carefully selected their final destination. Which — in most cases — is usually some kind of analytics system or SIEM technology where log data can be analyzed and stored based on your business requirements. Great! The future looks bright, but one question still remains to be answered. How are the logs going to get there?

Well, the answer seems obvious. There’s really no other practical way, except by sending them over the network. Although the answer is correct, it’s easier said than done. There are a few things to consider, such as the volume of data, which protocols to use, and of course security, for sending sensitive data over the network. Secure transport of data becomes exponentially important if logs need to leave your company’s internal network to be sent over public channels in order to get them into a cloud-based SIEM, which is often the case.

Fortunately, NXLog already comes with a complete set of software tools to match your industry’s specific logging needs. Whether your concern is security, size, flexibility, or any other requirement you might have regarding your logs, its modular design and scalable, distributed logging architecture can meet the requirements of almost any scenario. In this post, we’ll take a look at some of the most popular use cases.

Log forwarding with TCP: NXLog utilizes the TCP protocol in its om_tcp module to send log data. Inheriting the capabilities of the protocol, this module provides reliable and fast log transmission. Because of the absence of security, this method is mostly recommended for forwarding log data within a secured network, such as an internal network. Using this module is a straightforward method to send and receive logs to any node by configuring its IP address or hostname and a port number.
Log forwarding with UDP: Using UDP to forward log data is implemented in the om_udp module in NXLog. Like TCP, it is another simple and efficient way to forward logs over the network and it is configured the same way as TCP. However, it is important to remember that this protocol should only be used for data that can tolerate loss since UDP does not provide delivery acknowledgment, nor will it retransmit data packets if they are dropped.
Log forwarding with SSL: If security is the highest priority, we recommend that you use SSL to forward event logs in a secure manner. The NXLog om_ssl module can be used to leverage the benefits of SSL. It works just like the TCP module, except it performs an SSL handshake when the connection is first established. All data transmissions are then sent through a secure channel.
Log forwarding with HTTP(s): With NXLog’s om_http module it is possible to send log data to a specified URL. This can be done in either plain HTTP or in HTTPS mode for added security. The added security of HTTPS comes from using SSL as an additional protocol.
Log forwarding with Kafka: Apache Kafka is an increasingly popular way of forwarding data. Technologies such as Microsoft Azure Event Hubs use it to accept data. With NXLog, the om_kafka module can be used to send log data with Kafka.
Forwarding logs to Azure Sentinel: Microsoft Azure is a trusted and preferred cloud platform of many enterprises. The om_azure module can natively send log data to Microsoft Azure Sentinel. With om_azure, event data is sent in batches which significantly reduces the delay caused by HTTP responses, thus increasing the performance of the Azure Sentinel server.
Forwarding compressed logs: Messages can be forwarded using the om_batchcompress module which accumulates the data to be sent in batches and compresses it before being sent to a desired destination. Compressed data notably reduces storage space and makes processing of files faster, overall, increasing system performance.

This is only a short list compared to the multitude of options NXLog offers for forwarding log data from one point to another. Its rich feature set is one of the main reasons why many companies continue to choose NXLog as their log collection software of preference.