syslog-ng and NXLog are both powerful log collectors providing flexible log processing. However, you might be in a position where you need to switch from syslog-ng to NXLog. Whether it’s because syslog-ng doesn’t support an operating system or you want to upgrade your log collection solution to one that can be centrally managed, converting your syslog-ng configuration to NXLog is a simple task.

How do syslog-ng and NXLog differ?

syslog ng to nxlog syslog-ng and NXLog are alike in many ways. They can both parse, process, normalize, and forward logs over the network via secure transport, all at impressive speeds. They also implement caching mechanisms for reliable message delivery and their own protocol for more efficient log forwarding between nodes. However, NXLog’s rich feature set supports more input sources in more formats and ships to more output destinations.

Some of the predominant differences include the following:

  • syslog-ng does not support macOS. On the other hand, NXLog can be installed on macOS and has specialized modules for collecting macOS system and security logs.

  • Unlike syslog-ng, NXLog is made to be centrally managed. The agent includes a remote management module providing RESTful and SOAP APIs, so you can easily integrate it with whatever SCM tool you already use. In addition, we make your life easy with NXLog Manager and NXLog Minder.

  • NXLog supports custom extensions in Go, Perl, Python, Ruby, and Java. In comparison, syslog-ng only supports Python, which is still a preview feature at the time of writing.

  • Whereas syslog-ng requires Java to be installed when using certain destinations, such as Elasticsearch and HDFS, NXLog has no such requirements.

What can NXLog do that syslog-ng can’t?

Here is a non-exhaustive list of input/output sources and log formats NXLog supports out-of-the-box that are not available in syslog-ng.

Input Output Log format

Generic HTTP(s) input

Amazon S3

GELF

Google Cloud Logging (formerly Stackdriver)

Microsoft Azure - tables and blobs (in addition to Log Analytics workspaces, also supported by syslog-ng)

NetFlow

macOS - Apple’s unified logging system (ULS) and Endpoint Security auditing system

Microsoft 365 - Microsoft Graph and Office 365 Reporting web service (in addition to the Office 365 Management API, also supported by syslog-ng)

Microsoft DNS Server

Microsoft Azure - tables, blobs, and Log Analytics workspaces

Google Chronicle

SAP audit logs

Network packet capture

Raijin Database

W3C

Redis

Redis

Salesforce

ZeroMQ

Okta

File integrity monitoring

Windows Registry monitoring

ZeroMQ

syslog-ng and NXLog configuration comparison

syslog-ng v.s. NXLog nomenclature

Before going any further, it would be wise to familiarize yourself with how both products refer to configuration elements. We cover the basics in the following matrix.

Configuration element syslog-ng NXLog

Log ingestion element

source/source driver

input module

Log output element

destination/destination driver

output module

Log routing element

log path

route

Log filtering element

filter block

xm_pattern module or via regular expressions

Log parsing element

parser block

extension modules or via regular expressions

Log transformation element

rewrite rule

xm_rewrite module or Exec statements

Configuration parameters

options

directives

Values parsed from log records

macros

fields

NXLog on Linux

Both syslog-ng and NXLog use a file-based configuration system on Linux platforms. The default configuration file location for the two products is:

syslog-ng

/opt/syslog-ng/etc/syslog-ng.conf

NXLog

/opt/nxlog/etc/nxlog.conf

A key difference is that syslog-ng automatically disables Rsyslog and renames the /etc/init.d/sysklogd init script to /etc/init.d/sysklogd.backup upon installation. NXLog doesn’t make changes to other components on the system. Therefore, if you’d like to disable Rsyslog, you must do it manually by executing the following commands:

$ sudo systemctl disable --now rsyslog
$ sudo systemctl force-reload systemd-journald
$ sudo systemctl restart nxlog

Ready to start? Let’s dive right into the configuration.

Convert syslog-ng internal and Linux system log sources

The default syslog-ng configuration on Linux collects system and kernel logs and syslog-ng’s own internal logs. So let’s see how easy it is to replicate these sources in NXLog.

 
source s_local {
    internal();
    system();
};

down arrow

 
<Extension syslog>
    Module         xm_syslog
</Extension>

<Input kernel>
    Module         im_kernel
    Exec           parse_syslog_bsd();
</Input>

<Input systemd>
    Module         im_systemd
</Input>

<Input devlog>
    Module         im_uds
    UDS            /dev/log
    FlowControl    FALSE
    Exec           parse_syslog_bsd();
</Input>

<Input internal>
    Module         im_internal
</Input>

Start by choosing which Linux logs you want to collect:

  • Use the im_kernel module to collect logs from the kernel log buffer

  • Read daemon messages from the systemd journal socket with im_systemd

  • Collect other user-space messages from the /dev/log socket with im_uds

  • Set up auditing rules and collect logs directly from the kernel with the im_linuxaudit input module

In this NXLog configuration, we collect the first three logs corresponding to syslog-ng’s system() source. Since the kernel and user-space messages are in syslog-BSD format, we’re using the parse_syslog_bsd() procedure provided by the xm_syslog extension to parse log records into fields.

NXLog can also collect its own internal logs with the im_internal module.

Convert a syslog-ng file source

The use of log files is still widespread in the logging sphere, especially on Linux platforms. So let’s convert a syslog-ng file source collecting Apache access logs to NXLog.

 
source s_apache_access {
    file(
        "/var/log/apache/access.log"
        follow-freq(1)
        flags(no-parse)
    );
};

down arrow

 
<Input apache_access>
    Module          im_file
    File            '/var/log/apache/access.log'
    PollInterval    1
    Exec            $Message = $raw_event;
</Input>

Use the im_file input module to collect file-based logs. You can specify the File directive multiple times and use wildcards. The PollInterval directive, set to 1 second by default, corresponds to the syslog-ng follow-freq option.

Since the syslog-ng configuration uses the no-parse flag, in this NXLog configuration, we are using the Exec directive to simply add a $Message field containing the unparsed log record. However, NXLog can easily parse Apache access logs into fields. See our guide on parsing the Common & Combined Log Formats.

Convert syslog-ng destinations and log paths

Now that we sorted our input sources, we’ll see how to save the logs to a file and/or forward them to a remote host over TCP.

 
destination d_messages {
    file("/var/log/messages");
};

destination d_siem {
    syslog(
        "192.168.1.23"
        transport("tcp")
        port(601)
    );
};

log {
    source(s_local);
    destination(d_messages);
    destination(d_siem);
};

log {
    source(s_apache_access);
    destination(d_siem);
};

down arrow

 
<Output output_file>
    Module         om_file
    File           '/var/log/messages'
    Exec           to_syslog_bsd();
</Output>

<Output siem>
    Module         om_tcp
    Host           192.168.1.23:601
    OutputType     Syslog_TLS
    Exec           to_syslog_ietf();
</Output>

<Route r_siem>
    Path           kernel, systemd, devlog, internal, apache_access => siem
</Route>

<Route r_file>
    Path           kernel, systemd, devlog, internal => output_file
</Route>

Use the om_file output module to save logs to a file. For example, in this NXLog configuration, we’re writing the log records in syslog-BSD format by using the to_syslog_bsd() procedure provided by the xm_syslog extension.

To forward logs to a remote destination over TCP, the om_tcp output module will do the job. By default, this module outputs logs one record per line separated by a newline character. In this example, we change the output format to use octet-framing by setting the OutpuType directive to Syslog_TLS. We also convert log records to syslog-IETF messages by calling the to_syslog_ietf() procedure. Both the Syslog_TLS output writer function and the to_syslog_ietf() procedure are provided by the xm_syslog extension.

Finally, all that’s left will be to convert the syslog-ng log paths to NXLog routes. This configuration forwards all logs to the remote host and saves the Linux system and NXLog internal logs to a file.

That was easy, right? Read on to see how to convert your syslog-ng configuration on Windows.

NXLog on Windows

The syslog-ng Agent for Windows comes with an MMC-based configuration interface and supports importing and exporting the configuration to and from an XML file. syslog-ng Agent for Windows is also available as an MSI, so you can deploy and configure it via GPO. However, the configuration options are limited on Windows. According to the syslog-ng documentation:

  • It only supports log forwarding using the Snare, IETF syslog, or BSD syslog protocols.

  • Only basic filtering is available. Log parsing and classification are not possible.

  • The syslog-ng Agent for Windows only reads logs from files, including for Eventlog Containers, and does not implement disk buffering.

NXLog on Windows is configured the same as any other platform, using the configuration file. It is fully featured with no limitations and can be installed interactively or deployed via GPO.

Now, onto some concrete configuration examples.

Convert a syslog-ng Event Container source

Let’s start by converting a Windows Eventlog Source that collects Application, Security, System, and DNS logs.

syslog ng windows eventlog source

down arrow

 
<Input windows_event_log>
    Module             im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Application">*</Select>
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Input dns_server>
    Module             im_etw
    Provider           Microsoft-Windows-DNSServer
    ParsePacketData    TRUE
</Input>

Use the im_msvistalog input module to collect Application, Security, and System logs. This diverse module can collect logs directly from the Event Log API or event log files (.evtx) and automatically parse them into fields. In addition, it supports filtering by Channel or an XPath query. The example above uses the latter. Generating an XPath query is easy! All you need to do is to configure a filter in Windows Event Viewer and copy the corresponding query to the NXLog configuration. See our guide on XPath filtering.

On the other hand, use the im_etw input module to collect DNS Server audit logs. This dedicated module collects logs directly from ETW channels, including Analytic and Debug logs, and automatically parses log records into fields. In this NXLog configuration, we also enable the ParsePacketData directive to convert the DNS payload into JSON format.

Convert a syslog-ng File source

Next, let’s look at how to convert this syslog-ng configuration collecting Domain Controller Promoter (DCPromo) logs.

syslog ng windows file source

down arrow

 
<Input dcpromo_logs>
    Module             im_file
    File               'C:\Windows\debug\DCPROMO.log'
    File               'C:\Windows\debug\DCPROMO.*.log'
    <Exec>
        $ApplicationName = "Domain Controller";
        $EventSource = "DCPROMO";

        if $raw_event =~ /^(\S+ \S+) \[(\S+)\] (.+)$/
        {
            $EventTime = parsedate($1);
            $Severity = $2;
            $Message = $3;
        }
    </Exec>
</Input>

Use the im_file input module to collect file-based logs. You can specify the File directive multiple times and use wildcards.

Optionally, you can include further log processing in an Exec block. For example, in this NXLog configuration, we are adding two new fields, $ApplicationName and $EventSource, and parsing the log record into fields with a regular expression.

Convert a syslog-ng Server destination

Finally, we’ll forward the logs to a remote server over TCP. You simply need to create the output and bind it to the above sources.

syslog ng windows destination

down arrow

 
<Extension syslog>
    Module             xm_syslog
</Extension>

<Output siem>
    Module             om_tcp
    Host               192.168.1.23:601
    FlowControl        TRUE
    OutputType         Syslog_TLS
    Exec               to_syslog_ietf();
</Output>

<Route r1>
    Path               windows_event_log, dns_server, dcpromo_logs => siem
</Route>

Use the om_tcp output module to forward logs to their destination over TCP. The FlowControl directive, enabled by default, corresponds to the Enable flow-control option in syslog-ng.

By default, om_tcp outputs logs one record per line separated by a newline character. In this NXLog configuration, we change the output format to use octet-framing by setting the OutpuType directive to Syslog_TLS. We also convert log records to syslog-IETF messages by calling the to_syslog_ietf() procedure. Both the Syslog_TLS output writer function and the to_syslog_ietf() procedure are provided by the xm_syslog extension.

What’s next?

If you haven’t already, download a free NXLog Enterprise Edition trial and try it for yourself. Once you’ve mastered the basics, converting more complex syslog-ng configuration to NXLog will become second nature.

We recommend the following additional reading to help you in the process:

We hope you found our guide helpful in your quest to find a syslog-ng alternative. Finally, get in touch with us if you need advice or hit any walls. Our support team is always available to share their expertise!

GET STARTED TODAY:
CONTACT US Our experts are happy to help REQUEST A FREE TRIAL Give NXLog Enterprise Edition a try GET PRICING Request a quote

NXLog Ltd. develops multi-platform log collection tools that support many different log sources, formats, transports, and integrations. The tools help administrators collect, parse, and forward logs so they can more easily respond to security issues, investigate operational problems, and analyze event data. NXLog distributes the free and open source NXLog Community Edition and offers additional features and support with the NXLog Enterprise Edition.

This document is provided for informational purposes only and is subject to change without notice. Trademarks are the properties of their respective owners.

Share
Related Posts