syslog-ng and NXLog Agent are both established log collectors that provide flexible data processing. However, you might need to switch from syslog-ng to NXLog Agent. Whether it’s because syslog-ng doesn’t support an operating system or you want to upgrade your log collection solution to one that can be centrally managed, converting your syslog-ng configuration to NXLog Agent is simple.

How do syslog-ng and NXLog Agent differ?

syslog-ng and NXLog Agent are alike in many ways. They can both parse, process, normalize, and forward logs over the network via secure transport. They also implement caching mechanisms for reliable message delivery and their own protocol for more efficient log forwarding between nodes. However, NXLog Agent supports more input sources in more formats and ships to more output destinations.

Some of the main differences include:

  • Unlike syslog-ng, NXLog Agent can be centrally managed. It includes a remote management module providing RESTful and SOAP APIs, so you can easily integrate it with whatever SCM tool you already use. In addition, NXLog Platform provides a single console to configure, deploy, and monitor agents.

  • NXLog Agent supports custom extensions in Go, Perl, Python, Ruby, and Java. syslog-ng’s current versions (4.x) support only Python for custom extensions.

  • syslog-ng’s hdfs() destination requires a Java runtime on the host. NXLog Agent has no such requirement.

  • syslog-ng supports basic macOS log collection. NXLog Agent offers broader and deeper macOS coverage, with a full agent installation and specialized modules for collecting macOS system and security logs, including Endpoint Security.

What can NXLog Agent do that syslog-ng can’t?

Here is a non-exhaustive list of input/output sources and log formats NXLog Agent supports out of the box that are not available in syslog-ng.

Input Output Log format

Generic HTTP(s) input

Microsoft Azure - tables and blobs (in addition to Log Analytics workspaces, also supported by syslog-ng)

GELF

Google Cloud Logging

Microsoft 365 - Microsoft Graph and Office 365 Reporting web service (in addition to the Office 365 Management API, also supported by syslog-ng)

NetFlow

macOS - Apple’s unified logging system (ULS) and Endpoint Security auditing system

Google Security Operations

Microsoft DNS Server

Microsoft Azure - tables, blobs, and Log Analytics workspaces

Prometheus - built-in metrics endpoint for scraping (syslog-ng requires an external exporter)

SAP audit logs

Network packet capture

Redis

W3C

Redis

ZeroMQ

Salesforce

Okta

File integrity monitoring

Windows Registry monitoring

ZeroMQ

Check Point OPSEC LEA

syslog-ng and NXLog Agent configuration comparison

syslog-ng vs. NXLog Agent nomenclature

Before going any further, you should familiarize yourself with how both products refer to configuration elements. We cover the basics in the following matrix.

Configuration element syslog-ng NXLog Agent

Log ingestion element

source/source driver

input module

Log output element

destination/destination driver

output module

Log routing element

log path

route

Log filtering element

filter block

xm_pattern or via regular expressions

Log parsing element

parser block

extension modules or via regular expressions

Log transformation element

rewrite rule

xm_rewrite and xm_transform modules or Exec statements

Configuration parameters

options

directives

Values parsed from log records

macros

fields

NXLog Agent on Linux

Both syslog-ng and NXLog Agent use a file-based configuration system on Linux platforms. The default configuration file location for the two products is:

syslog-ng

/opt/syslog-ng/etc/syslog-ng.conf

NXLog Agent

/opt/nxlog/etc/nxlog.conf

On most Linux distributions, only one syslog daemon should be active at a time. When installing syslog-ng via the system package manager, the package manager may automatically remove rsyslog due to a package conflict. NXLog Agent doesn’t make changes to other components on the system. If rsyslog is still active after installing NXLog Agent, you can deactivate it manually by running the following commands:

$ sudo systemctl disable --now rsyslog
$ sudo systemctl force-reload systemd-journald
$ sudo systemctl restart nxlog

Let’s get into the configuration.

Convert syslog-ng internal and Linux system log sources

The default syslog-ng configuration on Linux collects system and kernel logs, as well as syslog-ng’s internal logs. So let’s see how to replicate these sources in NXLog Agent.

 
source s_local {
    internal();
    system();
};

down arrow

 
<Extension syslog>
    Module         xm_syslog
</Extension>

<Input kernel>
    Module         im_kernel
    Exec           parse_syslog_bsd();
</Input>

<Input systemd>
    Module         im_systemd
</Input>

<Input devlog>
    Module         im_uds
    UDS            /dev/log
    FlowControl    FALSE
    Exec           parse_syslog_bsd();
</Input>

<Input internal>
    Module         im_internal
</Input>

Start by choosing which Linux logs you want to collect:

  • Use the Kernel input module to collect logs from the kernel log buffer.

  • Read daemon messages from the systemd journal socket with the Systemd input module.

  • Collect other user-space messages from the /dev/log socket with the Unix Domain Sockets input module.

  • Set up auditing rules and collect logs directly from the kernel with the Linux Audit System input module

In this NXLog Agent configuration, we collect the first three logs corresponding to syslog-ng’s system() source. Since the kernel and user-space messages are in syslog-BSD format, we’re using the parse_syslog_bsd() procedure provided by the Syslog extension to parse log records into fields.

NXLog Agent can also collect its own internal logs with the Internal input module.

Convert a syslog-ng file source

Log files are still common, especially on Linux. So let’s convert a syslog-ng file source collecting Apache access logs to NXLog Agent.

 
source s_apache_access {
    file(
        "/var/log/apache/access.log"
        follow-freq(1)
        flags(no-parse)
    );
};

down arrow

 
<Input apache_access>
    Module          im_file
    File            '/var/log/apache/access.log'
    PollInterval    1
    Exec            $Message = $raw_event;
</Input>

Use the File input module to collect file-based logs. You can specify the File directive multiple times and use wildcards. The PollInterval directive, set to 1 second by default, corresponds to the syslog-ng follow-freq option.

Since the syslog-ng configuration uses the no-parse flag, in this NXLog Agent configuration, we are using the Exec directive to add a $Message field containing the unparsed log record. However, NXLog Agent can easily parse Apache access logs into fields. See our guide on parsing the Common & Combined Log Formats.

Convert syslog-ng destinations and log paths

Now that we’ve sorted our input sources, let’s move on to saving the logs to a file and/or forwarding them to a remote host over TCP.

 
destination d_messages {
    file("/var/log/messages");
};

destination d_siem {
    syslog(
        "192.168.1.23"
        transport("tcp")
        port(601)
    );
};

log {
    source(s_local);
    destination(d_messages);
    destination(d_siem);
};

log {
    source(s_apache_access);
    destination(d_siem);
};

down arrow

 
<Output output_file>
    Module         om_file
    File           '/var/log/messages'
    Exec           to_syslog_bsd();
</Output>

<Output siem>
    Module         om_tcp
    Host           192.168.1.23:601
    OutputType     Syslog_TLS
    Exec           to_syslog_ietf();
</Output>

<Route r_siem>
    Path           kernel, systemd, devlog, internal, apache_access => siem
</Route>

<Route r_file>
    Path           kernel, systemd, devlog, internal => output_file
</Route>

Use the File output module to save logs to a file. For example, in this NXLog Agent configuration, we’re writing the log records in syslog-BSD format by using the to_syslog_bsd() procedure provided by the Syslog extension.

To forward logs to a remote destination, the TCP output module does the job. By default, this module outputs logs one record per line separated by a newline character. In this example, we change the output format to use octet-framing by setting the OutputType directive to Syslog_TLS. We also convert records to syslog-IETF messages by calling the to_syslog_ietf() procedure. Both the Syslog_TLS output writer function and the to_syslog_ietf() procedure are provided by the Syslog extension.

Finally, all that’s left is to convert the syslog-ng log paths to NXLog Agent routes. This configuration forwards all logs to the remote host and saves the Linux system and NXLog Agent internal logs to a file.

That’s the Linux side covered. Read on to see how to convert your syslog-ng configuration on Windows.

NXLog Agent on Windows

The syslog-ng Agent for Windows comes with an MMC-based configuration interface and supports importing and exporting the configuration to and from an XML file. syslog-ng Agent for Windows is also available as an MSI, so you can deploy and configure it via GPO. However, the configuration options are limited on Windows. According to the syslog-ng documentation:

  • It only supports log forwarding using the Snare, IETF syslog, or BSD syslog protocols.

  • Only basic filtering is available. Log parsing and classification are not possible.

  • The syslog-ng Agent for Windows only reads logs from files, including for Eventlog Containers, and does not implement disk buffering.

You configure NXLog Agent on Windows the same as any other platform, using the configuration file. NXLog Agent on Windows has none of these limitations and can be installed interactively or deployed via GPO.

Now, onto some concrete configuration examples.

Convert a syslog-ng Event Container source

Let’s start by converting a Windows Eventlog Source that collects Application, Security, System, and DNS logs.

syslog ng windows eventlog source

down arrow

 
<Input windows_event_log>
    Module             im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Application">*</Select>
                <Select Path="Security">*</Select>
                <Select Path="System">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Input dns_server>
    Module             im_etw
    Provider           Microsoft-Windows-DNSServer
    ParsePacketData    TRUE
</Input>

Use the Event Log for Windows input module to collect Application, Security, and System logs. This module can collect logs directly from the Event Log API or event log files (.evtx) and automatically parse them into fields. In addition, it supports filtering by Channel or an XPath query. The example above uses the latter. To generate an XPath query, configure a filter in Windows Event Viewer and copy the corresponding query to the NXLog Agent configuration. See our guide on XPath filtering.

On the other hand, use the Event Tracing for Windows input module to collect DNS Server audit logs. This dedicated module collects logs directly from ETW channels, including Analytic and Debug logs, and automatically parses log records into fields. In this NXLog Agent configuration, we also activate the ParsePacketData directive to convert the DNS payload into JSON format.

Convert a syslog-ng File source

Next, let’s look at how to convert this syslog-ng configuration to collect Domain Controller Promoter (DCPromo) logs.

syslog ng windows file source

down arrow

 
<Input dcpromo_logs>
    Module             im_file
    File               'C:\Windows\debug\DCPROMO.log'
    File               'C:\Windows\debug\DCPROMO.*.log'
    <Exec>
        $ApplicationName = "Domain Controller";
        $EventSource = "DCPROMO";

        if $raw_event =~ /^(\S+ \S+) \[(\S+)\] (.+)$/
        {
            $EventTime = parsedate($1);
            $Severity = $2;
            $Message = $3;
        }
    </Exec>
</Input>

Use the File input module to collect file-based logs. You can specify the File directive multiple times and use wildcards.

Optionally, you can include further log processing in an Exec block. For example, in this NXLog Agent configuration, we add two new fields, $ApplicationName and $EventSource, and parse the record into fields with a regular expression.

Convert a syslog-ng Server destination

Finally, we’ll forward the logs to a remote server over TCP. You simply need to create the output and bind it to the above sources.

syslog ng windows destination

down arrow

 
<Extension syslog>
    Module             xm_syslog
</Extension>

<Output siem>
    Module             om_tcp
    Host               192.168.1.23:601
    FlowControl        TRUE
    OutputType         Syslog_TLS
    Exec               to_syslog_ietf();
</Output>

<Route r1>
    Path               windows_event_log, dns_server, dcpromo_logs => siem
</Route>

Use the TCP output module to forward logs to their destination over TCP. The FlowControl directive, activated by default, corresponds to the Enable flow-control option in syslog-ng.

By default, the TCP output module outputs logs one record per line separated by a newline character. In this NXLog Agent configuration, we change the output format to use octet-framing by setting the OutputType directive to Syslog_TLS. We also convert log records to syslog-IETF messages by calling the to_syslog_ietf() procedure. Both the Syslog_TLS output writer function and the to_syslog_ietf() procedure are provided by the Syslog extension.

What’s next?

If you haven’t already, sign up for a free NXLog Platform account and try it for yourself. Once you’ve mastered the basics, converting more complex syslog-ng configurations to NXLog Agent becomes second nature.

We recommend the following additional reading to help you in the process:

We hope you found our guide helpful if you’re evaluating syslog-ng alternatives. Finally, get in touch with us if you need advice or hit any walls. Our support team is happy to help.

NXLog Platform is an on-premises solution for centralized log management with
versatile processing forming the backbone of security monitoring.

With our industry-leading expertise in log collection and agent management, we comprehensively
address your security log-related tasks, including collection, parsing, processing, enrichment, storage, management, and analytics.

Start free Contact us
Share
Related Posts