Basic Configuration from syslog flat file to IBM QRadar - Connectivity seems to work, can't tell if I'm sending data.

Tags: siem | syslog | QRadar

#1 nhart

We're using NX Log (CE) as a test to see if it will work for our purposes. The overall idea is to use it as a forwarder of syslog flat files to any brand of SIEM.

My config looks like this:

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension syslog> Module xm_syslog </Extension>

#<Input Alerts>

Module im_file

File 'E:\DGQradarExports\ForwarderCust\Alerts\*'

ReadFromLast True

Exec parse_syslog();

#</Input>

<Input Events> Module im_file File 'E:\DGQradarExports\ForwarderCust\Events\*' ReadFromLast True Exec parse_syslog(); </Input>

<Input Process> Module im_file File 'E:\DGQradarExports\ForwarderCust\Process\*' ReadFromLast True Exec parse_syslog(); </Input>

<Output Customer> Module om_tcp Host 192.168.160.141 Port 514 </Output>

<Route customer_siem> Path Events,Process => Customer </Route>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') &gt;= 5M)) \
                file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>

My log just shows this: 2018-11-19 08:50:43 INFO nxlog-ce-2.10.2102 started 2018-11-19 08:50:43 INFO connecting to 192.168.160.141:514

QRadar shows an information source has registered but no data ever flows.

I should see a 'connection was successful' message shouldn't I? Is there any way to up the logging so I can tell if NX Log is even reading the files and attempting to send them? I really can't tell what it's doing currently.

Multiple files exist in the input directories, I'm trying to have NX Log work through all of them, send them to SIEM and then wait for more files. Config examples seem straightforward, I just can't tell what it's doing.

Any help is appreciated.

#2 Zhengshi Nxlog ✓
#1 nhart
We're using NX Log (CE) as a test to see if it will work for our purposes. The overall idea is to use it as a forwarder of syslog flat files to any brand of SIEM. My config looks like this: Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension syslog> Module xm_syslog </Extension> #<Input Alerts> Module im_file File 'E:\DGQradarExports\ForwarderCust\Alerts\*' ReadFromLast True Exec parse_syslog(); #</Input> <Input Events> Module im_file File 'E:\DGQradarExports\ForwarderCust\Events\*' ReadFromLast True Exec parse_syslog(); </Input> <Input Process> Module im_file File 'E:\DGQradarExports\ForwarderCust\Process\*' ReadFromLast True Exec parse_syslog(); </Input> <Output Customer> Module om_tcp Host 192.168.160.141 Port 514 </Output> <Route customer_siem> Path Events,Process => Customer </Route> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> My log just shows this: 2018-11-19 08:50:43 INFO nxlog-ce-2.10.2102 started 2018-11-19 08:50:43 INFO connecting to 192.168.160.141:514 QRadar shows an information source has registered but no data ever flows. I should see a 'connection was successful' message shouldn't I? Is there any way to up the logging so I can tell if NX Log is even reading the files and attempting to send them? I really can't tell what it's doing currently. Multiple files exist in the input directories, I'm trying to have NX Log work through all of them, send them to SIEM and then wait for more files. Config examples seem straightforward, I just can't tell what it's doing. Any help is appreciated.

One tip I love for working out config issues in foreground mode and using log_info() to show me events in real time.
nxlog -f to run in foreground mode.

Something like the following to add log_info() to the modules.

<Input Process>
    Module im_file
    File 'E:\\DGQradarExports\\ForwarderCust\\Process\\*'
    ReadFromLast True
    Exec parse_syslog();
    Exec log_info("Input Event: " + $raw_event);
</Input>

<Output Customer>
    Module om_tcp
    Host 192.168.160.141
    Port 514
    Exec log_info("Output Event: " + $raw_event);
</Output>

Once you see if NXLog is processing events how you would like, you could use something like wireshark or tcpdump to make sure the events are making it to your end server. Port not open all the way is a common issue we see.