responses
We're using NX Log (CE) as a test to see if it will work for our purposes. The overall idea is to use it as a forwarder of syslog flat files to any brand of SIEM.
My config looks like this:
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension syslog>
Module xm_syslog
</Extension>
#<Input Alerts>
# Module im_file
# File 'E:\\DGQradarExports\\ForwarderCust\\Alerts\\*'
# ReadFromLast True
# Exec parse_syslog();
#</Input>
<Input Events>
Module im_file
File 'E:\\DGQradarExports\\ForwarderCust\\Events\\*'
ReadFromLast True
Exec parse_syslog();
</Input>
<Input Process>
Module im_file
File 'E:\\DGQradarExports\\ForwarderCust\\Process\\*'
ReadFromLast True
Exec parse_syslog();
</Input>
<Output Customer>
Module om_tcp
Host 192.168.160.141
Port 514
</Output>
<Route customer_siem>
Path Events,Process => Customer
</Route>
<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Extension _fileop>
Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
My log just shows this:
2018-11-19 08:50:43 INFO nxlog-ce-2.10.2102 started
2018-11-19 08:50:43 INFO connecting to 192.168.160.141:514
QRadar shows an information source has registered but no data ever flows.
I should see a 'connection was successful' message shouldn't I?
Is there any way to up the logging so I can tell if NX Log is even reading the files and attempting to send them? I really can't tell what it's doing currently.
Multiple files exist in the input directories, I'm trying to have NX Log work through all of them, send them to SIEM and then wait for more files. Config examples seem straightforward, I just can't tell what it's doing.
Any help is appreciated.
Comments (5)
I added the logging sections as per the below.
<Input Events>
Module im_file
File 'E:\\DGQradarExports\\ForwarderCust\\Events\\*'
ReadFromLast False
Exec parse_syslog();
Exec log_info("Input Event: " + $raw_event);
</Input>
<Input Process>
Module im_file
File 'E:\\DGQradarExports\\ForwarderCust\\Process\\*'
ReadFromLast False
Exec parse_syslog();
Exec log_info("Input Event: " + $raw_event);
</Input>
<Output Customer>
Module om_udp
Host 192.168.160.141
Port 514
Exec log_info("Input Event: " + $raw_event);
</Output>
Running nxlog.exe in the foreground doesn't show me any more info.
CMD Output:
C:\Program Files (x86)\nxlog>nxlog.exe -f
2018-11-20 04:21:49 INFO nxlog-ce-2.10.2102 started
I can however see all the files being touched by the nxlog.exe process in procmon.
For testing you should also add
SavePos FALSE
or removeconfigcache.dat
manually to make sure the files are read from the beginning every time.Also
Exec parse_syslog();
doesn't make much sense in this scenario if the input file isn't syslog and the parsed fields are not used.I've added SavPos False.
The flat files are syslog, I assumed NX Log needed to Parse them to send them over as Syslog via UDP?
If you aren't modifying the underlying fields, then it isn't important. NXLog will forward as is, and your receiver will see them as Syslog if they are Syslog in your
Input
.When you use
parse_syslog
, it will populate the individual fields from their values in$raw_event
. The manual goes into detail about which fields are created. This is useful if you need to modify a field to change or insert a new value, do calculations based on a field, make decisions on the event based on a field, etc.Using SavPos = False seemed to shunt data over. I can only assume this is because connectivity was finally right and it reprocessed the files.
I'm leaving it running for 24hrs to ensure it works with fresh flat files that are added and that my 'cycling' scripts work for the old stuff.
I've installed the enterprise trial for now so I can test leef also.