jd01 created
Hi,
I'm currently using nxlog to forward RADIUS messages via syslog to my firewall. However, it has recently started complaining that the packets are too big, and so fragmentation is occurring which it doesn't like.
The temporary fix was to force the packets to cut at 1450 bytes, and this is my current config:
Panic Soft #NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
<Extension _xml> Module xm_xml </Extension>
<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>
<Extension _exec> Module xm_exec </Extension>
<Extension _fileop> Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
<Input NPS>
Module im_file
File "C:\Windows\System32\LogFiles\IN*.log"
InputType LineBased
SavePos TRUE
ReadFromLast TRUE
<Exec>
# Discard everything that doesn't seem to be an xml event
if $raw_event !~ /^<Event>/ drop();
# Filter to only events containing all required data (type, username and ip)
if $raw_event !~ /(Type\sdata_type="0">)(\d{1,2})(<\/Acct)(.+)(Name\sdata_type="1">)([a-zA-Z0-9\$\._-]{3,15})(.*)(<\/User)(.+)(Address\sdata_type="3">)([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})(<\/Framed)/ drop ();
# Truncates event to 1400 bytes due to MTU limits
$raw_event = substr($raw_event, 0, 1450);
# Reduces event string to just required data (type, username and ip)
# Parse xml
parse_xml();
</Exec>
</Input>
<Output Firewall> Module om_udp # Put your Firewal Management interface IP address # Don't change port or protocol (should be UDP 514 or TCP 6514) Host 192.168.1.1 Port 514 </Output>
<Output SyslogServer> # Put your Syslog Server IP address and port # Allows monitoring of messages being sent to firewall Module om_udp Host 192.168.1.10 Port 514 </Output>
<Route 1> Path NPS => Firewall </Route>
<Route 2> Path NPS => SyslogServer </Route>
However, I'd prefer a neater solution, rather than just chopping the end off the packet. The only parts of the packet I'm interested in forwarding are:
Event Regex: <Acct-Status-Type\sdata_type="0">1</Acct-Status-Type>{1} Username Regex: <User-Name\sdata_type="1">([a-zA-Z0-9\._-]+)</User-Name> Address Regex: <Framed-IP-Address\sdata_type="3">([0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3})</Framed-IP-Address>
Is there a way to extract just those bits and parse that to the output?
Apologies if it's obvious, but I don't really understand how nxlog works! Give me powershell and I'm happy....
Thanks, Stephen
SP_895311 created
jd01 created
We're using NX Log (CE) as a test to see if it will work for our purposes. The overall idea is to use it as a forwarder of syslog flat files to any brand of SIEM.
My config looks like this:
Panic Soft #NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
<Extension syslog> Module xm_syslog </Extension>
#<Input Alerts>
Module im_file
File 'E:\DGQradarExports\ForwarderCust\Alerts\*'
ReadFromLast True
Exec parse_syslog();
#</Input>
<Input Events> Module im_file File 'E:\DGQradarExports\ForwarderCust\Events\*' ReadFromLast True Exec parse_syslog(); </Input>
<Input Process> Module im_file File 'E:\DGQradarExports\ForwarderCust\Process\*' ReadFromLast True Exec parse_syslog(); </Input>
<Output Customer> Module om_tcp Host 192.168.160.141 Port 514 </Output>
<Route customer_siem> Path Events,Process => Customer </Route>
<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>
<Extension _exec> Module xm_exec </Extension>
<Extension _fileop> Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
My log just shows this: 2018-11-19 08:50:43 INFO nxlog-ce-2.10.2102 started 2018-11-19 08:50:43 INFO connecting to 192.168.160.141:514
QRadar shows an information source has registered but no data ever flows.
I should see a 'connection was successful' message shouldn't I? Is there any way to up the logging so I can tell if NX Log is even reading the files and attempting to send them? I really can't tell what it's doing currently.
Multiple files exist in the input directories, I'm trying to have NX Log work through all of them, send them to SIEM and then wait for more files. Config examples seem straightforward, I just can't tell what it's doing.
Any help is appreciated.
nhart created
c.scharfenberg created
wiht the community eddition when the nxlog-ce is listening on /dev/log and for some reasons the systemd-journald (debian 9) removes the socket the nxlog blocks the compleate host. (even no login possible) after a restart of nxlog the host recovers.
Mario.Fetka created
Here is a sample event when using to_syslog_snare() in the nxlog.conf:
<14>Jan 27 10:03:39 event_computer MSWinEventLog 1 Security 32630749 Wed Jan 27 10:03:39 2016 4624 Microsoft-Windows-Security-Auditing N/A N/A Success Audit event_computer Logon An account was successfully logged on. Subject: Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 Impersonation Level: Impersonation New Logon: Security ID: S-1-5-21-2705889813-1605608894-1661845433-43745 Account Name: account_name Account Domain: account_domain Logon ID: 0x23820B882 Logon GUID: {00000000-0000-0000-0000-000000000000} Process Information: Process ID: 0x0 Process Name: - Network Information: Workstation Name: workstation_name Source Network Address: source_address Source Port: 54241 Detailed Authentication Information: Logon Process: NtLmSsp Authentication Package: NTLM Transited Services: - Package Name (NTLM only): NTLM V2 Key Length: 0 This event is generated when a logon session is created. It is generated on the computer that was accessed. The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network). The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on. The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The impersonation level field indicates the extent to which a process in the logon session can impersonate. The authentication information fields provide detailed information about this specific logon request. - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested. 35284558
My issue is that I would NOT want to collect the "informational text" representing the event - in this case everything starting from the string "This event is generated---" all the way up until "--was requested."
Before I go any deeper into this, let me state that in the logs of this format I call the "<14>Jan 27 10:03:39 event_computer MSWinEventLog 1 Security 32630749 Wed Jan 27 10:03:39 2016 4624 Microsoft-Windows-Security-Auditing N/A N/A Success Audit event_computer Logon" portion of the whole log message the HEADER, and the rest is called MESSAGE.
Putting it another way, I would like to forward the message using syslog in a format constructed according to the pseudocode below:
parse fields from windows event /* e.g. SubjectUserName, LogonType, IpAddress, etc. */
/* print the header "as is" already in the to_syslog_snare() format, i.e. from "<14>---" until and including "---Logon"
print HEADER /* e.g. event_time,event_computer,event_type,event_id,... */
for all fields parsed
print "'field_name=field_value'" /* e.g. SubjectUserName=value,LogonType=value,IpAddress=value,... /*
The reason I would like to do this is that the informational text, which gets appended to some Windows events (not all, it seems), takes a lot of space, and we do not really need this information text for anything.
Another way to do this would be to statically list all the fields POSSIBLY found in an Windows event and construct the message that way, but this would often leave me with a lot of empty key-value pairs. THUS I would only like to print out those fields that were found in that specific log message while leaving out the informational message.
I do acknowledge, though, that especially Application and System events might not contain most or any of the fields that are present in a Security log event. Take for example the following System log event:
<14>Jan 27 11:09:21 event_computer MSWinEventLog 1 System 32633951 Wed Jan 27 11:09:21 2016 7036 Service Control Manager N/A N/A Information event_computer N/A The Remote Registry service entered the stopped state. 319889
In the example above, the "header" portion of the whole message only contains the string "The Remote Registry service entered the stopped state." I do hope, though, that the variable where this string is stored is actually the same that hosted the string "An account was successfully logged on.", which would mean that my approach in the pseudocode would still work (i.e. the array or list of fields that is iterated and printed would only contain one field. The HEADER portion of the field is exactly the same in all messages.
The description of to_syslog_snare() in the nxlog documentation states:
"Create a SNARE Syslog formatted log message in $raw_event. Uses the following fields to construct $raw_event: $EventTime, $Hostname, $SeverityValue, $FileName, $EventID, $SourceName, $AccountName, $AccountType, $EventType, $Category, $Message."
Thus when reflecting back to what I said, it seems that what I call the HEADER includes all the fields from $EventTime to (and including) $Category - this I would like to keep as it is. But according to the documentation, the $Message variable actually then holds all the other information in the log, or what I call the MESSAGE portion. So I guess the question is that can the contents of the $Message variable be further filtered, as it obviously is constructed from e.g. EventData's Data fields listed below. I would like to only change the $Message contents so that it would never contain the informational text if there exists such a message in a given log message, and that preferably the Data fields inside $Message would be formatted using key-value pairs instead of the to_syslog_snare format seen in the first example (one or more whitespace as delimiter).
tsigidibam created
