0
responses

I'm using NXLog to sending Windows events and IIS logs to Loggly. We've recently onboarded a new MSSP and they have asked us to check off all IIS logging fields. This seems to break parsing of IIS logs that need to be sent to Loggly. I've contact Loggly support and they can't seem to come to a resolution.

Below is the code that we had been using for Loggly previously.

## This is a sample NXLog configuration file created by Loggly. June 2013
## See the nxlog reference manual about the configuration options.
## It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\\Program Files\\nxlog
#define ROOT_STRING C:\\Program Files\\nxlog
define ROOT C:\\Program Files (x86)\\nxlog
define ROOT_STRING C:\\Program Files (x86)\\nxlog
define CERTDIR %ROOT%\\cert
define LOGFILE %ROOT%\data\nxlog.log

Moduledir %ROOT%\\modules
CacheDir %ROOT%\\data
Pidfile %ROOT%\\data\\nxlog.pid
SpoolDir %ROOT%\\data
LogFile %ROOT%\\data\\nxlog.log

# Include fileop while debugging, also enable in the output module below
<Extension fileop>
Module xm_fileop
</Extension>

<Extension json>
Module xm_json
</Extension>

<Extension syslog>
Module xm_syslog
</Extension>

<Input internal>
Module im_internal
Exec $Message = to_json();
</Input>

# Windows Event Log
<Input eventlog>
#Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog

Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>

#Uncomment im_mseventlog for Windows XP/2000/2003
#Module im_mseventlog

Exec $Message = to_json();
</Input>

<Processor buffer>
Module pm_buffer
# 100Mb disk buffer
MaxSize 102400
Type disk
</Processor>

<Processor buffer_iis>
Module pm_buffer
# 100Mb disk buffer
MaxSize 102400
Type disk
</Processor>

<Output out>
Module om_tcp
Host logs-01.loggly.com
Port 514

Exec to_syslog_ietf();
Exec $raw_event =~ s/(\[.*] )//g; $raw_event = replace($raw_event, '{', '[CUSTOMER ID tag="windows"] {', 1);

#Use the following line for debugging (uncomment the fileop extension above as well)
#Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Output _nxlog>
Module om_file
File '%LOGFILE%'

<Schedule>
When @hourly
Exec if (file_size('%LOGFILE%') >= 1M) { file_cycle('%LOGFILE%', 5); _nxlog->reopen(); }
</Schedule>
</Output>

<Route 1>
Path internal, eventlog => buffer => out
</Route>

<Route 2>
Path internal => _nxlog
</Route>

# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
Module xm_csv
Fields $date, $time, $s-computername, $cs-method, $cs-uri-stem, $cs-uri-query, $c-ip, $cs(User-Agent), $cs(Referer), $cs-host, $sc-status, $sc-substatus, $sc-bytes, $cs-bytes, $time-taken, X-Forwarded-For, RequestId, PrincipalId
FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, integer, integer, integer, string, string, string
Delimiter ' '
QuoteChar '"'
EscapeControl FALSE
UndefValue -
</Extension>

# Convert the IIS logs to JSON and use the original event time
<Input SC>
Module im_file
File "C:\\inetpub\\logs\\LogFiles\\W3SVC1\\u_ex*"
SavePos TRUE

Exec if $raw_event =~ /^#/ drop(); \
else \
{ \
w3c->parse_csv(); \
$SourceName = "IIS"; \
$Message = to_json(); \
}
</Input>
<Input SC_WebAPI>
Module im_file
File "C:\\inetpub\\logs\\LogFiles\\W3SVC2\\u_ex*"
SavePos TRUE

Exec if $raw_event =~ /^#/ drop(); \
else \
{ \
w3c->parse_csv(); \
$SourceName = "IIS"; \
$Message = to_json(); \
}
</Input>

<Route IIS>
Path SC,SC_WebAPI => buffer_iis => out
</Route>

The error we received in data.log looks like below:

2021-02-18 14:11:12 "SERVERNAME" ERROR if-else failed at line 144, character 261 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted
procedure 'parse_csv' failed at line 144, character 156 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted
couldn't parse integer: ELB-HealthChecker/2.0
2021-02-18 14:11:26 "SERVERNAME" ERROR if-else failed at line 131, character 261 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted
procedure 'parse_csv' failed at line 131, character 156 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted
couldn't parse integer: ELB-HealthChecker/2.0

It seems as if the cs-useragent is being evaluated as an integer. or possible the input has fewer fields than expected.

Any help would be appreciated.

AskedFebruary 18, 2021 - 4:20pm

Answers (0)