How Do I Send IIS v8.5 Logs in W3C Format to a Linux Syslog Server?


#1 bk

I would like to send IIS v8.5 logs over to a Linux syslog server. I have all parts installed, but need help with the nxlog agent configuration on the IIS server (Win2012 R2). My current configuration is attached.

There are errors in the Win2012 nxlog agent's log file, and I am unable to fix them all, they are attached as well. I had to comment out parts like writing to a local file in order to get the agent running. I would like to have that work as well. It created the file but it's empty.

Config (errors are further below):

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
	Module	xm_syslog
</Extension>

<Extension fileop>#
	Module xm_fileop
</Extension>

<Extension w3c_parser>
    Module          xm_csv
    Fields          date, time, s-ip, cs-method, cs-uri-stem, cs-uri-query, \
                    s-port, cs-username, c-ip, cs(User-Agent), cs(Referer), \
                    sc-status, sc-substatus, sc-win32-status, time-taken
    FieldTypes      string, string, string, string, string, string, integer, \
                    string, string, string, string, integer, integer, integer, \
                    integer
    Delimiter       ' '
    EscapeChar      '"'
    QuoteChar       '"'
    EscapeControl   FALSE
    UndefValue      -
</Extension>

<Input iis_w3c>
    Module  im_file
    File  "C:\\inetpub\\logs\\LogFiles\\W3SVC1\\\\*.log"
    <Exec>
        if $raw_event =~ /^#/ drop();
        else
        {
            w3c_parser->parse_csv();
            $EventTime = parsedate($date + "T" + $time + ".000Z");
        }
    </Exec>
</Input>

#<Output out_file_iis>
#    Module om_tcp
#    File 'C:\outputiis.log'
#    Exec to_syslog_bsd();
#</Output>

<Output out_tcp>
    Module om_tcp
    Host 10.0.3.163
    Port 514
    Exec to_syslog_bsd();
</Output>

<Route send_iis_to_syslog_server>
    Path iis_w3c => out_tcp
</Route>

<Route iis> 
    Path iis_w3c => out_file_iis
</Route>

<Extension json>
	Module	xm_json
</Extension>

<Extension charconv>
    Module      xm_charconv
    AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2, ucs-2le
</Extension>


##########################################
##          NXLOG INTERNAL LOG          ##
##########################################
# Nxlog internal logs - Recommended to keep this turned ON so error(s)/Issues with NXLog are reported.
<Input internal>
   Module im_internal
   Exec $Hostname = hostname_fqdn();
   Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>



##########################################
##             FLAT FILES               ##
##########################################
## http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#im_file
## Input to watch a file of your choosing. After Input, name it whatever you want to describe that NXLog
## is pulling, then add that name to the path in Route 1 after eventlog. Can be separated for filtering diff logs.
## After setting the Message as raw_event this converts the message to UTF-8, drops empty messages
## removes extra whitespace, grabs the file name as LogFile, adds the FQDN, and deletes a useless var


## Pulls all logfiles from the default ITS Log Location
## It is Recommended to LEAVE THIS ENABLED
## Ensure that "ITS_Logs" is specified in the correct Route at the bottom for output
<Input ITS_Logs>
   Module   im_file
   File     "C:\\ITS\\Logs\\\\*.log"
   SavePos  TRUE
   Recursive TRUE
   Exec $Message = $raw_event;
   Exec $Message = convert($Message, "ucs-2le", "utf-8");
   Exec if $Message == '' drop();
   Exec if $Message =~ s/^\s+//g log_debug("whitespace removed");
   Exec if file_name() =~ /([^\\]+)$/ $LogFile = $1;
   Exec $Hostname = hostname_fqdn();
   Exec delete($SourceModuleType);
   Exec $EventTime = $EventReceivedTime;
   Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;
   Exec  to_json(); 
</Input>



Error Message:
2019-09-11 12:08:56 ERROR if-else failed at line 46, character 9 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 36 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer "/", invalid modifier: '/'
2019-09-11 12:09:26 ERROR last message repeated 5 times
#2 Zhengshi Nxlog ✓
#1 bk
I would like to send IIS v8.5 logs over to a Linux syslog server. I have all parts installed, but need help with the nxlog agent configuration on the IIS server (Win2012 R2). My current configuration is attached. There are errors in the Win2012 nxlog agent's log file, and I am unable to fix them all, they are attached as well. I had to comment out parts like writing to a local file in order to get the agent running. I would like to have that work as well. It created the file but it's empty. Config (errors are further below): ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog> Module xm_syslog </Extension> <Extension fileop># Module xm_fileop </Extension> <Extension w3c_parser> Module xm_csv Fields date, time, s-ip, cs-method, cs-uri-stem, cs-uri-query, \ s-port, cs-username, c-ip, cs(User-Agent), cs(Referer), \ sc-status, sc-substatus, sc-win32-status, time-taken FieldTypes string, string, string, string, string, string, integer, \ string, string, string, string, integer, integer, integer, \ integer Delimiter ' ' EscapeChar '"' QuoteChar '"' EscapeControl FALSE UndefValue - </Extension> <Input iis_w3c> Module im_file File "C:\\inetpub\\logs\\LogFiles\\W3SVC1\\\\*.log" <Exec> if $raw_event =~ /^#/ drop(); else { w3c_parser->parse_csv(); $EventTime = parsedate($date + "T" + $time + ".000Z"); } </Exec> </Input> #<Output out_file_iis> # Module om_tcp # File 'C:\outputiis.log' # Exec to_syslog_bsd(); #</Output> <Output out_tcp> Module om_tcp Host 10.0.3.163 Port 514 Exec to_syslog_bsd(); </Output> <Route send_iis_to_syslog_server> Path iis_w3c => out_tcp </Route> <Route iis> Path iis_w3c => out_file_iis </Route> <Extension json> Module xm_json </Extension> <Extension charconv> Module xm_charconv AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2, ucs-2le </Extension> ########################################## ## NXLOG INTERNAL LOG ## ########################################## # Nxlog internal logs - Recommended to keep this turned ON so error(s)/Issues with NXLog are reported. <Input internal> Module im_internal Exec $Hostname = hostname_fqdn(); Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Input> ########################################## ## FLAT FILES ## ########################################## ## http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#im_file ## Input to watch a file of your choosing. After Input, name it whatever you want to describe that NXLog ## is pulling, then add that name to the path in Route 1 after eventlog. Can be separated for filtering diff logs. ## After setting the Message as raw_event this converts the message to UTF-8, drops empty messages ## removes extra whitespace, grabs the file name as LogFile, adds the FQDN, and deletes a useless var ## Pulls all logfiles from the default ITS Log Location ## It is Recommended to LEAVE THIS ENABLED ## Ensure that "ITS_Logs" is specified in the correct Route at the bottom for output <Input ITS_Logs> Module im_file File "C:\\ITS\\Logs\\\\*.log" SavePos TRUE Recursive TRUE Exec $Message = $raw_event; Exec $Message = convert($Message, "ucs-2le", "utf-8"); Exec if $Message == '' drop(); Exec if $Message =~ s/^\s+//g log_debug("whitespace removed"); Exec if file_name() =~ /([^\\]+)$/ $LogFile = $1; Exec $Hostname = hostname_fqdn(); Exec delete($SourceModuleType); Exec $EventTime = $EventReceivedTime; Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; Exec to_json(); </Input> Error Message: 2019-09-11 12:08:56 ERROR if-else failed at line 46, character 9 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 36 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer "/", invalid modifier: '/' 2019-09-11 12:09:26 ERROR last message repeated 5 times
2019-09-11 12:08:56 ERROR if-else failed at line 46, character 9 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 36 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer "/", invalid modifier: '/'

This error message seems to indicate that one of the fields is expected to be an integer, but is seen as / instead.
In our example in the manual (which looks like you are using), we are expecting the default fields. If the fields have changed or been modified in the IIS Manager, you will need to update the CSV fields appropriately. You can see the fields if you open the log file.

Below is an example log file from one of my test servers. You will notice that I have changed the fields at some point, so my #Fields line was updated.
These are the fields you have to match in the xm_csv Fields/FieldTypes sections. Again, make sure it matches the fields you see in the file on your system.

#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2019-08-30 22:52:31
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2019-08-30 22:52:31 127.0.0.1 GET / - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko - 200 0 0 93
2019-08-30 22:52:31 127.0.0.1 GET /iisstart.png - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko http://127.0.0.1/ 200 0 0 0
2019-08-30 22:52:31 127.0.0.1 GET /favicon.ico - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko - 404 0 2 15
#Software: Microsoft Internet Information Services 10.0
#Version: 1.0
#Date: 2019-08-30 22:55:44
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken
2019-08-30 22:55:43 W3SVC1 WIN-LU43V8BOQ6J 127.0.0.1 GET /favicon.ico - 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko - 404 0 2 0