Problems sending Microsoft IIS logs to Graylog

Tags:

#1 bercom

Hi, I am doing a test with the agent nxlog EE v.5.3.6735_windows_x64 and after the installation of the agent on the server I stopped receiving the windows audit logs (im_msvistalog module) that I was receiving regularly with the version of nxlog CE v.2.10.2150 and I am not getting the IIS logs either. I attach the implemented configuration file:

Panic Soft

define INSTALLDIR C:\Program Files\nxlog

#ModuleDir %INSTALLDIR%\modules #CacheDir %INSTALLDIR%\data #SpoolDir %INSTALLDIR%\data

define CERTDIR %INSTALLDIR%\cert define CONFDIR %INSTALLDIR%\conf\nxlog.d

Note that these two lines define constants only; the log file location

is ultimately set by the LogFile directive (see below). The

MYLOGFILE define is also used to rotate the log file automatically

(see the _fileop block).

define LOGDIR %INSTALLDIR%\data define MYLOGFILE %LOGDIR%\nxlog.log

If you are not using NXLog Manager, disable the include line

and enable LogLevel and LogFile.

#include %CONFDIR%*.conf

LogLevel INFO LogFile %MYLOGFILE%

<Extension gelf> Module xm_gelf </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _json> Module xm_json </Extension>

<Extension syslog> Module xm_syslog </Extension>

This block rotates %MYLOGFILE% on a schedule. Note that if LogFile

is changed in managed.conf via NXLog Manager, rotation of the new

file should also be configured there.

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    &lt;Exec&gt;
        if ( file_exists('%MYLOGFILE%') and
             (file_size('%MYLOGFILE%') &gt;= 5M) )
        {
             file_cycle('%MYLOGFILE%', 8);
        }
    &lt;/Exec&gt;
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>

<Input eventlog> Module im_msvistalog Query <QueryList>
<Query Id="0">
<Select Path="System"></Select>
<Select Path="Security">
</Select>
<Select Path="Microsoft-IIS-Logging/Logs">*</Select>
</Query>
</QueryList> </Input>

<Input iis_w3c> Module im_file File "X:\inetpub\logs\LogFiles\W3SVC2\u_ex*.log" SavePos TRUE InputType LineBased

Exec		if $raw_event =~ /^#/ drop(); \
            else \
            { \
                w3c_parser-&gt;parse_csv(); \
				$EventTime = parsedate($date + &quot; &quot; + $time); \
				$EventTime = strftime($EventTime, &quot;%Y-%m-%dT%H:%M:%SZ&quot;); \
				$SourceName = &quot;IIS&quot;; \
				$SiteName   = &quot;Test&quot;; \
				$Message = to_json(); \
            }

</Input>

<Output udp> Module om_udp Host XXX.XXX.XXX.XXX --> my_graylog_server Port XXXX --> port OutputType GELF_UDP Exec to_syslog_bsd();

Exec log_info("sending data: " + $raw_event);

</Output>

<Route eventlog_to_udp_win> Path eventlog => udp </Route>

<Route iis_w3c_to_udp_iis> Path iis_w3c => udp </Route>

#2 carlos.caro Nxlog ✓
#1 bercom
Hi, I am doing a test with the agent nxlog EE v.5.3.6735_windows_x64 and after the installation of the agent on the server I stopped receiving the windows audit logs (im_msvistalog module) that I was receiving regularly with the version of nxlog CE v.2.10.2150 and I am not getting the IIS logs either. I attach the implemented configuration file: Panic Soft define INSTALLDIR C:\Program Files\nxlog #ModuleDir %INSTALLDIR%\modules #CacheDir %INSTALLDIR%\data #SpoolDir %INSTALLDIR%\data define CERTDIR %INSTALLDIR%\cert define CONFDIR %INSTALLDIR%\conf\nxlog.d Note that these two lines define constants only; the log file location is ultimately set by the LogFile directive (see below). The MYLOGFILE define is also used to rotate the log file automatically (see the _fileop block). define LOGDIR %INSTALLDIR%\data define MYLOGFILE %LOGDIR%\nxlog.log If you are not using NXLog Manager, disable the include line and enable LogLevel and LogFile. #include %CONFDIR%*.conf LogLevel INFO LogFile %MYLOGFILE% <Extension gelf> Module xm_gelf </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> This block rotates %MYLOGFILE% on a schedule. Note that if LogFile is changed in managed.conf via NXLog Manager, rotation of the new file should also be configured there. <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour &lt;Exec&gt; if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') &gt;= 5M) ) { file_cycle('%MYLOGFILE%', 8); } &lt;/Exec&gt; &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Input eventlog> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="System"></Select> <Select Path="Security"></Select> <Select Path="Microsoft-IIS-Logging/Logs">*</Select> </Query> </QueryList> </Input> <Input iis_w3c> Module im_file File "X:\inetpub\logs\LogFiles\W3SVC2\u_ex*.log" SavePos TRUE InputType LineBased Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c_parser-&gt;parse_csv(); \ $EventTime = parsedate($date + &quot; &quot; + $time); \ $EventTime = strftime($EventTime, &quot;%Y-%m-%dT%H:%M:%SZ&quot;); \ $SourceName = &quot;IIS&quot;; \ $SiteName = &quot;Test&quot;; \ $Message = to_json(); \ } </Input> <Output udp> Module om_udp Host XXX.XXX.XXX.XXX --> my_graylog_server Port XXXX --> port OutputType GELF_UDP Exec to_syslog_bsd(); Exec log_info("sending data: " + $raw_event); </Output> <Route eventlog_to_udp_win> Path eventlog => udp </Route> <Route iis_w3c_to_udp_iis> Path iis_w3c => udp </Route>

Hi Bernardo,

The configuration that you have provided appears to have issues, the xm_csv extension module should be loaded by the Extension block with specific directives to provide the parse_csv() procedure which you can use for parsing the records.

For further information, please refer to Example 435. Collecting W3C format logs with xm_csv in 93. Microsoft IIS section available from our documentation.

Regards,
Carlos.