3
responses

Setup NXLog to send IIS logs to Syslog.

Using the following modules:

xm_syslog xm_csv im_file om_tcp

My config file is the following:

Panic Soft
#NoFreeOnExit TRUE

define ROOT     e:\Program Files (x86)\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf
define LOGDIR   %ROOT%\data
define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _charconv>
    Module      xm_charconv
    AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>

<Extension _exec>
    Module      xm_exec
</Extension>

<Extension _fileop>
    Module      xm_fileop

    # Check the size of our log file hourly, rotate if larger than 5MB
    <Schedule>
        Every   1 hour
        Exec    if (file_exists('%LOGFILE%') and \
                   (file_size('%LOGFILE%') >= 5M)) \
                    file_cycle('%LOGFILE%', 8);
    </Schedule>

    # Rotate our log file every week on Sunday at midnight
    <Schedule>
        When    @weekly
        Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
    </Schedule>
</Extension>

<Extension w3c_parser>
    Module          xm_csv
    Fields          date, time, s-ip, cs-method, cs-uri-stem, cs-uri-query, \
                    s-port, cs-username, c-ip, cs(User-Agent), cs(Referer), \
                    sc-status, sc-substatus, sc-win32-status, time-taken
    FieldTypes      string, string, string, string, string, string, integer, \
                    string, string, string, string, integer, integer, integer, \
                    integer
    Delimiter       ' '
    EscapeChar      '"'
    QuoteChar       '"'
    EscapeControl   FALSE
    UndefValue      -
</Extension>

<Input iis_w3c>
    Module          im_file
    File            'L:\Logs\W3SVC1\u_ex*.log'
    <Exec>
        if $raw_event =~ /^#/ drop();
        else
        {
            w3c_parser->parse_csv();
            $EventTime = parsedate($date + "T" + $time + ".000Z");
        }
    </Exec>
</Input>

<Output out>
    Module          om_tcp
    Host            REDACTED
    Port            514
    Exec            to_syslog_ietf();
</Output>

<Output test>
    Module          om_file
    File        'E:\Program Files (x86)\nxlog\test.log'
    CreateDir
</Output>

<Route w3c>
    Path iis_w3c => w3c_parser => out
</Route>    

When I start NXLog I keep getting the following error:

ERROR ### ASSERTION FAILED at line 879 in module.c/resume_senders(): "curr->type == NX_MODULE_TYPE_INPUT" ###

Also, I can see the NXLog client is opening a TCP connection with the syslog destination but doesn't actually send any data in the transaction. (Packet trace shows SYN-SYN/ACK-ACK then immediately FIN from the client)

I tried testing with "om-file" to see if it would at least write to a local file, this failed as well (same error in the log) - so something tells me I have an error in the input module IM_FILE but I can'T figure out what the error is. I took the config example from the user-guide (https://nxlog.co/documentation/nxlog-user-guide/iis.html).

Any help appreciated!

AskedJune 4, 2020 - 5:57pm

Comments (3)

  • Arkadiy's picture
    (NXLog)

    Hello,

    Two points to troubleshoot this:
    - you may try to use im_file just pointing at certain file, without additional logic;
    - also would you drop us a full log?

    Regards, Arch

  • bmalenfant's picture

    @Arkadiy,

    Thanks for the hint.

    So I attempted the following:

    When I start NXlog without the EXEC tags section for IM_FILE:

    <Input iis_w3c>
        Module          im_file
        File            'L:\Logs\W3SVC1\u_ex*.log'
    </Input>
    

    I also changed the path to write to a local file (instead of syslog) to better be able to test-- note that this doesn't change the behavior

    The service starts, I see the process reading all the files in my IIS folder (ran a procmon to monitor) and then it crashes after reading the last file in the folder.

    Here is the full log:

    2020-06-05 10:45:23 WARNING not starting unused module out
    2020-06-05 10:45:23 INFO nxlog-ce-2.10.2150 started
    2020-06-05 10:46:08 CRITICAL ### PANIC at line 879 in module.c/resume_senders(): "### ASSERTION FAILED: "curr->type == NX_MODULE_TYPE_INPUT" ###" ###
    

    I also have a PROCDUMP of the process when it crashed that I can send you (50mbytes)

  • bmalenfant's picture

    So it looks like I found the issue.

    There was a logic problem with my ROUTE

    <Input iis_w3c>
        Module          im_file
        File            'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
        <Exec>
            if $raw_event =~ /^#/ drop();
            else
            {
                w3c_parser->parse_csv();
                $EventTime = parsedate($date + "T" + $time + ".000Z");
            }
        </Exec>
    </Input>
    
    <Route w3c>
    Path iis_w3c => w3c_parser => out
    </Route>    
    

    So in the IM_FILE section, there is an EXEC block that tells IM_FILE to use the W3C_PARSER extension (XM_CSV module) to parse the IIS logs it reads:

    w3c_parser->parse_csv();
    

    Then in the ROUTE section, I'm telling NXLog to use IM_FILE as the input, then put it through XM_CSV as the processor and then output to OM_TCP.

    Problem here is that when IM_FILE is done reading the files in the IIS folder, it will execute the XM_CSV parser on the events and then in the Path it's told to re-run the processor XM_CSV after it's read the file, creating a loop and crashing the process.

    The solution to all this was simply to change the ROUTE to:

    <Route w3c>
       Path iis_w3c => out
    </Route>    
    

    Bingo.

Answers (0)