Struggling getting SMTP logs to work.

Tags:

#1 Mr_M_Cox

Hi All,

I have previously got smtp logs to go into Graylog using NXlog, it was worknig fine. I then had a disk sapce issue on the graylog host so had to redo some bits, including the nxlog.conf for our SMTP server. 

The SMTP log header specifes the following

#Software: Microsoft Internet Information Services 8.5
#Version: 1.0
#Date: 2017-03-26 23:00:10
#Fields: date time c-ip cs-username s-sitename s-computername s-port cs-method cs-uri-query sc-bytes time-taken 

This is the error I am getting

ERROR if-else failed at line 44, character 436 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 224 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 11, got 8 in input

I ahve checked and rechecked and their should be 11 items as per the .conf.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module         xm_gelf
</Extension>

<Extension fileop>
    Module         xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module             xm_csv
    Fields             $date, $time, $c-ip, $cs-username, $s-sitename, $s-computername, $s-port, $cs-method, $cs-uri-query, $sc-bytes, $time-taken
    FieldTypes         string, string, string, string, string, string, integer, string, string, integer, integer
    Delimiter         ' '
    QuoteChar         '"'
    EscapeControl     FALSE
    UndefValue         '-'
</Extension>

<Input smtp>
    Module        im_file
    File        "C:\\Logs\\SMTPSVC1\\\ex*.log"
    SavePos      TRUE

    Exec        if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "smtp";                         \
                    $Message = to_json();                         \
                }
</Input>

<Input eventlog>
    Module      im_msvistalog
</Input>

<Output graylog>
    Module      om_udp
    Host        graylog.mydomain.com
    Port        12201
    OutputType    GELF

    #Use the following line for debugging (uncomment the fileop extension above as well)
    #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Route eventlog>
   Path        eventlog => graylog
</Route>

<Route smtp-to-graylog>
    Path        smtp => graylog
</Route>

Its so frustracting that I know this was working correctly.

Any help would be great.

Thanks

#6 b0ti Nxlog ✓
#1 Mr_M_Cox
Hi All, I have previously got smtp logs to go into Graylog using NXlog, it was worknig fine. I then had a disk sapce issue on the graylog host so had to redo some bits, including the nxlog.conf for our SMTP server.  The SMTP log header specifes the following #Software: Microsoft Internet Information Services 8.5 #Version: 1.0 #Date: 2017-03-26 23:00:10 #Fields: date time c-ip cs-username s-sitename s-computername s-port cs-method cs-uri-query sc-bytes time-taken  This is the error I am getting ERROR if-else failed at line 44, character 436 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 224 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 11, got 8 in input I ahve checked and rechecked and their should be 11 items as per the .conf. define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf>     Module         xm_gelf </Extension> <Extension fileop>     Module         xm_fileop </Extension>  <Extension json>     Module      xm_json </Extension> # Create the parse rule for IIS logs. You can copy these from the header of the IIS log file. <Extension w3c>     Module             xm_csv     Fields             $date, $time, $c-ip, $cs-username, $s-sitename, $s-computername, $s-port, $cs-method, $cs-uri-query, $sc-bytes, $time-taken     FieldTypes         string, string, string, string, string, string, integer, string, string, integer, integer     Delimiter         ' '     QuoteChar         '"'     EscapeControl     FALSE     UndefValue         '-' </Extension> <Input smtp>     Module        im_file     File        "C:\\Logs\\SMTPSVC1\\\ex*.log"     SavePos      TRUE     Exec        if $raw_event =~ /^#/ drop();                    \                 else                                             \                 {                                                \                     w3c->parse_csv();                            \                     $EventTime = parsedate($date + " " + $time); \                     $SourceName = "smtp";                         \                     $Message = to_json();                         \                 } </Input> <Input eventlog>     Module      im_msvistalog </Input> <Output graylog>     Module      om_udp     Host        graylog.mydomain.com     Port        12201     OutputType    GELF     #Use the following line for debugging (uncomment the fileop extension above as well)     #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event); </Output> <Route eventlog>    Path        eventlog => graylog </Route> <Route smtp-to-graylog>     Path        smtp => graylog </Route> Its so frustracting that I know this was working correctly. Any help would be great. Thanks

> I ahve checked and rechecked and their should be 11 items as per the .conf.

It got 8 as per the error message you quoted:

> Not enough fields in CSV input, expected 11, got 8 in input