3
responses

Hello,

I am using nxlog to parse log files.

Everything is working perfectly, however, when the log file is rotated, the nxlog application has errors in the parse of the logs.

I did a research, and found the following document:

https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer

Apparently this bug had been solved in the past.

Is it possible that it has returned in some file merge?

Or, could it be a new bug?

 

2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042106-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042101-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042103-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042105-1.LOG
2017-05-10 17:00:03 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server'
2017-05-10 17:00:39 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMS2017041020-1.LOG
2017-05-10 17:00:39 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server'

https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer

<Input exmsgtrk>
   Module im_file
   File "%EXMSGTRK%\MSGTRK????????*-*.LOG"
   SavePos TRUE
   InputType LineBased
   Exec    if $raw_event =~ /HealthMailbox/ drop();
   Exec    if $raw_event =~ /Mbx_/ drop();
   Exec    if $raw_event =~ /^#/ drop();

   Exec    ExtMessageTracking->parse_csv();
   Exec    $EventTime = parsedate($EmailTime);
   Exec    $Message = "Subject: " + $EmailSubject;
</Input>

Version: nxlog-ce-2.9.1716.msi

AskedMay 10, 2017 - 10:20pm

Comments (1)

Answer (1)

I think this is caused by the UTF-8 BOM header in the file. See the solution under this forum question.

Comments (1)

  • samysilva's picture

    The reason for the problem is the encoding of the log files.

    Logs in UTF8-BOM format have a header.
    This way the first line was not removed by the filter (#)

    C:\Users\ssoliveira\Desktop> type MSGTRK2017051122-1.LOG | More
     # Software: Microsoft Exchange Server
    #Version: 15.00.1130.005
    # Log-type: Message Tracking Log
    #Date: 2017-05-11T22: 00: 01.520Z

    Shown in the above example by the characters ()

    To resolve it was necessary to add the following line:

    Exec if $raw_event =~ /^#/ drop();
    Exec if $raw_event =~ /^\xEF\xBB\xBF#/ drop();