Parse_Csv Failed when input file was deleted

Tags:

#1 samysilva

Hello,

I am using nxlog to parse log files.

Everything is working perfectly, however, when the log file is rotated, the nxlog application has errors in the parse of the logs.

I did a research, and found the following document:

https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer

Apparently this bug had been solved in the past.

Is it possible that it has returned in some file merge?

Or, could it be a new bug?

 

2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042106-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042101-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042103-1.LOG
2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042105-1.LOG
2017-05-10 17:00:03 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server'
2017-05-10 17:00:39 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMS2017041020-1.LOG
2017-05-10 17:00:39 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server'


https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer

<Input exmsgtrk>
   Module im_file
   File "%EXMSGTRK%\MSGTRK????????*-*.LOG"
   SavePos TRUE
   InputType LineBased
   Exec    if $raw_event =~ /HealthMailbox/ drop();
   Exec    if $raw_event =~ /Mbx_/ drop();
   Exec    if $raw_event =~ /^#/ drop();

   Exec    ExtMessageTracking->parse_csv();
   Exec    $EventTime = parsedate($EmailTime);
   Exec    $Message = "Subject: " + $EmailSubject;
</Input>

Version: nxlog-ce-2.9.1716.msi

#2 samysilva
#1 samysilva
Hello, I am using nxlog to parse log files. Everything is working perfectly, however, when the log file is rotated, the nxlog application has errors in the parse of the logs. I did a research, and found the following document: https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer Apparently this bug had been solved in the past. Is it possible that it has returned in some file merge? Or, could it be a new bug?   2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042106-1.LOG 2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042101-1.LOG 2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042103-1.LOG 2017-05-10 17:00:03 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMD2017042105-1.LOG 2017-05-10 17:00:03 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server' 2017-05-10 17:00:39 WARNING input file was deleted: D:\Program Files\Microsoft\Exchange\V15\TransportRoles\Logs\MessageTracking\MSGTRKMS2017041020-1.LOG 2017-05-10 17:00:39 ERROR procedure 'parse_csv' failed at line 64, character 43 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 27, got 1 in input '#Software: Microsoft Exchange Server' https://nxlog.co/support-tickets/xmcsv-fails-parse-line-when-cr-split-lf-and-its-integer <Input exmsgtrk>    Module im_file    File "%EXMSGTRK%\MSGTRK????????*-*.LOG"    SavePos TRUE    InputType LineBased    Exec    if $raw_event =~ /HealthMailbox/ drop();    Exec    if $raw_event =~ /Mbx_/ drop();    Exec    if $raw_event =~ /^#/ drop();    Exec    ExtMessageTracking->parse_csv();    Exec    $EventTime = parsedate($EmailTime);    Exec    $Message = "Subject: " + $EmailSubject; </Input> Version: nxlog-ce-2.9.1716.msi

Another user with the same problem in 2.9.17.16 version
https://community.graylog.org/t/smtp-logs-not-parsing-correctly/625