2 messages sent by NXLog instead of 1

Tags:

#1 StanJames

Hello Guys,

I have a problem, that NXLog community edition sometimes sends 2 messages to GrayLog instead of 1, from a log file.
These are the messages, that came through together for example:

08 2016/12/02 13:13:28.581 Response: << MerchantId^XXXXXX~TransactionType^XX~OrderNumber^XXXXXXXX~StrId^XXXXXXXXXX~PTTID^XXXXXXXXXXX~MOP^XX~CurrencyId^XXX~Amount^XX.XX~AuthCode^XXXXXX~RequestType^X~MessageCode^XXXX~Message^XXXXXXXXX XXXXXXXX~CVNMessageCode^X~CVNMessage^XXXXXXXX/XXXXXX XXXXXXX >>
08 2016/12/02 13:13:28.581 ReportResult: (IDMMSITransaction = XXXXXXXX, Result = , MessageCode = XXXX)

This is the nxlog.conf part for this log:
 

<Extension exlogs>
    Module        xm_multiline
    HeaderLine    /^.. \d{4}\/\d{2}\/\d{2} \d{2}\:\d{2}\:\d{2}\.\d{3}/
</Extension>

<Input Logs>
    Module        im_file
    File        "D:\\path\\to\\logs\\log_*"
    SavePos        TRUE
    InputType    exlogs
</Input>

<Route logs-graylog>
    Path        Logs => graylog
</Route>

 

The problem doesn't happen with all the messages, but quite often and prevents me to create proper extractors.

Do you have any idea what could cause this?

 

Thanks,

Tamas Juhasz

tjuhasz@stanjames.com

#2 b0ti Nxlog ✓ (Last updated )
#1 StanJames
Hello Guys, I have a problem, that NXLog community edition sometimes sends 2 messages to GrayLog instead of 1, from a log file. These are the messages, that came through together for example: 08 2016/12/02 13:13:28.581 Response: << MerchantId^XXXXXX~TransactionType^XX~OrderNumber^XXXXXXXX~StrId^XXXXXXXXXX~PTTID^XXXXXXXXXXX~MOP^XX~CurrencyId^XXX~Amount^XX.XX~AuthCode^XXXXXX~RequestType^X~MessageCode^XXXX~Message^XXXXXXXXX XXXXXXXX~CVNMessageCode^X~CVNMessage^XXXXXXXX/XXXXXX XXXXXXX >> 08 2016/12/02 13:13:28.581 ReportResult: (IDMMSITransaction = XXXXXXXX, Result = , MessageCode = XXXX) This is the nxlog.conf part for this log:   <Extension exlogs>     Module        xm_multiline     HeaderLine    /^.. \d{4}\/\d{2}\/\d{2} \d{2}\:\d{2}\:\d{2}\.\d{3}/ </Extension> <Input Logs>     Module        im_file     File        "D:\\path\\to\\logs\\log_*"     SavePos        TRUE     InputType    exlogs </Input> <Route logs-graylog>     Path        Logs => graylog </Route>   The problem doesn't happen with all the messages, but quite often and prevents me to create proper extractors. Do you have any idea what could cause this?   Thanks, Tamas Juhasz tjuhasz@stanjames.com

This issue is sometimes caused by a regular expression used in HeaderLine that does not properly match for every event record. I couldn't spot an issue with that in this case though.

If you think the problem is caused by xm_multiline not being able to separate event records properly then please create a reproducible test case that demonstrates the issue. It should have the following:

  • nxlog.conf with im_file and om_file
  • Input sample
  • Output sample that shows the separated records

See this example for a similar setup.