Sending multi-line messages across om_tcp

Post a different question
4
responses

Hello

I am a newbie to using NXLog. I am attempting to send custom multi-line messages read from a txt file using im_file to a server using om_tcp. I understand that I can use xm_multiline to read the file but sending NewLine characters across om_tcp will result in new messages per line on the receiving end.

I have tried replacing the NewLine character in the read block but the replace doesn't seem to replace the NewLine Character.

Can someone help me find a way to send multi-line messages to a tcp listener using NXlog.

This is a sample of the log file with the message start being the @timestamp

@12:02:23.7990 [ISCC] Party removed [ssp view]:
    @ c:007c02772ce2e0f0,03b23dd8 @ m:0000000000000000,00000000,0000000000000000 p:3 i:00001170 nw:00000000:000111705693da93 t:2
      p:041c88c8 @ c:007c02772ce2e0f0,03b23dd8 r:2 t:0 s:c n:233061
    - p:03d51b00 @ c:007c02772ce2e0f0,03b23dd8 r:1 t:1 s:0 n:T4464#1
      p:041ceeb0 @ c:007c02772ce2e0f0,03b23dd8 r:10 t:1 s:ac n:233624

This is the block that I have tried. The \t escape character does work and will replace tabs with the replacement value

<Input IN>
    Module   im_file
    File     'd:\logs\nxlog.txt'
    SavePos  FALSE
    ReadFromLast FALSE
    InputType    multiline
    Exec $raw_event = replace($raw_event, "\r\n", " ");
    Exec $raw_event = replace($raw_event, "\r", " ");
    Exec $raw_event = replace($raw_event, "\n", " ");
    Exec $raw_event = replace($raw_event, "0x0A", " ");
    Exec $raw_event = replace($raw_event, "0x0DA", " ");
    Exec $raw_event = replace($raw_event, "0x0D", " ");
</Input>

Thanks

Brent

 

AskedApril 6, 2016 - 4:19pm

Comments (4)

  • adm's picture
    (NXLog)

    Have you confirmed that the multiline is properly picked up ? You could test with something like this:

    <Output filetest>
 Module om_file
 File "/tmp/test.out"
 Exec $raw_event = "------\n" + $raw_event + "\n--------";
</Output>

     

    April 6, 2016 - 6:04pm
  • bpedersen's picture

    Thanks very much for you quick reply

    I tried putting your statement in and it provided the following results

    ------
    @12:02:23.7990 [ISCC] Party removed [ssp view]:
    --------

    This lead me to believe it had in fact removed the newline character. I changed the statement and removed the \n so it looked like this

    Exec $raw_event = "------" + $raw_event + "--------";

    When I ran this I received the following

    ------@12:02:23.7990 [ISCC] Party removed [ssp view]:--------
    ------    @ c:007c02772ce2e0f0,03b23dd8 @ m:0000000000000000,00000000,0000000000000000 p:3 i:00001170 nw:00000000:000111705693da93 t:2--------

    This is telling me that the newline is being removed in the xm_multiline but another is being added which is preventing the multi-line event from being sent. 

    Is it possible to build the entire event in the In block before it is written in the out block?

    Thanks again

    Brent

    April 6, 2016 - 7:30pm
  • adm's picture
    (NXLog)

    You will need to tune your xm_multiline instance so that it properly concatenates the lines because the above shows that it's picked up in two pieces.

    Without \r and \n replaced with a space you would need to get this first:

    ------
@12:02:23.7990 [ISCC] Party removed [ssp view]:
    @ c:007c02772ce2e0f0,03b23dd8 @ m:0000000000000000,00000000,0000000000000000 p:3 i:00001170 nw:00000000:000111705693da93 t:2
      p:041c88c8 @ c:007c02772ce2e0f0,03b23dd8 r:2 t:0 s:c n:233061
    - p:03d51b00 @ c:007c02772ce2e0f0,03b23dd8 r:1 t:1 s:0 n:T4464#1
      p:041ceeb0 @ c:007c02772ce2e0f0,03b23dd8 r:10 t:1 s:ac n:233624
------

     

    April 6, 2016 - 8:19pm
  • bpedersen's picture

    Than you very much, you pointed me in the right direction.

    I found the HeaderLine value was not correct in the xm_multiline block

    HeaderLine     /^@/    #missing excape character

    HeaderLine     /^\@/   #correct

    April 7, 2016 - 4:17pm

Answers (0)