1
response

Hi,

 

I have following log:

 

23.08.2016 22:00:00: [20740] INFO: Line 1
23.08.2016 22:00:00: [20740] Line 2
23.08.2016 22:00:00: [20740] Line 3
23.08.2016 22:00:00: [20740] Line 4
23.08.2016 22:00:00: [20740] Line 5

23.08.2016 22:00:00: [20745] INFO: Line 1
23.08.2016 22:00:00: [20745] Line 2
23.08.2016 22:00:00: [20745] Line 3
23.08.2016 22:00:00: [20745] Line 4
23.08.2016 22:00:00: [20745] Line 5

 

Each multiline log line is beginning with an empty line. So I tried to use the empty line as header:

<Extension multilineEmtpyLine>
    Module xm_multiline
    HeaderLine /^$/
</Extension>

<Input foo>
    Module im_file
    File "C:/logfile/foo.log"
    
    #enabling multilining
    InputType multilineEmtpyLine
    SavePos TRUE
    Exec $Message = $raw_event;
</Input>

<Output localTCP>
    Module om_tcp
    Host localhost
    Port 5544
   
    Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
    Exec $raw_event = to_json();
  
    # Uncomment for debug output
    Exec file_write('c:\nxlog\nxlog_localtcp_debug_output.log', $raw_event + "\n");
</Output>

<Route nxlogLocal>
    #Path topbeat_debug, ttp_debug => localTCP
    Path foo=> localTCP
</Route>

As I see in debug output and logstash each source line will be transmitted as single line. Multilining is not working.

How can I get it work?

thanks, Andreas

AskedAugust 25, 2016 - 1:30pm

Answer (1)

Here is an example to show you that it works:

<Extension multi>
    Module      xm_multiline
    HeaderLine  /^$/
</Extension>

<Extension json>
    Module      xm_json
</Extension>

<Input in>
    Module      im_file
    File        "tmp/input.txt"
    ReadFromLast FALSE
    InputType   multi
    Exec        $Message = $raw_event;
</Input>

<Output out>
    Module      om_file
    File        "tmp/multi.out"
    Exec        to_json();
</Output>

<Route 2>
    Path        in => out
</Route>

Input:


event 1 first line

event 2 first line
event 2 second line

event 3 first line

The output produced is the following:

{"EventReceivedTime":"2016-08-25 13:41:59","SourceModuleName":"in","SourceModuleType":"im_file","Message":"\nevent 1 first line"}
{"EventReceivedTime":"2016-08-25 13:41:59","SourceModuleName":"in","SourceModuleType":"im_file","Message":"\nevent 2 first line\nevent 2 second line"}
{"EventReceivedTime":"2016-08-25 13:41:59","SourceModuleName":"in","SourceModuleType":"im_file","Message":"\nevent 3 first line"}

You can see that event 2 is in a single json record.