Multiline not working as expected

Post a different question
5
responses

I am trying to use multiline to ship a log file. Here is my config:

<Extension log>

Module xm_multiline
HeaderLine /^\---Begin event transaction---/
EndLine /^\---Event Reporting Complete---/
</Extension>

<Extension json>

Module xm_json

</Extension>

<Input in>

Module im_file

File "C:\\Users\\Administrator\\Desktop\\log.txt"
SavePos FALSE
ReadFromLast FALSE
Exec to_json();

</Input>

<Output out>

Module om_tcp

Host (server)
Port 5010

</Output>

<Route>

Path in => out

</Route>

I have blank entries in my database; 1 blank entry for each line that should be a part of the multiline (37 in one case).

Any help would be appreciated.

Thanks,

AskedMay 7, 2018 - 10:41pm

Answers (2)

The xm_multiline module also requires an InputType with the name of the xm_multiline instance inside your <Input in> directive. So something like the following in your case :

<Input in>
    Module im_file
    InputType  log
    File "C:\\Users\\Administrator\\Desktop\\log.txt"
    SavePos FALSE
    ReadFromLast    FALSE
    Exec    to_json();
</Input>

Please see https://nxlog.co/documentation/nxlog-user-guide#xm_multiline

If this is one of your first setups, I would also suggest breaking it down into smaller chunks. Make sure you get output to file the way you are expecting on the same system, for instance, before shipping it across the network to a database. It will make troubleshooting easier if you need to perform anymore.

AnsweredMay 7, 2018 - 11:26pm

Comments (3)

  • motts's picture

    I seen in another post where someone used the input type of "multiline", so I tried that, but got an error saying that was an invalid InputType. Using "log" worked just fine though. I changed the conf to this:

    <Input in>

    Module im_file

    File "C:\\Users\\Administrator\\Desktop\\log.txt"
    InputType log
    SavePos FALSE
    ReadFromLast FALSE
    Exec to_json();

    </Input>

    Now I am only getting one blank entry value instead of 37, so I am assuming at least part of the multiline is working. I do not have any parsing or dropping of any data going on.

    I change the output module to be a file on the local server, but this is the only thing in that file:

    {"EventReceivedTime":"2018-05-07 17:54:34","SourceModuleName":"in","SourceModuleType":"im_file"}

    Any other Ideas?

    May 7, 2018 - 11:58pm
  • b0ti's picture
    (NXLog)

    The manual about to_json() states the following:

    string to_json()
Convert the fields to JSON and return this as a string value. The $raw_event field and any field having a 
leading dot (.) or underscore (_) will be automatically excluded.

    If you want your $raw_event to appear in the json as a field then you should do something like this:

    Exec $Message = $raw_event; to_json();

    May 8, 2018 - 8:47am