How to parse JSON logs to syslog format?

Tags:

#1 cbitterfield

nxlog-ce-2.9.1716
Ubuntu 16.04 LTS (OS)

I am trying to parse JSON to SYSLOG

There are two issues.
1. the NoCache TRUE does not seem to work. (I also tried SavePOS false) and it always caches the file.

2. When I do get it to work; it uses the localhost (which is not where the logs are from) and the time/date stamp from the JSON file (but no other data is there).

Is there an example of parsing JSON to _syslog_bsd()? for forwarding to a SEIM

<br/>####################################### <br/># Global directives # <br/>######################################## <br/>User nxlog <br/>Group nxlog <br/> <br/>LogFile /var/log/nxlog/nxlog.log <br/>LogLevel DEBUG <br/>NoCache TRUE <br/> <br/>######################################## <br/># Modules # <br/>######################################## <br/><Extension _json> <br/> Module xm_json <br/></Extension> <br/> <br/><Extension syslog> <br/> Module xm_syslog <br/></Extension> <br/> <br/><Input in> <br/> SavePos FALSE <br/> Module im_file <br/> File '/tmp/json/*.json' <br/> Exec $Message = to_json(); to_syslog_bsd(); <br/></Input> <br/> <br/><Output fileout1> <br/> Module om_file <br/> File "/var/log/nxlog/syslog.log" <br/> Exec to_syslog_bsd(); <br/></Output> <br/> <br/> <br/>######################################## <br/># Routes # <br/>######################################## <br/><Route 1> <br/> Path in => fileout1 <br/></Route> <br/>

resulting output is

<br/><13>Feb 16 02:43:14 [redacted hostname] {"EventReceivedTime":"2018-02-16 02:43:14","SourceModuleName":"in","SourceModuleType":"im_file"} <br/>
Date is localtime for the conversion

Source JSON is OKTA logs

#2 b0ti Nxlog ✓
#1 cbitterfield
nxlog-ce-2.9.1716 Ubuntu 16.04 LTS (OS) I am trying to parse JSON to SYSLOG There are two issues. 1. the NoCache TRUE does not seem to work. (I also tried SavePOS false) and it always caches the file. 2. When I do get it to work; it uses the localhost (which is not where the logs are from) and the time/date stamp from the JSON file (but no other data is there). Is there an example of parsing JSON to _syslog_bsd()? for forwarding to a SEIM <br/>####################################### <br/># Global directives # <br/>######################################## <br/>User nxlog <br/>Group nxlog <br/> <br/>LogFile /var/log/nxlog/nxlog.log <br/>LogLevel DEBUG <br/>NoCache TRUE <br/> <br/>######################################## <br/># Modules # <br/>######################################## <br/><Extension _json> <br/> Module xm_json <br/></Extension> <br/> <br/><Extension syslog> <br/> Module xm_syslog <br/></Extension> <br/> <br/><Input in> <br/> SavePos FALSE <br/> Module im_file <br/> File '/tmp/json/*.json' <br/> Exec $Message = to_json(); to_syslog_bsd(); <br/></Input> <br/> <br/><Output fileout1> <br/> Module om_file <br/> File "/var/log/nxlog/syslog.log" <br/> Exec to_syslog_bsd(); <br/></Output> <br/> <br/> <br/>######################################## <br/># Routes # <br/>######################################## <br/><Route 1> <br/> Path in => fileout1 <br/></Route> <br/> resulting output is <br/><13>Feb 16 02:43:14 [redacted hostname] {"EventReceivedTime":"2018-02-16 02:43:14","SourceModuleName":"in","SourceModuleType":"im_file"} <br/> Date is localtime for the conversion Source JSON is OKTA logs

You should not need to convert it to json if it is already json, so the below should work:

    File    '/tmp/json/*.json'
    Exec  to_syslog_bsd();

If you are interested we have an NXLog EE add-on for Okta.