How to parse JSON logs to syslog format?
nxlog-ce-2.9.1716
Ubuntu 16.04 LTS (OS)
I am trying to parse JSON to SYSLOG
There are two issues.
1. the NoCache TRUE does not seem to work. (I also tried SavePOS false) and it always caches the file.
2. When I do get it to work; it uses the localhost (which is not where the logs are from) and the time/date stamp from the JSON file (but no other data is there).
Is there an example of parsing JSON to _syslog_bsd()? for forwarding to a SEIM
<br/>####################################### <br/># Global directives # <br/>######################################## <br/>User nxlog <br/>Group nxlog <br/> <br/>LogFile /var/log/nxlog/nxlog.log <br/>LogLevel DEBUG <br/>NoCache TRUE <br/> <br/>######################################## <br/># Modules # <br/>######################################## <br/><Extension _json> <br/> Module xm_json <br/></Extension> <br/> <br/><Extension syslog> <br/> Module xm_syslog <br/></Extension> <br/> <br/><Input in> <br/> SavePos FALSE <br/> Module im_file <br/> File '/tmp/json/*.json' <br/> Exec $Message = to_json(); to_syslog_bsd(); <br/></Input> <br/> <br/><Output fileout1> <br/> Module om_file <br/> File "/var/log/nxlog/syslog.log" <br/> Exec to_syslog_bsd(); <br/></Output> <br/> <br/> <br/>######################################## <br/># Routes # <br/>######################################## <br/><Route 1> <br/> Path in => fileout1 <br/></Route> <br/>
resulting output is
<br/><13>Feb 16 02:43:14 [redacted hostname] {"EventReceivedTime":"2018-02-16 02:43:14","SourceModuleName":"in","SourceModuleType":"im_file"} <br/>
Date is localtime for the conversion
Source JSON is OKTA logs
You should not need to convert it to json if it is already json, so the below should work:
File '/tmp/json/*.json'
Exec to_syslog_bsd();
If you are interested we have an NXLog EE add-on for Okta.