Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Request a simple example for processing AWS Logs from the S3 Bucket.
cbitterfield created
I am trying to parse AWS S3 Logs. They are in JSON format (One line no CR/LF) and Gzip'd.
I need to ingest these into syslog TCP or UDP (Testing with file out)
I can't get a reliable working nxlog.conf that will process the JSON files.
NXLOG
define ROOT /usr/local/libexec/nxlog/
Pidfile /var/run/nxlog.pid
LogFile ./nxlog.log
define WORK /Users/cbitterfield/awslogs-project
SpoolDir %WORK%/data
CacheDir %WORK%/data
LogLevel DEBUG
Module xm_syslog
Module xm_json
Module im_file
File "%WORK%/data19/*.json"
Exec parse_json();
# Dump $raw_event
Exec to_syslog_bsd();
SavePos FALSE
ReadFromLast False
Module om_file
File "./output"
Path in => out
Yields the following errors and no output.
2016-04-10 22:13:00 DEBUG '^KE<F4>t^G<C7>C^D' does not match wildcard '859121128579_CloudTrail_ap-northeast-.json'
2016-04-10 22:13:00 DEBUG checking '^KE<F4>t^G<C7>C^D' against wildcard '859121128579_CloudTrail_ap-northeast-.json':
2016-04-10 22:13:00 DEBUG '^KE<F4>t^G<C7>C^D' does not match wildcard '859121128579_CloudTrail_ap-northeast-.json'
2016-04-10 22:13:00 DEBUG checking '^KE<F4>t^G<C7>C^D' against wildcard '859121128579_CloudTrail_ap-northeast-.json':
2016-04-10 22:13:00 DEBUG '^KE<F4>t^G<C7>C^D' does not match wildcard '859121128579_CloudTrail_ap-northeast-.json'
2016-04-10 22:13:00 DEBUG checking '^KE<F4>t^G<C7>C^D' against wildcard '859121128579_CloudTrail_ap-northeast-.json':
cbitterfield created