I am trying to send some logs into ELK and I am running into a bit of a snag. The logs are delimited by space and there doesn't seem to be an options to change that easily.
I am not really sure how to go about getting the logs sent to my monitoring solution in a formatted way, preferably JSON.
These are apache error logs:
[Fri May 31 14:21:38 2019] [error] [client 18.104.22.168] File does not exist: /home/test/test.xml, referer: https://www.test-group.com/
define REGEX /(?x)^\[\S+\ ([^\]]+)\]\ \[(\S+):(\S+)\]\ (\[client\ (\S+)\]\ )?(.+)$/
if $raw_event =~ /(?x)^\[\S+\ ([^\]]+)\]\ \[(\S+):(\S+)\]\ (\[client\ (\S+)\]\ )?(.+)$/
$EventTime = parsedate($1);
$ApacheModule = $2;
$ApacheLogLevel = $3;
$Message = $4;
It's sending the logs to ELK, but the data isn't in a usable format there. Everything looks just like it does in plain text, no fields or values. If I add the "exec to_json();" option, then I have empty logs in ELK. Maybe something is wrong with my regex, but I copied what I could from the reference manual for this log, though this log is missing data from the example in the guide.
Thanks for your time