Hello,
I'm quite new to nxlog, so forgive me if my question is trivial but I'm having hard time to get the values I extract from my logs using exec and a regex. I have a very large stash of old windows logs in text file (in multiline format), what I want to do is use nxlog to load them in graylog, but I want to format the log in a different way.
It works perfectly when I do not use the the exec and the regexp, but it fails with the message:
2015-08-31 12:12:42 ERROR invalid keyword: $timestamp at C:\Program Files (x86)\nxlog\conf\nxlog.conf:36
The regex works when I test it using http://www.regexr.com/
The error seems in the way I'm trying to assign/write the variables matched by the regex
My nxlog.conf is like the below:
## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension gelf>
Module xm_gelf
</Extension>
<Extension multiline>
Module xm_multiline
HeaderLine /^{/
EndLine /^}/
</Extension>
<Input in>
Module im_file
File "C:\\tmp\\\\example-log.txt"
SavePos TRUE
Recursive TRUE
InputType multiline
exec if $raw_event =~ /(?:\{"([0-9]+?), ([0-9]+?), "(.+?)", "(.+?)", "(?:.+?)","(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)New\sLogon:\s*(?:.+?)\n\s*Account\sName:\s*(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)Source\sNetwork\sAddress:\s*([0-9\.]{7,15})\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)"\})/g;
$timestamp = $1;
$event = $2;
$status = $3;
$type = $4;
$short = $5;
$user = $6;
$source = $7;
</Input>
#<Output out>
# Module om_tcp
# Host 192.168.1.15
# Port 12201
# OutputType GELF_TCP
#</Output>
<Output out>
Module om_file
File 'C:\\tmp\\output'
</Output>
<Route 1>
Path in => out
</Route>
Could someone kindly help me out on this? I tried to read the nxlog manual but I need to confess that I could not understand what I'm doing wrong
Basically I just would the output to be composed by the raw message and the few fields I match with the regex
Thanks very much!
Mark
The Exec directive needs a single line. You need to escape the linebreaks as follows if you want to supply it multiple lines:
exec if $raw_event =~ /(?:\{"([0-9]+?), ([0-9]+?), "(.+?)", "(.+?)", "(?:.+?)","(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)New\sLogon:\s*(?:.+?)\n\s*Account\sName:\s*(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)Source\sNetwork\sAddress:\s*([0-9\.]{7,15})\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)"\})/g; \
$timestamp = $1; \
$event = $2; \
$status = $3; \
$type = $4; \
$short = $5; \
$user = $6; \
$source = $7;