I am using nxLog CE as log shipper, but I found issue for GELP module with millisecond precision in time stamp. The millisecond precision is not added to timestamp (EventTime as well) When I use gelf extension the millisecond precision is dropped. Basically although I have in log file something like this: <br/>... <br/>2021-04-26T14:28:14.269+00:00 [sync_ldap_trigger.sh] [DEBUG] Get list of users from AD/LDAP successfully (with 1 records) <br/>... <br/> and for parsing I am using REGEX for multi line detection. Doesn't matter if I am using parsing date function parsedate or strptime or nothing just pure $1 <br/>#Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = parsedate($1); $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/>#Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = strptime($1, '%Y-%m-%d%t%H:%M:%S.%N%Z'); $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/>Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = $1; $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/> I excluded "buffer" from path to be sure and all parsing are done in im_file section. For sure I am testing it with file output from the same source. But on the tcpdump I still have timestamp without the millisecond precision - which I confirmation form me that this information is lost on shipper site. tcpdump detailed packet output: <br/>CNSCNc@{"version":"1.1","_EventReceivedTime":"2021-04-26 14:32:15","_SourceModuleName":"peosynclog","_SourceModuleType":"im_file","_script":"sync_ldap_trigger.sh","_Severity":"DEBUG","_SyslogSeverityValue":7,"_FileName":"/home/adsync/bin/adsync2amish/log/sync_general.log","_order":"VNB","_environment":"TEST","host":"amishd-test.vnbrno.cz","short_message":"Get list of users from AD/LDAP successfully (with 1 records)","full_message":"Get list of users from AD/LDAP successfully (with 1 records)","timestamp":1619447535,"level":7} <br/> nxlog.conf: <br/>######################################## <br/># Global directives # <br/>######################################## <br/>User nxlog <br/>Group nxlog <br/>LogFile /var/log/nxlog/nxlog.log <br/>LogLevel INFO <br/># PeoSYNC <br/>define PEOSYNCREGEX /^(?<EventTime>\d{4}\-\d{2}\-\d{2}T\d{2}\:\d{2}\:\d{2}.\d{3}\+\d{4})[\ ,\t]*\[(?<Script>[\S,\.,\_]+)\][\ ,\t]*\[(?<Severity>[\S,\-,\ ]+)\][\ ,\t]*(?<Message>[\S,\s]*)/ <br/>define ROOT /var/spool/nxlog <br/>CacheDir %ROOT%/data <br/>SpoolDir %ROOT%/data <br/>######################################## <br/># Modules # <br/>######################################## <br/><Extension multilinepeosync> <br/> Module xm_multiline <br/> HeaderLine %PEOSYNCREGEX% <br/></Extension> <br/><Extension gelf> <br/> Module xm_gelf <br/></Extension> <br/>######################################## <br/># Inputs # <br/>######################################## <br/><Input peosynclog> <br/> Module im_file <br/> File "/home/adsync/bin/engine/log/sync_general.log" <br/> InputType multilinepeosync <br/> SavePos TRUE <br/> #Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = parsedate($1); $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/> #Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = strptime($1, '%Y-%m-%d%t%H:%M:%S.%N%Z'); $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/> Exec if $raw_event =~ %PEOSYNCREGEX% $EventTime = $1; $script = $2 ; $Severity = $3 ; $Message = $4 ; <br/> <Exec> <br/> if $Severity !~ /NEVERUSEDSTRING/ <br/> { <br/> if $Severity =~ /ALERT/ $SyslogSeverityValue = 1; <br/> if $Severity =~ /CRIT/ $SyslogSeverityValue = 2; <br/> if $Severity =~ /ERR/ $SyslogSeverityValue = 3; <br/> if $Severity =~ /WARN/ $SyslogSeverityValue = 4; <br/> if $Severity =~ /NOTICE/ $SyslogSeverityValue = 5; <br/> if $Severity =~ /INFO/ $SyslogSeverityValue = 6; <br/> if $Severity =~ /---/ $SyslogSeverityValue = 6; <br/> if $Severity =~ /SQL/ $SyslogSeverityValue = 7; <br/> if $Severity =~ /DEBUG/ $SyslogSeverityValue = 7; <br/> } else { <br/> # Parsing and dropping all messages which are NEVERUSEDSTRING (or INFO|DEBUG if you want change) severity <br/> drop(); <br/> } <br/> $FileName = file_name(); <br/> $Hostname = 'someserver-dev.loc'; <br/> $order = 'ORDER'; <br/> $environment = 'DEV'; <br/> # if not defined($EventTime) $EventTime = now(); <br/> </Exec> <br/></Input> <br/>######################################## <br/># Outputs # <br/>######################################## <br/><Output graylogtcppeosyncrewrite> <br/> Module om_tcp <br/> Host 127.0.0.1 <br/> Port 20228 <br/> OutputType GELF_TCP <br/></Output> <br/><Output file> <br/> Module om_file <br/> File "/tmp/sync_general.log" <br/></Output> <br/>######################################## <br/># Define Processor Modules # <br/>######################################## <br/><Processor bufferpeosync> <br/> Module pm_buffer <br/> MaxSize 524288 <br/> Type disk <br/> WarnLimit 393216 <br/></Processor> <br/>######################################## <br/># Routes # <br/>######################################## <br/><Route 1> <br/> #Path peosynclog => bufferpeosync => graylogtcppeosyncrewrite <br/> Path peosynclog => graylogtcppeosyncrewrite <br/></Route> <br/><Route 2> <br/> Path peosynclog => file <br/></Route> <br/> Please anyone for help or solution for this issue. Michal BTW: Of course that I tried more format for timestamp as 2021-04-26T14:28:04.240+02:00 or 2021-04-26T14:28:04.240+0200 etc. The format in log is in my hands too.

Hello everybody, I'm sorry to bother you with another question concerning Windows Eventlog forwarding to graylog. Unfortunately I'm not able to figure this out on my own. used versions: nxlog 2.10.2102 (running on Windows Server 2016) graylog 2.4.6 (running on Debian 9) I have two nxlog setups. One using syslog and another one using GELF. Both do not work as I would expect. 1. Syslog <Extension syslog> Module xm_syslog </Extension> <Input eventlog> Module im_msvistalog Exec delete($Keywords); Exec if ($EventType == "VERBOSE") drop(); </Input> <Output out_graylog> Module om_tcp Host graylog Port 5140 Exec $raw_event = replace($raw_event, "\n", " "); Exec $raw_event = replace($raw_event, "\r", " "); Exec $raw_event = replace($raw_event, "\t", " "); Exec to_syslog_ietf(); </Output> <Route route_eventlog> Path eventlog => out_graylog </Route> The problem is that there are eventlog entries containing line breaks. Unfortunately they are not removed by the replace commands. So in graylog one message is split into many messages with every linebreak. Using wireshark I can observe that the linebreaks consist of LF characters (Unix line endings). 2. Gelf <Extension gelf> Module xm_gelf </Extension> <Input eventlog> Module im_msvistalog Exec delete($Keywords); Exec if ($EventType == "VERBOSE") drop(); </Input> <Output out_graylog> Module om_tcp Host graylog Port 12201 OutputType GELF </Output> <Route route_eventlog> Path eventlog => out_graylog </Route> Unfortunately this setup does not work at all. No messages are showing up in Graylog (of course I've activated the correspnding input). Using wireshark I can observe that a lot of TCP packets are sent to graylog but none of them contain readable messages. Can anybody help me with either setup? Thanks and Regards, Carsten

Hi, i am using the nxlog-ce-2.9.1716-1_rhel7.x86_64.rpm and i have the following conf <Output graylogout> Module om_udp Host somehost.com Port 12201 OutputType GELF </Output> If i do "host somehost.com" -> It returns 2 IP in different order $>host somehost.com X.X.X.X Y.Y.Y.Y $>host somehost.com Y.Y.Y.Y X.X.X.X But nxlog always send messages to only one host (X.X.X.X). Does nxlog do a DNS request once? if is that the case, how do i balance the traffic?

We have nxlog CE pushing to a GELF TCP input in Graylog, and the timestamp field received from nxlog appears to not have the milliseconds (i.e. it ends in ".000"), resulting in out-of-order messages in Graylog within a 1-second window.  Other sources (Graylog Collectors, apps pushing directly, etc.) include the original millisecond value as expected.  For Graylog inputs receiving nxlog messages we've had to set up an extractor to extract the timestamp from the message itself.  Are there any options to keep millisecond precision with nxlog?

Hi all, I tried to use the gelf extension with the tcp module, but it doesn't work. I saw tht the commercial version includes a special tcp gelf enxtension. Supports the community edition the usage of GELF over TCP? Kind regards Michael