hello, I am testing the NXlog EE, but the module xm_w3c does not work, do not parse the logs of BRO, you can help me. <Extension w3c> Module xm_w3c Delimiter , </Extension> <Input i.bro.log> Module im_file File "/mnt/*.log" InputType w3c </Input> <Output o.bro.log> Module om_ssl Host 192.168.0.38 Port 10525 CAFile /data/conf/ca.crt AllowUntrusted TRUE </Output> <Route r.bro.log> Path i.bro.log => o.bro.log </Route> # ./nxlog-processor 2017-12-27 20:38:33 INFO connecting to 192.168.0.38:10525 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 15 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 10 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 34 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE Bro Files ![Bro files][Bro files] Graylog2

HI all Is the module available for community version of nxlog and if yes how do we download? Thanks all for your time. Chew

Hello I have a question about Suppressed in pm_evcorr. Having following example from official documentation: <Input in> Module im_file File "/tmp/testfile" SavePos FALSE ReadFromLast FALSE Exec if ($raw_event =~ /^(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d) (.+)/) { \ $EventTime = parsedate($1); \ $Message = $2; \ $raw_event = $Message; \ } </Input> <Input internal> Module im_internal Exec $raw_event = $Message; Exec $EventTime = 2010-01-01 00:01:00; </Input> <Output out> Module om_file File '/tmp/output' </Output> <Processor evcorr> Module pm_evcorr TimeField EventTime <Suppressed> # match input event and execute an action list, but ignore the following # matching events for the next t seconds. Condition $Message =~ /^suppressed/ Interval 30 Exec $raw_event = "suppressing.."; </Suppressed> <Simple> Exec if $Message =~ /^simple/ $raw_event = "got simple"; </Simple> </Processor> <Route 1> Path in, internal => evcorr => out </Route> Wrote following logs into the file: [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:44 suppressed" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:47 simple" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:49 simple" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:07:00 suppressed" >> testfile In output I got: suppressing.. got simple got simple suppressed Suppressing condition worked. But I thought that will stop processing all subsequent log entries. And it's not. Why the Simple condition is still matched?

Hello Is it possible to monitor the log file modification date? I do not want to check log file contents, to check whether pattern was found or not. The only thing which I want to is to get modification date of a log file, and if it's older than X minutes -> generate an event. I tried different configs, with schedule, with im_null modules, exec, file_mtime function... And nothing... Still doesn't work.

Good morning, Noob to nxlog - installed in Windows and running on an OOB config, Host is set to localhost. All I get in the logs is: 2017-12-21 11:31:44 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it. 2017-12-21 11:32:16 INFO connecting to Localhost:514 2017-12-21 11:32:17 INFO reconnecting in 64 seconds 2017-12-21 11:32:17 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it. 2017-12-21 11:33:21 INFO connecting to Localhost:514 2017-12-21 11:33:22 INFO reconnecting in 128 seconds 2017-12-21 11:33:22 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it. The firewall is disabled, 514 not showing as listening in netstat so not sure what I'm supposed to be looking at. Re-installed it, tried 1514 - no joy. Can anyone assist?

Hi, I'm looking at trying to convert an XML file from one of our filers containing this XML file below (top line is different to rest of the xml) into a syslog output: `<Events xmlns="http://www.netapp.com/schemas/ONTAP/2007/AuditLog"> <Event><System><Provider Name="NetApp-Security-Auditing" Guid="{3CB2A168-FE19-4A4E-BDAD-DCF422F13473}"/><EventID>4656</EventID><EventName>Open Object</EventName><Version>101.3</Version><Source>CIFS</Source><Level>0</Level><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><Result>Audit Success</Result><TimeCreated SystemTime="2017-12-15T10:34:51.979061000Z"/><Correlation/><Channel>Security</Channel><Computer>server</Computer><ComputerUUID>cf380853-6606-11e6-9638-00a098a5e1db/2fe0edc3-723f-11e7-ab83-00a098a627d4</ComputerUUID><Security/></System><EventData><Data Name="SubjectIP" IPVersion="4">192.168.0.24</Data><Data Name="SubjectUnix" Uid="65534" Gid="65534" Local="false"></Data><Data Name="SubjectUserSid">S-1-5-21-1997283580-3459341067-486214353-122727</Data><Data Name="SubjectUserIsLocal">false</Data><Data Name="SubjectDomainName">Domain</Data><Data Name="SubjectUserName">firstname.lastname</Data><Data Name="ObjectServer">Security</Data><Data Name="ObjectType">Directory</Data><Data Name="HandleID">000000000004cc;00;00000061;2a5f8706</Data><Data Name="ObjectName">(server);/share</Data><Data Name="AccessList">%%4416 %%4423 </Data><Data Name="AccessMask">81</Data><Data Name="DesiredAccess">Read Data; List Directory; Read Attributes; </Data><Data Name="Attributes"></Data></EventData></Event>` Currently I have the following config but I'm not getting anything sent to the syslog server running on the same box (for testing purposes at present): define ROOT C:\Program Files (x86)\nxlog <Extension gelf> Module xm_gelf </Extension> Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension multiline> Module xm_multiline HeaderLine /^<event>/ EndLine /^</event>/ </Extension> <Extension xmlparser> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "C:\\audit.xml" SavePos FALSE ReadFromLast FALSE InputType multiline <Exec> # Discard everything that doesn't seem to be an xml event if $raw_event !~ /^<event>/ drop(); # Parse the xml event parse_xml(); # Rewrite some fields #$EventTime = parsedate($timestamp); #delete($timestamp); #delete($EventReceivedTime); # Convert to JSON to_json(); </Exec> </Input> <Output out> Module om_udp Host 192.168.0.12 Port 2548 </Output> <Route 1> Path in => out </Route> Can anyone point me at where I'm going wrong? Thanks for your help.

Hi I'm newbie to nxlog. I installed NXlog on windows machine and I would like to capture only specified Events. Facility Severity System warning security/auth information user information logaudit information kernel error please help me with the query list that has to be configured in nxlog.conf file in windows. Thank you so much Regards, Pradeep pradeeepramesh87@gmail.com 00917032845100

Hello, I was testing the SockBufSize option in im_udp because I got the following error and had to reboot the service: "Module inUDP couldn't read from socket; A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself." I though it would change something but I found that with or without SockBufSize, my message size limit is ~64K. I was not able to find confirmation but I suppose this is the max and can't be changed even with SockBufSize. Is it right? (for the tests I set SockBufSize to 150000000 as it is suggested in the documentation) Module im_udp Host localhost Port 514 SockBufSize 150000000 # tested with and without this line I found this interesting post https://nxlog.co/question/2757/execasync-cant-run-powershell-script and will probably apply this solution but I wanted to know if I could increased the size anyway. Thank you.

Hi, New to this community . I use nxlog community edition. My collegue sends from the source side (nxlog) hundreds of msgs in UDP GELF format to graylog syslog utility . Half of them are accepted, the other half get rejected with error "short_message" field is empty. I trieded tcpdump , but nothing visible can be seen. Is there a way that nxlog can berecofigured, so that it will send msgs in more readable format, so I can decide if it is OK that those msgs are rejected. It can be even sent sent to TCP , Most important config details in nxlog: Module xm_gelf ShortMessageLength -1 Module im_file File "C:\DNSLog\DNSDebug.txt" SavePos TRUE InputType LineBased Module om_udp Host x.x.x.x Port yyyy OutputType GELF <Route 2> Path dns => out </Route> Module im_msvistalog Exec if not ($Severity == 'ERROR' or $Severity == 'CRITICAL' or $EventID IN (624, 630, 631, 634, 635, 638, 658, 662, 4624, 4625, 4720, 4726, 4727, 4728, 4729, 4730, 4731, 4732, 4733, 4734, 4735, 4737, 4740, 4741, 4742, 4743, 4754, 4755, 4756, 4757, 4758, 4764, 4767)) drop(); Exec if ($EventID == 4769) drop(); Module om_udp Host x.x.x.x Port yyyz OutputType GELF <Route 1> Path in => out2 </Route> Thanks in advance.

I am trying to read in logs stored in a flat file from an application and the output is adding a space between every characterI've change my patch to the local windows firewall log and I do not get this problem but I can see nothing strange with the source file define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input Lenel> Module im_file File 'C:\Program Files (x86)\program\logs\Dataexchange.log' SavePos TRUE Recursive TRUE Exec $Message = $raw_event; Exec $Hostname = hostname_fqdn(); </input> <Output local> Module om_file File 'c:\_nxlog.txt' </Output> <Route test> Path test => local </Route>

Hi, when configuring nxlog-CE on a Server 2016, there are limitations for collecting all eventlog sources. After starting the nxlog service, I see the following information in the nxlog-logfile: 2017-12-12 18:18:38 INFO nxlog-ce-2.9.1716 started 2017-12-12 18:18:50 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources. here is my nxlog-configuration: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> <Input in> Module im_msvistalog Exec if ($EventType == 'VERBOSE') OR ($EventType == 'INFO') OR ($EventType == 'AUDIT_SUCCESS') drop(); Exec if ($SourceName == 'Microsoft-Windows-KnownFolders' AND $EventID == 1002) drop(); </Input> <Output out> Module om_udp OutputType GELF Host our.graylog.server Port 1515 </Output> <Route 1> Path in => out </Route> We use the same configuration on our Windows Server 2012 / 2012 R2 systems without any issues. Will there be a fix in the a new edition? We don't want to filter the eventlog sources in the configuration. Kind regards, Markus

I'm trying to get the DNS logging going with the im_etw input module with no luck. I get this error on my log, ERROR Failed to load module from C:\Program Files (x86)\nxlog\modules\input\im_etw.dll, The specified module could not be found. ; The specified module could not be found. I have a enterprise version of nxlog running. Not sure how to install that module. thx

Hi, as described in the user manual, I am trying to use a PowerShell script to dynamically get the IIS Log path. The problem is that the include_stdout directive is not being recognized as a valid one. This is my input module Module im_file include_stdout %ROOT%\get_iis_log_paths.cmd if $raw_event =~ /^#/ drop(); else { w3c_parser->parse_csv(); $EventTime = parsedate($date + "T" + $time + ".000Z"); } In the nxlog.log file I see the following error message 2017-12-06 13:27:02 ERROR invalid keyword: include_stdout at C:\Program Files (x86)\nxlog\conf\nxlog.conf:62 2017-12-06 13:27:02 ERROR module 'iis_w3c' has configuration errors, not adding to route 'IIS_Site1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:107 Any help would be appreciated

hi, When I tried to use PollInterval parameter in the im_file module, I see updates in my output file is more often then I use in this parameter. Why that? This parameter is not working?

Hi, I tried to use pm_blocker module. My configuration looks like: <Processor buffer> Module pm_buffer # 100Mb disk buffer MaxSize 102400 Type disk </Processor> <Processor blocker> Module pm_blocker <Schedule> Every 5 min First 2017-11-27 13:12:20 Exec blocker->block(TRUE); </Schedule> <Schedule> Every 5 min First 2017-11-27 13:12:00 Exec blocker->block(FALSE); </Schedule> </Processor> <Input in> Module im_batchcompress ListenAddr 0.0.0.0 Port 1514 </Input> <Output out> Module om_file File 'C:\Temp\NXLog\ + $Hostname + '' + $FileName Exec if $FileName =~ s/-/./g; CreateDir TRUE </Output> <Output out2> Module om_file File 'C:\Temp\NXLog2' + $Hostname + '' + $FileName Exec if $FileName =~ s/-/./g; CreateDir TRUE </Output> <Route 1> Path in => out </Route> <Route 2> Path in => buffer => blocker => out2 </Route> If we can see, pm_blocker used only in second Route. But if we run nxlog with this configuration, we can see block and in first Route. Why? I dont understand. How I can use update our files periodically in Route2??

Hi, Deu to we have multiple collector of GrayLog in multiple locations. I was thinking could I use if else to send log? for example: <Output out_wineventlog> Module om_udp EXEC if $location =~ /^(us)/\ {\ $collector = 'collector.test.us';\ }\ else\ {\ $collector = 'collector.test.eu';\ } Host $collector Port 15001 OutputType GELF </Output> I have tried many statement, but all failure. E.g. string($collector), "$collector", {$collector}, (EXEC $collector;)..etc. I always got the following error. ERROR apr_sockaddr_info failed for [$GLogCollector]:15001; No such host is known. If I config the 'collector.test.us' for Host of output, I can see the $collector is working.

Hello, we have compiled latest NXLOG Community Edition on AIX V.7.1 with GCC 4.8.xx. Actually we have one issue with "im_file" and logfiles with wildcards like "*". NXLOG quits after writing an "core dump" .... Follwoing Output we are receiving in "DEBUG" mode: 017-11-28 12:13:10 DEBUG pidfile /usr/local/var/run/nxlog/nxlog.pid created 2017-11-28 12:13:10 DEBUG parsing path: itm6_custom_log => out_file 2017-11-28 12:13:10 DEBUG adding module itm6_custom_log to route 1 2017-11-28 12:13:10 DEBUG adding module out_file to route 1 2017-11-28 12:13:10 DEBUG jobgroup created with priority 99 2017-11-28 12:13:10 DEBUG jobgroup created with priority 10 2017-11-28 12:13:10 DEBUG spawning 4 worker threads 2017-11-28 12:13:10 DEBUG worker thread 0 started 2017-11-28 12:13:10 DEBUG worker thread 1 started 2017-11-28 12:13:10 DEBUG worker thread 2 started 2017-11-28 12:13:10 DEBUG worker thread 3 started 2017-11-28 12:13:10 DEBUG event thread started 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (_syslog) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (json) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (was_sys_multi) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (itm6_custom_log) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 WARNING not starting unused module out 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (out_file) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 INFO nxlog-ce-2.8.1248 started 2017-11-28 12:13:10 DEBUG no events or no future events, event thread sleeping in condwait 2017-11-28 12:13:10 DEBUG worker 3 processing event 0x301763f8 2017-11-28 12:13:10 DEBUG PROCESS_EVENT: MODULE_START (itm6_custom_log) 2017-11-28 12:13:10 DEBUG START: itm6_custom_log 2017-11-28 12:13:10 DEBUG Value specified for File parameter contains wildcards: '/usr/app/sw/log/itm6*.log' 2017-11-28 12:13:10 DEBUG reading directory entries under '/usr/app/sw/log' to check for matching files Segmentation fault (core dumped) Somone has the same failure or could help us soliving this issue ??? Greets Alaettin from Stuttgart/Germany

I currently have NXlog community version installed on Windows 2012 R2 server. SEIM Manager is requesting that I stop sending Windows Security Event ID 5156 traffic from server. Is this possible. Thank you.

Hi, I tried schedule 2 jobs for blocking log-messages, as is described in documentation: Example 4.6. Two scheduled jobs in the context of the im_tcp module But I need change blocking mode every minute. For example: every even minut block messages, and every odd minutes pass all messages. I tried to use default syntax from cron: <Processor blocker> Module pm_blocker <Schedule> When 0-59/2 * * * * Exec blocker->block(TRUE); Exec log_info("Block: True"); </Schedule> <Schedule> When 1-59/2 * * * * Exec blocker->block(FALSE); Exec log_info("Block: False"); </Schedule> </Processor> But all this schedules was run simultaneously. How I can schedule this jobs?

It seems I have a problem with Nxlog-ce and Windows eventlog after power resume/reconnect to the network. On the high level we won't get any logs from a a machine before we restart the nxlog service. It shows as runnig but sends no logs. As soon as you restart it, the logs are sent. I Enabled debug logging and got the following 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 26 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG executing statements 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:3 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:4 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:5 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:6 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:7 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:8 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:9 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:10 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:11 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:12 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 27 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 ERROR Exception was caused by "apr_sockaddr_info_get(&sa, omconf->host, APR_INET, omconf->port, 0, pool)" at om_udp.c:279/om_udp_connect(); [om_udp.c:279/om_udp_connect()] apr_sockaddr_info failed for Myhost.mydomain.XX:12235; Det begärda namnet är giltigt men data för den begärda typen kunde inte hittas. 2017-11-27 08:02:40 DEBUG worker 2 processing event 0x27a5078 2017-11-27 08:02:40 DEBUG PROCESS_EVENT: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG om_udp_write 2017-11-27 08:02:40 DEBUG module eventlogOUT is not running, not reading any more data 2017-11-27 08:02:40 DEBUG worker 2 waiting for new event 2017-11-27 08:02:40 DEBUG executing statements my NXlog.conf looks like this Nxlog.conf Created: 10/12/2017 15:21:54 LogLevel DEBUG define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> Include plug-in directory include %ROOT%\conf\add-on\*.conf and I have an include file for the eventlog that looks like this <Input eventlogIN> Module im_msvistalog </Input> <Output eventlogOUT> Module om_udp Host myhost.mydomain.xx Port 12235 OutputType GELF </Output> <Route eventlog> Path eventlogIN => eventlogOUT </Route> Has anyone seen this before or got some ideas?