Hello, what is best way to merge information from two events to a new one.

I have one evenet with connectioninformation and a second event with the userid. And I need the user ID addtionalt to the first event with the connection information forwarded in a syslog stream. There is a connectio ID in the event that I can use as filter.

Problem is, that there are some more events too with the same connection ID.

I was trying to change the global DateFormat as stated in the docs to be able to have the milliseconds included in the output after parsing json, but when i start nxlog i get the message "Invalid keyword: DateFormat at /etc/nxlog.conf"

########################################
# Global directives                    #
########################################
User nxlog
Group nxlog

LogFile /var/log/nxlog/nxlog.log
LogLevel INFO

DateFormat "YYYY-MM-DDThh:mm:ss.sUTC"

The version of nxlog is 2.9.1716 (nxlog-ce)

Am i doing something wrong? Or is this function not supported in the community edition?

Thanks,

Roman

Hi All,

Is it possible to use Exec command to add username to logs? If so, can someone point me to resource on how to do it?

Can the same be used to add current assigned IP address?

Regards

Jake

Hello folks, since weeks i am trying to get filtered informations from a domain controller but i dont get the right informations. If i choose the EVENT IDs i want to get, there comes no input on the graylog side but if i select * from Application, Security or System., all the messages are coming. but i dont want that. i only want add,modify,delete account for example. How do i have to do that? Here is one of my spectacular config files with filters:

https://pastebin.com/cptCmt9e

and thats the simple working one

https://pastebin.com/aXt5waFT

wiht the community eddition when the nxlog-ce is listening on /dev/log and for some reasons the systemd-journald (debian 9) removes the socket the nxlog blocks the compleate host. (even no login possible) after a restart of nxlog the host recovers.

We are running into errors running nxlog-ce on Windows 2016. When is this going to be supported? Our only alternative is to move to Beats which will happen quickly if there is no ETA on this.

Here are some examples of the errors we see:
2018-01-11 14:34:22 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 512 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs.
2018-01-11 14:34:22 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Windows PowerShell log, will try to reopen in 512 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs.
```=

Hi, It seems that nxlog does not send to the output all lines of a file which is monitored. Here is an example of what I want to send to my syslog server : ``` 30,01/16/18,09:24:23,Requête de mise à jour DNS,192.168.31.66,volant2.enterprise.local,,,0,6,,, 10,01/16/18,09:24:23,Assigner,192.168.31.66,volant2.enterprise.local,F01FAF2F23D7,,2412417530,0,,, 32,01/16/18,09:24:23,Mise à jour DNS réussie,192.168.31.66,volant2.enterprise.local,,,0,6,,, 30,01/16/18,09:25:55,Requête de mise à jour DNS,192.168.31.68,volant3.enterprise.local,,,0,6,,, 10,01/16/18,09:25:55,Assigner,192.168.31.68,volant3.enterprise.local,5C514FDCA690,,2181532597,0,,, 32,01/16/18,09:25:55,Mise à jour DNS réussie,192.168.31.68,volant3.enterprise.local,,,0,6,,, ``` And here is what I have received : ``` 2018-01-16T09:24:23+01:00 DC 30,01/16/18,09: 24:23,Requ▒te de mise ▒ jour DNS,192.168.31.66,volant2.enterprise.local,,,0,6,,, 2018-01-16T09:24:23+01:00 DC 10,01/16/18,09: 24:23,Assigner,192.168.31.66,volant2.enterprise.local,F01FAF2F23D7,,2412417530,0,,, 2018-01-16T09:24:23+01:00 DC 32,01/16/18,09: 24:23,Mise ▒ jour DNS r▒ussie,192.168.31.66,volant2.enterprise.local,,,0,6,,, 2018-01-16T09:25:55+01:00 DC 32,01/16/18,09: 25:55,Mise ▒ jour DNS r▒ussie,192.168.31.68,volant3.enterprise.local,,,0,6,,, ``` Here is my whole nxlog configuration ``` define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_syslog Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog define DHCPDIR C:\Windows\Sysnative\dhcp Module im_file File '%DHCPDIR%\DhcpSrvLog-*.log' SavePos TRUE ReadFromLast TRUE PollInterval 1 Exec $Message = $raw_event; $SyslogFacilityValue = 17; Module om_udp Host 192.168.2.12 Port 514 Exec to_syslog_bsd(); Path inDhcp => outSyslogSrv ``` Did I miss something ? Thanks

Simple Question

Is there any way to configure NXlog to send data i Gelf HTTP format. I need to pass data through a HTTP only proxy

Hi, I have a problem with nxlog. Try to start service nxlog with kafka configuration (including installation librdkafka) and unfortunately i have an error with starting nxlog:

error: "Unit nxlog.service has begun starting up.

Jan 03 17:24:12 Kafka4 nxlog[19220]: 2018-01-03 17:24:12 ERROR Failed to load module from /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so, /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so: undefined symbol: rd_kafka_last_error;DSO load failed Jan 03 17:24:12 Kafka4 systemd[1]: nxlog.service: control process exited, code=exited status=1 Jan 03 17:24:12 Kafka4 systemd[1]: Failed to start NXLog daemon. -- Subject: Unit nxlog.service has failed -- Defined-By: systemd"

hello, I am testing the NXlog EE, but the module xm_w3c does not work, do not parse the logs of BRO, you can help me. Module xm_w3c Delimiter , Module im_file File "/mnt/*.log" InputType w3c Module om_ssl Host 192.168.0.38 Port 10525 CAFile /data/conf/ca.crt AllowUntrusted TRUE Path i.bro.log => o.bro.log # ./nxlog-processor 2017-12-27 20:38:33 INFO connecting to 192.168.0.38:10525 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 15 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 10 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE 2017-12-27 20:39:47 ERROR cannot parse integer "SUCCESS", invalid modifier: 'S' 2017-12-27 20:39:47 ERROR last message repeated 34 times 2017-12-27 20:39:47 ERROR couldn't parse integer: LOGON_FAILURE Bro Files ![Bro files][Bro files] [Graylog2]: https://image.ibb.co/fCCGHG/Screenshot_20171227_225122.png ![Graylog2] [Graylog2]

HI all

Is the module available for community version of nxlog and if yes how do we download? Thanks all for your time.

Chew

Hello I have a question about Suppressed in pm_evcorr. Having following example from official documentation: Module im_file File "/tmp/testfile" SavePos FALSE ReadFromLast FALSE Exec if ($raw_event =~ /^(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d) (.+)/) { \ $EventTime = parsedate($1); \ $Message = $2; \ $raw_event = $Message; \ } Module im_internal Exec $raw_event = $Message; Exec $EventTime = 2010-01-01 00:01:00; Module om_file File '/tmp/output' Module pm_evcorr TimeField EventTime # match input event and execute an action list, but ignore the following # matching events for the next t seconds. Condition $Message =~ /^suppressed/ Interval 30 Exec $raw_event = "suppressing.."; Exec if $Message =~ /^simple/ $raw_event = "got simple"; Path in, internal => evcorr => out Wrote following logs into the file: [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:44 suppressed" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:47 simple" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:06:49 simple" >> testfile [root@server:[DEV] /tmp]# echo "2017-12-01 13:07:00 suppressed" >> testfile In output I got: suppressing.. got simple got simple suppressed Suppressing condition worked. But I thought that will stop processing all subsequent log entries. And it's not. Why the Simple condition is still matched?

Hello

Is it possible to monitor the log file modification date? I do not want to check log file contents, to check whether pattern was found or not. The only thing which I want to is to get modification date of a log file, and if it's older than X minutes -> generate an event.

I tried different configs, with schedule, with im_null modules, exec, file_mtime function... And nothing... Still doesn't work.

Good morning,

Noob to nxlog - installed in Windows and running on an OOB config, Host is set to localhost. All I get in the logs is:

2017-12-21 11:31:44 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it.
2017-12-21 11:32:16 INFO connecting to Localhost:514 2017-12-21 11:32:17 INFO reconnecting in 64 seconds 2017-12-21 11:32:17 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it.
2017-12-21 11:33:21 INFO connecting to Localhost:514 2017-12-21 11:33:22 INFO reconnecting in 128 seconds 2017-12-21 11:33:22 ERROR couldn't connect to tcp socket on Localhost:514; No connection could be made because the target machine actively refused it.

The firewall is disabled, 514 not showing as listening in netstat so not sure what I'm supposed to be looking at. Re-installed it, tried 1514 - no joy.

Can anyone assist?

Hi, I'm looking at trying to convert an XML file from one of our filers containing this XML file below (top line is different to rest of the xml) into a syslog output: ` 4656Open Object101.3CIFS000x8020000000000000Audit SuccessSecurityservercf380853-6606-11e6-9638-00a098a5e1db/2fe0edc3-723f-11e7-ab83-00a098a627d4192.168.0.24S-1-5-21-1997283580-3459341067-486214353-122727falseDomainfirstname.lastnameSecurityDirectory000000000004cc;00;00000061;2a5f8706(server);/share%%4416 %%4423 81Read Data; List Directory; Read Attributes; ` Currently I have the following config but I'm not getting anything sent to the syslog server running on the same box (for testing purposes at present): define ROOT C:\Program Files (x86)\nxlog Module xm_gelf Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_multiline HeaderLine /^/ EndLine /^/ Module xm_xml Module xm_json Module im_file File "C:\\audit.xml" SavePos FALSE ReadFromLast FALSE InputType multiline # Discard everything that doesn't seem to be an xml event if $raw_event !~ /^/ drop(); # Parse the xml event parse_xml(); # Rewrite some fields #$EventTime = parsedate($timestamp); #delete($timestamp); #delete($EventReceivedTime); # Convert to JSON to_json(); Module om_udp Host 192.168.0.12 Port 2548 Path in => out Can anyone point me at where I'm going wrong? Thanks for your help.

Hi I'm newbie to nxlog. I installed NXlog on windows machine and I would like to capture only specified Events.

Facility Severity System warning security/auth information user information logaudit information kernel error

please help me with the query list that has to be configured in nxlog.conf file in windows.

Thank you so much

Regards, Pradeep pradeeepramesh87@gmail.com 00917032845100

Hello, I was testing the SockBufSize option in im_udp because I got the following error and had to reboot the service: "Module inUDP couldn't read from socket; A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself." I though it would change something but I found that with or without SockBufSize, my message size limit is ~64K. I was not able to find confirmation but I suppose this is the max and can't be changed even with SockBufSize. Is it right? (for the tests I set SockBufSize to 150000000 as it is suggested in the documentation) Module im_udp Host localhost Port 514 SockBufSize 150000000 # tested with and without this line I found this interesting post https://nxlog.co/question/2757/execasync-cant-run-powershell-script and will probably apply this solution but I wanted to know if I could increased the size anyway. Thank you.

Hi, New to this community . I use nxlog community edition. My collegue sends from the source side (nxlog) hundreds of msgs in UDP GELF format to graylog syslog utility . Half of them are accepted, the other half get rejected with error "short_message" field is empty. I trieded tcpdump , but nothing visible can be seen. Is there a way that nxlog can berecofigured, so that it will send msgs in more readable format, so I can decide if it is OK that those msgs are rejected. It can be even sent sent to TCP , Most important config details in nxlog: Module xm_gelf ShortMessageLength -1 Module im_file File "C:\DNSLog\DNSDebug.txt" SavePos TRUE InputType LineBased Module om_udp Host x.x.x.x Port yyyy OutputType GELF Path dns => out Module im_msvistalog Exec if not ($Severity == 'ERROR' or $Severity == 'CRITICAL' or $EventID IN (624, 630, 631, 634, 635, 638, 658, 662, 4624, 4625, 4720, 4726, 4727, 4728, 4729, 4730, 4731, 4732, 4733, 4734, 4735, 4737, 4740, 4741, 4742, 4743, 4754, 4755, 4756, 4757, 4758, 4764, 4767)) drop(); Exec if ($EventID == 4769) drop(); Module om_udp Host x.x.x.x Port yyyz OutputType GELF Path in => out2 Thanks in advance.

I am trying to read in logs stored in a flat file from an application and the output is adding a space between every characterI've change my patch to the local windows firewall log and I do not get this problem but I can see nothing strange with the source file define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_json Module xm_syslog Module im_file File 'C:\Program Files (x86)\program\logs\Dataexchange.log' SavePos TRUE Recursive TRUE Exec $Message = $raw_event; Exec $Hostname = hostname_fqdn(); Module om_file File 'c:\_nxlog.txt' Path test => local

Hi, when configuring nxlog-CE on a Server 2016, there are limitations for collecting all eventlog sources. After starting the nxlog service, I see the following information in the nxlog-logfile: 2017-12-12 18:18:38 INFO nxlog-ce-2.9.1716 started 2017-12-12 18:18:50 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources. here is my nxlog-configuration: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_gelf Module im_msvistalog Exec if ($EventType == 'VERBOSE') OR ($EventType == 'INFO') OR ($EventType == 'AUDIT_SUCCESS') drop(); Exec if ($SourceName == 'Microsoft-Windows-KnownFolders' AND $EventID == 1002) drop(); Module om_udp OutputType GELF Host our.graylog.server Port 1515 Path in => out We use the same configuration on our Windows Server 2012 / 2012 R2 systems without any issues. Will there be a fix in the a new edition? We don't want to filter the eventlog sources in the configuration. Kind regards, Markus