I'm trying to create a stat which gives a RATE for the last 5 minutes whenever you run get_stat. The code example below runs within a im_msvistalog module Each of the stats are updates using add_stat('NAME', 1, $EventReceivedTime) in the main Exec block. The first schedule (re)creates the stats each hour - see below The second schedule outputs the stats each minute - see below <Schedule> Every 1 hour <Exec> create_stat('ep1m', 'RATE', 60, now(), 3600); # Change of count of events in the last 1 MINUTE create_stat('ep5m', 'RATE', 300, now(), 3600); # Change of count (i.e. rate) of events in the last 5 MINUTES </Exec> </Schedule> <Schedule> Every 1 min <Exec> log_info(get_stat('ep1m') + ' events collected from Windows Security Event Log in the last 1 minutes'); log_info(get_stat('ep5m') + ' events collected from Windows Security Event Log in the last 5 minutes'); </Exec> </Schedule> The stat that gives me a rate per minute is working fine. The stat that gives me a rate per 5 minutes is not working as expected. create_stat('ep5m', 'RATE', 300, now(), 3600); My expectation was that each minute, when I write the log, it would give me the RATE (i.e. change in count) for the last 300 seconds. I expected this value to go up and down each minute when I write the log info output. Instead it seems to be grouping the statistics into fixed 5 minute windows from the creation of the counter. That is, a RATE for the first 0-5 mins, then a new RATE for the next 5-10 mins, etc. Is there a way to ensure that when I query the statistics for the last 5 minutes it always gives me the interval between now() and 300 seconds ago?

Hello, The question is whether there is encryption of the logs in community edition? And can I deploy nxlog community edition with custom config via group policy? Thanks in advance!

Im looking for ways to reload nxlog config globally using the <schedule> option without having to restart the service or use the 'Update and Reload' button in nxlog manager. Is there a certain paramerter that can be used in the conf file that will do this on a specific cadence?

Hi, we use ODBC (im_odbc) to read some data from Oracle database and write them on a file. The polling interval is 15 mins. We noticed that when database service restarts, ODBC (im_odbc) do not restart automatically the connections. The only way I found to solve this problem is to stop and start nxlog service. Is there any parameter that force the nxlog to reconnect when existing connection is lost? Thanks

Hi, I'd like to log my Microsoft Defender Events (EventID:1116) so when a malware is detected, I get it on my Graylog server. Problem is that with Server 2016/Windows 10, the logs are too many for a simple input (with the 256 limit). So I decided to filter some, and to only get some of them : <Input in> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security"></Select> <Select Path="Windows PowerShell"></Select> <Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select> </Query> </QueryList> I receive my App, Sys, Security, and Powershell in Graylog but not my Windows Defender events. I tried to generate logs multiple times, with some EICAR files, the logs appear in the Event Viewer, but nothing appears in my Graylog Server. Any help please ? :) thanks

Hi. I am relatively new to nxlog and I hope the community can help me with my question. I have on a Windows Server an Windows Firewall Log File and following nxlog configuration: <Input in> Module im_file File "C:\Windows\system32\LogFiles\Firewall\pfirewall.log" SavePos TRUE ReadFromLast TRUE parse_syslog_ietf(); </Input> <Output out> Module om_tcp Host 3.125.146.97:514 #Exec to_syslog_ietf(); </Output> <Route 1> Path in => out </Route> So far so good. The logs I receive are now as following: <13>1 2022-07-08T13:44:08+00:00 ec2-3-66-101-32 1 - - - 2022-07-08T15:44:17.208477+02:00 WINFS02 - - - [NXLOG@14506 EventReceivedTime="2022-07-08 15:44:17" SourceModuleName="in" SourceModuleType="im_file"] 2022-07-08 15:44:18 ALLOW TCP X.X.X.X Y.Y.Y.Y 43031 Z 0 - 0 0 0 - - - RECEIVE Can I shorten the message, that I receive just the end of the message? 2022-07-08 15:44:18 ALLOW TCP X.X.X.X Y.Y.Y.Y 43031 Z 0 - 0 0 0 - - - RECEIVE

My question here is: If I want to scan an asset to verify that event logs are getting forwarded via NXLog configuration, where will I find the evidence on the device that is sending the logs to the NXLog collector? Will this be somewhere in the registry, or should I look somewhere else?

We currently have 1 Nagios Log server to record logs and 1 windows server with NXlog installed, which has 2 types of logs, TLIB and SIP, from one folder. There are 16 TLIB logs and only 1 SIP log with around 25 increments of each. Both generate a 51,201kb file with 429780 lines and have a total of 483 files in the log folder. When less logs are produced, the 483 logs are overwritten less often and are recorded to Nagios Log successfully. Both log types are recorded to Nagios Log within 1 second of the time stamp of the log entry. When more logs are produced, the 483 logs are overwritten every few minutes. TLIB logs are recorded to Nagios Log successfully within 1 second of the time stamp of the log entry. However the SIP logs starts to fall behind. Entries recorded to Nagios Log can be upto 2 hours different from the time stamp of the log entry. Does anyone know why one log location would fall behind when the other remains unaffected? Is there anyway to improve the reliability of the SIP logs that fall behind? nxlog.conf See the nxlog reference manual at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html Please set the ROOT to the folder your nxlog was installed into, otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define CERT %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Output out1> Module om_tcp Host xx.xx.xx.xx Port 3515 Exec $tmpmessage = $Message; delete($Message); rename_field(&quot;tmpmessage&quot;,&quot;message&quot;); Exec $raw_event = to_json(); # Uncomment for debug output # Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + &quot;\n&quot;); </Output> <Output out2> Module om_tcp Host xx.xx.xx.xx Port 3515 Exec $tmpmessage = $Message; delete($Message); rename_field(&quot;tmpmessage&quot;,&quot;message&quot;); Exec $raw_event = to_json(); # Uncomment for debug output # Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + &quot;\n&quot;); </Output> <Route 1> Path SIP => out1 </Route> <Route 2> Path TLIB => out2 </Route> <Extension multiline_SIPTLIB> Module xm_multiline HeaderLine /^@?\d\d:\d\d:\d\d./ </Extension> <Input TLIB> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001_TLIB-0*' SavePos TRUE Exec $Message = $raw_event; </Input> <Input SIP> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001.*' SavePos TRUE Exec $Message = $raw_event; </Input>

I'm receiving this error when sending Palo Alto logs to my NXLog v5 environment. On Palo Alto side, I have selected CEF format, and the Delimiter field is set to Space:" " (that's my only option) 2022-07-01 18:18:48 ERROR [im_ssl|ssl] binary header not found at position 0 in data received from logforwarding.us.cdl.paloaltonetworks.com (34.67.106.77), is input really binary? Any idea's?? #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files\nxlog define CERTDIR C:\Program Files\nxlog\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension cef> Module xm_cef </Extension> <Extension syslog> Module xm_syslog </Extension> <Input ssl> Module im_ssl ListenAddr 0.0.0.0:16514 CAFile %CERTDIR%/datalake.cert CertFile %CERTDIR%/plzwork.crt CertKeyFile %CERTDIR%/plzwork.key KeyPass secret InputType Binary </Input> <Input udp_input> Module im_udp ListenAddr 0.0.0.0:16514 Exec parse_syslog(); parse_cef($Message); </Input> <Output udp_output> Module om_udp Host 127.0.0.1 Port 16515 Exec $Message = to_cef(); to_syslog_bsd(); </Output> <Route 1> Path ssl => udp_input => udp_output </Route> Thank you!!

Hi Team, I have a single log source that is pumping around 40K EPS, which our NX server is unable to handle, my question is how do I increase the log ingestion capacity. Current setup on an AWS VM: Ubuntu 20.04 LTS 8 CPU, 32GB Ram, 32gb SSD As per my understanding we needed to increase the number of routes tied to the input, as well as the average event size and batch sizes, hence edited the nxlog.con file with following 1 input, 8 routes, 2048 byte average event size, 25000 event batch size. Even with these settings, we are not processing more then 6k EPS. Can anyone advice, what else we can do, please? Note: filtering of events at the source is not an option.

Hi, The log folder is /opt/nxlog-manager/log 2 files will be there. nxlog-manager.log and nxlog-manager.err How do I control the housekeeping of these 2 files ? Please kindly advise. Thanks !

Hello I'm trying to use xm_netflow in NXLog EE. My configuration: <Extension netflow> Module xm_netflow </Extension> <Extension json> Module xm_json </Extension> <Input in_10533_netflow_udp> Module im_udp Host 0.0.0.0 Port 10533 InputType netflow </Input> <Route route_10533_netflow> Path in_10533_netflow_udp => out_file </Route> <Output out_file> Module om_file File "/opt/nxlog/var/log/out.log" Exec to_json(); </Output> But I get an error when I try run nxlog: Jun 24 12:27:50 xxx.evil.corp nxlog[3734]: 2022-06-24 12:27:50 ERROR [CORE|main] Invalid InputType 'netflow' at netflow.conf What i do wrong? RPMs: rpm -qa | grep nxlog nxlog-kafka-5.3.6735-1.el7_9.x86_64 nxlog-python-5.3.6735-1.el7_9.x86_64 nxlog-5.3.6735-1.el7_9.x86_64

We tested nxlog ce a few year back but did not end up using it in our company. We recently ran a vulnerability scan of the servers we had this installed on and they reported that nxlog was still installed. We can confirm that nxlog isn't visible in add/remove programs and the nxlog service isn't running. There is still a nxlog folder in c:\program files (x86). It only contains a data folder with two files in it. I'm assuming its registry entries are whats getting detected. Is there any documentation on what registry entries I should remove to completely remove this software?

After the upgrade from 5.5 to 5.6, the web UI is unable to load, jetty service unavailable. Not sure why this is happening.

I have installed NXlog community free edition on my WEC. How do I forward logs from NXlog to WEC?

HELLO hello sir or madam i have a question about NXlog. when I got NXLOG from my central logs i run into problem like this error that i sent on attachment i look forward to your answering. i in advance say thank you from your favor. ERROR data size (67324) is over the limit (65000), will be truncated y

When installing nxlog- ce, I get problems with dependency for the package. sudo apt install ./nxlog-ce_3.0.2272_ubuntu_focal_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'nxlog-ce' instead of './nxlog-ce_3.0.2272_ubuntu_focal_amd64.deb' Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies. nxlog-ce : Depends: libperl5.30 (>= 5.30.0) but it is not installable Depends: libpython3.8 (>= 3.8.2) but it is not installable E: Unable to correct problems, you have held broken packages. I can do a lot of trickery but was wondering if you will create a new upgrade soon? Pinning Python3.8 is a bit tricky and so is libperl5.30 Regards, Mats Tage Axelsson

HI guys after upgrading to the latest CE edition i noticed that the nxlog.log file wasn't being updated. This is the beginning of my config: **define MYLOGFILE /home/nxlog_ce/nxlog.log** <Extension _syslog> Module xm_syslog </Extension> **LogLevel INFO** **LogFile /home/nxlog_ce/nxlog.log** I went ahead and added the lines marked with ** and still no logs. can anyone help me?

Hi, I installed recently the last version of NXLOG-CE (3.0.2284) on my windows server 2016 Standard. I noticed that the configcache is not functionnal in my case because of an erreor when nxlog service stop => "System Error 109 has occurred. The pipe has ended" My configcache.dat file is not updated and when I restart the service, a lot of old logs are sended to my logstash Here's my nxlog conf file Panic Soft define ROOT C:\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% LogLevel INFO Moduledir %ROOT%\Modules Pidfile %ROOT%\nxlog.pid SpoolDir %ROOT%\SpoolDir CacheDir %ROOT%\CacheDir <Extension _syslog> Module xm_gelf </Extension> <Extension json> Module xm_json </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> Collecting event log <Input eventlog> Module im_msvistalog ReadFromLast TRUE SavePos TRUE Exec to_json(); </Input> <Output ssl> Module om_ssl Host XX.XX.XX.XX CertFile %CERTDIR%\MyCertFile.crt Port XXXX AllowUntrusted TRUE </Output> <Route 1> Path eventlog => ssl </Route> Thank's a lot for your help

Hi, I have to forward tsmlogs to the server, I have below configuration . My audit team needs hostname printed when they receive the logs on their end. I have below configuration <Output tsmout> Module om_tcp Host 10.24.8.23 Port 30133 Exec $FQDN = hostname_fqdn(); Exec parse_syslog_bsd(); </Output> <Input tsmlogs> Module im_file File "/scripts/tsm/log/tsm.log" #File "/scripts/tsm/log/" + $FQDN + "-" + "tsm.log" Exec $Hostname = hostname_fqdn(); </Input> <Route tsmroute> Path tsmlogs => tsmout </Route> How do I add hostname when they get forwarded.