We currently have 1 Nagios Log server to record logs and 1 windows server with NXlog installed, which has 2 types of logs, TLIB and SIP, from one folder. There are 16 TLIB logs and only 1 SIP log with around 25 increments of each. Both generate a 51,201kb file with 429780 lines and have a total of 483 files in the log folder. When less logs are produced, the 483 logs are overwritten less often and are recorded to Nagios Log successfully. Both log types are recorded to Nagios Log within 1 second of the time stamp of the log entry. When more logs are produced, the 483 logs are overwritten every few minutes. TLIB logs are recorded to Nagios Log successfully within 1 second of the time stamp of the log entry. However the SIP logs starts to fall behind. Entries recorded to Nagios Log can be upto 2 hours different from the time stamp of the log entry. Does anyone know why one log location would fall behind when the other remains unaffected? Is there anyway to improve the reliability of the SIP logs that fall behind? nxlog.conf See the nxlog reference manual at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html Please set the ROOT to the folder your nxlog was installed into, otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define CERT %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Output out1> Module om_tcp Host xx.xx.xx.xx Port 3515 Exec $tmpmessage = $Message; delete($Message); rename_field(&quot;tmpmessage&quot;,&quot;message&quot;); Exec $raw_event = to_json(); # Uncomment for debug output # Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + &quot;\n&quot;); </Output> <Output out2> Module om_tcp Host xx.xx.xx.xx Port 3515 Exec $tmpmessage = $Message; delete($Message); rename_field(&quot;tmpmessage&quot;,&quot;message&quot;); Exec $raw_event = to_json(); # Uncomment for debug output # Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + &quot;\n&quot;); </Output> <Route 1> Path SIP => out1 </Route> <Route 2> Path TLIB => out2 </Route> <Extension multiline_SIPTLIB> Module xm_multiline HeaderLine /^@?\d\d:\d\d:\d\d./ </Extension> <Input TLIB> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001_TLIB-0*' SavePos TRUE Exec $Message = $raw_event; </Input> <Input SIP> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001.*' SavePos TRUE Exec $Message = $raw_event; </Input>

I'm receiving this error when sending Palo Alto logs to my NXLog v5 environment. On Palo Alto side, I have selected CEF format, and the Delimiter field is set to Space:" " (that's my only option) 2022-07-01 18:18:48 ERROR [im_ssl|ssl] binary header not found at position 0 in data received from logforwarding.us.cdl.paloaltonetworks.com (34.67.106.77), is input really binary? Any idea's?? #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files\nxlog define CERTDIR C:\Program Files\nxlog\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension cef> Module xm_cef </Extension> <Extension syslog> Module xm_syslog </Extension> <Input ssl> Module im_ssl ListenAddr 0.0.0.0:16514 CAFile %CERTDIR%/datalake.cert CertFile %CERTDIR%/plzwork.crt CertKeyFile %CERTDIR%/plzwork.key KeyPass secret InputType Binary </Input> <Input udp_input> Module im_udp ListenAddr 0.0.0.0:16514 Exec parse_syslog(); parse_cef($Message); </Input> <Output udp_output> Module om_udp Host 127.0.0.1 Port 16515 Exec $Message = to_cef(); to_syslog_bsd(); </Output> <Route 1> Path ssl => udp_input => udp_output </Route> Thank you!!

Hi Team, I have a single log source that is pumping around 40K EPS, which our NX server is unable to handle, my question is how do I increase the log ingestion capacity. Current setup on an AWS VM: Ubuntu 20.04 LTS 8 CPU, 32GB Ram, 32gb SSD As per my understanding we needed to increase the number of routes tied to the input, as well as the average event size and batch sizes, hence edited the nxlog.con file with following 1 input, 8 routes, 2048 byte average event size, 25000 event batch size. Even with these settings, we are not processing more then 6k EPS. Can anyone advice, what else we can do, please? Note: filtering of events at the source is not an option.

Hi, The log folder is /opt/nxlog-manager/log 2 files will be there. nxlog-manager.log and nxlog-manager.err How do I control the housekeeping of these 2 files ? Please kindly advise. Thanks !

Hello I'm trying to use xm_netflow in NXLog EE. My configuration: <Extension netflow> Module xm_netflow </Extension> <Extension json> Module xm_json </Extension> <Input in_10533_netflow_udp> Module im_udp Host 0.0.0.0 Port 10533 InputType netflow </Input> <Route route_10533_netflow> Path in_10533_netflow_udp => out_file </Route> <Output out_file> Module om_file File "/opt/nxlog/var/log/out.log" Exec to_json(); </Output> But I get an error when I try run nxlog: Jun 24 12:27:50 xxx.evil.corp nxlog[3734]: 2022-06-24 12:27:50 ERROR [CORE|main] Invalid InputType 'netflow' at netflow.conf What i do wrong? RPMs: rpm -qa | grep nxlog nxlog-kafka-5.3.6735-1.el7_9.x86_64 nxlog-python-5.3.6735-1.el7_9.x86_64 nxlog-5.3.6735-1.el7_9.x86_64

We tested nxlog ce a few year back but did not end up using it in our company. We recently ran a vulnerability scan of the servers we had this installed on and they reported that nxlog was still installed. We can confirm that nxlog isn't visible in add/remove programs and the nxlog service isn't running. There is still a nxlog folder in c:\program files (x86). It only contains a data folder with two files in it. I'm assuming its registry entries are whats getting detected. Is there any documentation on what registry entries I should remove to completely remove this software?

After the upgrade from 5.5 to 5.6, the web UI is unable to load, jetty service unavailable. Not sure why this is happening.

I have installed NXlog community free edition on my WEC. How do I forward logs from NXlog to WEC?

HELLO hello sir or madam i have a question about NXlog. when I got NXLOG from my central logs i run into problem like this error that i sent on attachment i look forward to your answering. i in advance say thank you from your favor. ERROR data size (67324) is over the limit (65000), will be truncated y

When installing nxlog- ce, I get problems with dependency for the package. sudo apt install ./nxlog-ce_3.0.2272_ubuntu_focal_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'nxlog-ce' instead of './nxlog-ce_3.0.2272_ubuntu_focal_amd64.deb' Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies. nxlog-ce : Depends: libperl5.30 (>= 5.30.0) but it is not installable Depends: libpython3.8 (>= 3.8.2) but it is not installable E: Unable to correct problems, you have held broken packages. I can do a lot of trickery but was wondering if you will create a new upgrade soon? Pinning Python3.8 is a bit tricky and so is libperl5.30 Regards, Mats Tage Axelsson

HI guys after upgrading to the latest CE edition i noticed that the nxlog.log file wasn't being updated. This is the beginning of my config: **define MYLOGFILE /home/nxlog_ce/nxlog.log** <Extension _syslog> Module xm_syslog </Extension> **LogLevel INFO** **LogFile /home/nxlog_ce/nxlog.log** I went ahead and added the lines marked with ** and still no logs. can anyone help me?

Hi, I installed recently the last version of NXLOG-CE (3.0.2284) on my windows server 2016 Standard. I noticed that the configcache is not functionnal in my case because of an erreor when nxlog service stop => "System Error 109 has occurred. The pipe has ended" My configcache.dat file is not updated and when I restart the service, a lot of old logs are sended to my logstash Here's my nxlog conf file Panic Soft define ROOT C:\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% LogLevel INFO Moduledir %ROOT%\Modules Pidfile %ROOT%\nxlog.pid SpoolDir %ROOT%\SpoolDir CacheDir %ROOT%\CacheDir <Extension _syslog> Module xm_gelf </Extension> <Extension json> Module xm_json </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> Collecting event log <Input eventlog> Module im_msvistalog ReadFromLast TRUE SavePos TRUE Exec to_json(); </Input> <Output ssl> Module om_ssl Host XX.XX.XX.XX CertFile %CERTDIR%\MyCertFile.crt Port XXXX AllowUntrusted TRUE </Output> <Route 1> Path eventlog => ssl </Route> Thank's a lot for your help

Hi, I have to forward tsmlogs to the server, I have below configuration . My audit team needs hostname printed when they receive the logs on their end. I have below configuration <Output tsmout> Module om_tcp Host 10.24.8.23 Port 30133 Exec $FQDN = hostname_fqdn(); Exec parse_syslog_bsd(); </Output> <Input tsmlogs> Module im_file File "/scripts/tsm/log/tsm.log" #File "/scripts/tsm/log/" + $FQDN + "-" + "tsm.log" Exec $Hostname = hostname_fqdn(); </Input> <Route tsmroute> Path tsmlogs => tsmout </Route> How do I add hostname when they get forwarded.

Hi all, NXLOG newbie. I'm trying to inject logs from a specific hostname into a specific folder. For example: if hostname = x then store log in folder = y I'm running a very bare-bones set up. I'd like to use the FQDN as the variable for hostname rather than an IP. Current config: Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf\nxlog.d define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension syslog> Module xm_syslog </Extension> <Extension fileop> Module xm_fileop </Extension> <Input udp> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); </Input> <Output file> Module om_file File "C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log" CreateDir TRUE <Exec> if file_size("C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log") > 10M { file_cycle("C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log",2); reopen(); } </Exec> </Output> <Route 1> Path udp => file </Route> Snare compatible example configuration Collecting event log <Input in> Module im_msvistalog </Input> Converting events to Snare format and sending them out over TCP syslog <Output out> Module om_tcp Host 192.168.1.1 Port 514 Exec to_syslog_snare(); </Output> Connect input 'in' to output 'out' <Route 1> Path in => out </Route>

Hello, I'm trying to send Windows events to two different syslog servers. I have two output modules and one route. I see syslog packets getting sent to both collectors but the packets are not coming in the correct format The configuration looks like LogLevel INFO Logfile %LOGDIR%/nxlog.log <Extension agent_managment> Module xm_soapadmin Connect 10.x.x.n Port 4041 SocketType SSL CAFile %CERTDIR%/agent-ca.pem AllowUntrusted FALSE RequireCert TRUE <ACL conf> Directory %CONFDIR% AllowRead TRUE AllowWrite TRUE </ACL> <ACL cert> Directory %CERTDIR% AllowRead TRUE AllowWrite TRUE </ACL> </Extension> <Extension Extension_json> Module xm_json </Extension> <Input eventlog> Module im_msvistalog SavePos TRUE ReadFromLast TRUE Exec if ($EventID == 5156) drop(); </Input> <Input internal_mod> Module im_internal </Input> <Output out> Module om_tcp Host 10.x.x.a Port 514 OutputType LineBased <Exec> $Hostname = string(host_ip()); to_syslog_ietf(); parse_syslog(); $Message = '@cee: ' + to_json(); to_syslog_bsd(); </Exec> </Output> <Output outlcp> Module om_udp Host 10.x.x.b Port 514 OutputType Dgram Exec $Hostname = string(host_ip()); Exec to_syslog_snare(); </Output> <Route Syslog_lcp> Priority 1 Path eventlog, internal_mod => outlcp, out </Route>

I have an XML that I am trying to strip some data out so it can be pre-processed by software on another machine. XML file is being generated on a windows logging to an XML file. I am at bit of a lost I have tried too many things to list here. What I am trying to do is remove our domain name and our domain email address before it is sent to the machine to be pre-processed. I only want the username. Any records that have host\domain.com don’t need to be sent and I figured out how to drop that data. (number 4). If it helps I am running nxlog-ce-3.0.2284. Here is an example of event: (Removed bunch of xml fields for clarity of this post) 1: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">DOMAIN\username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 2: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 3: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username@domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 4: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">host\domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> Nxlog.conf: #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension xmlparser> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "C:\LogFiles\log*.log" InputType LineBased Exec $Message = $raw_event; SavePos TRUE ReadFromLast TRUE <Exec> Discard everything that doesn't seem to be an xml event if $raw_event !~ /^<Event>/ drop(); if $raw_event =~ /^(.+)host(.+)/ drop(); parse_xml(); Convert to JSON to_json(); </Exec> </Input> <Output out> Module om_udp Host yy.xx.xx.xx Port 514 </Output> <Route 1> Path in => out </Route>

Hey All, A bit of a newbee and trying to get NXLOG working with GrayLog. It is working and I'm seeing the information. The issue is that the information I'm seeing does not seem to match the PC's event logs. Please see below Config File: <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension _gelf> Module xm_gelf </Extension> <Input win> Module im_msvistalog </Input> Converting events to Snare format and sending them out over TCP syslog <Output graylog> Module om_udp Host X.X.X.X Port 3514 OutputType GELF </Output> <Route graylog_route> Path win => graylog </Route> Any ideas about what I'm doing wrong????

HTTP ERROR: 503 Problem accessing /nxlog-manager. Reason: Service Unavailable New install. Not sure why this is fighting me :)

I am betting I am simply missing a step. Anyone able to help? Unpacking nxlog-manager (5.6.5633) ... dpkg-deb (subprocess): decompressing archive member: lzma error: compressed data is corrupt dpkg-deb: error: <decompress> subprocess returned error exit status 2 dpkg: error processing archive nxlog-manager-5.6.5633-jdk1.7.1.deb (--install): cannot copy extracted data for './opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/spring-js-resources-2.4.2.RELEASE.jar' to '/opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/spring-js-resources-2.4.2.RELEASE.jar.dpkg-new': unexpected end of file or stream Processing triggers for systemd (245.4-4ubuntu3.17) ... Errors were encountered while processing: nxlog-manager-5.6.5633-jdk1.7.1.deb

Dears, I'm running nxlog-ce 3.0.2272 under Windows Server 2022 to search the Application event log for specific events using a custom query in order to forward them as GELF messages to some Graylog server. The connection is secured by SSL. With Server 2019, everything runs smoothly but with Server 2022, nxlog.exe keeps crashing after a few events have been collected and sent to Graylog. Worst fact is, that events are omitted and not transferred to Graylog. The related event log entries (event id 1000): Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x00000000 Faulting module name: ntdll.dll, version: 10.0.20348.681, time stamp: 0x69d3cd31 Exception code: 0xc0000374 Fault offset: 0x0000000000103ad9 Faulting process id: 0x2b1c Faulting application start time: 0x01d86901d76501a6 Faulting application path: C:\Program Files (x86)\nxlog\nxlog.exe Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll Report Id: 362167c9-9922-4158-8d56-ee4bafd21e67 Faulting package full name: Faulting package-relative application ID: Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x00000000 Faulting module name: RPCRT4.dll, version: 10.0.20348.707, time stamp: 0xd31f9dd8 Exception code: 0xc0000005 Fault offset: 0x00000000000272e3 Faulting process id: 0x1a24 Faulting application start time: 0x01d868da69310cd8 Faulting application path: C:\Program Files (x86)\nxlog\nxlog.exe Faulting module path: C:\WINDOWS\System32\RPCRT4.dll Report Id: 96a6d244-74ca-4f6f-8667-8bb5082a452a Faulting package full name: Faulting package-relative application ID: Any idea? Thanks ahead, Elix