HI guys after upgrading to the latest CE edition i noticed that the nxlog.log file wasn't being updated. This is the beginning of my config: **define MYLOGFILE /home/nxlog_ce/nxlog.log** <Extension _syslog> Module xm_syslog </Extension> **LogLevel INFO** **LogFile /home/nxlog_ce/nxlog.log** I went ahead and added the lines marked with ** and still no logs. can anyone help me?

Hi, I installed recently the last version of NXLOG-CE (3.0.2284) on my windows server 2016 Standard. I noticed that the configcache is not functionnal in my case because of an erreor when nxlog service stop => "System Error 109 has occurred. The pipe has ended" My configcache.dat file is not updated and when I restart the service, a lot of old logs are sended to my logstash Here's my nxlog conf file Panic Soft define ROOT C:\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% LogLevel INFO Moduledir %ROOT%\Modules Pidfile %ROOT%\nxlog.pid SpoolDir %ROOT%\SpoolDir CacheDir %ROOT%\CacheDir <Extension _syslog> Module xm_gelf </Extension> <Extension json> Module xm_json </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> Collecting event log <Input eventlog> Module im_msvistalog ReadFromLast TRUE SavePos TRUE Exec to_json(); </Input> <Output ssl> Module om_ssl Host XX.XX.XX.XX CertFile %CERTDIR%\MyCertFile.crt Port XXXX AllowUntrusted TRUE </Output> <Route 1> Path eventlog => ssl </Route> Thank's a lot for your help

Hi, I have to forward tsmlogs to the server, I have below configuration . My audit team needs hostname printed when they receive the logs on their end. I have below configuration <Output tsmout> Module om_tcp Host 10.24.8.23 Port 30133 Exec $FQDN = hostname_fqdn(); Exec parse_syslog_bsd(); </Output> <Input tsmlogs> Module im_file File "/scripts/tsm/log/tsm.log" #File "/scripts/tsm/log/" + $FQDN + "-" + "tsm.log" Exec $Hostname = hostname_fqdn(); </Input> <Route tsmroute> Path tsmlogs => tsmout </Route> How do I add hostname when they get forwarded.

Hi all, NXLOG newbie. I'm trying to inject logs from a specific hostname into a specific folder. For example: if hostname = x then store log in folder = y I'm running a very bare-bones set up. I'd like to use the FQDN as the variable for hostname rather than an IP. Current config: Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf\nxlog.d define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension syslog> Module xm_syslog </Extension> <Extension fileop> Module xm_fileop </Extension> <Input udp> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); </Input> <Output file> Module om_file File "C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log" CreateDir TRUE <Exec> if file_size("C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log") > 10M { file_cycle("C:/Syslogs/nxlog/" + $HostName + "/" + $HostName + ".log",2); reopen(); } </Exec> </Output> <Route 1> Path udp => file </Route> Snare compatible example configuration Collecting event log <Input in> Module im_msvistalog </Input> Converting events to Snare format and sending them out over TCP syslog <Output out> Module om_tcp Host 192.168.1.1 Port 514 Exec to_syslog_snare(); </Output> Connect input 'in' to output 'out' <Route 1> Path in => out </Route>

Hello, I'm trying to send Windows events to two different syslog servers. I have two output modules and one route. I see syslog packets getting sent to both collectors but the packets are not coming in the correct format The configuration looks like LogLevel INFO Logfile %LOGDIR%/nxlog.log <Extension agent_managment> Module xm_soapadmin Connect 10.x.x.n Port 4041 SocketType SSL CAFile %CERTDIR%/agent-ca.pem AllowUntrusted FALSE RequireCert TRUE <ACL conf> Directory %CONFDIR% AllowRead TRUE AllowWrite TRUE </ACL> <ACL cert> Directory %CERTDIR% AllowRead TRUE AllowWrite TRUE </ACL> </Extension> <Extension Extension_json> Module xm_json </Extension> <Input eventlog> Module im_msvistalog SavePos TRUE ReadFromLast TRUE Exec if ($EventID == 5156) drop(); </Input> <Input internal_mod> Module im_internal </Input> <Output out> Module om_tcp Host 10.x.x.a Port 514 OutputType LineBased <Exec> $Hostname = string(host_ip()); to_syslog_ietf(); parse_syslog(); $Message = '@cee: ' + to_json(); to_syslog_bsd(); </Exec> </Output> <Output outlcp> Module om_udp Host 10.x.x.b Port 514 OutputType Dgram Exec $Hostname = string(host_ip()); Exec to_syslog_snare(); </Output> <Route Syslog_lcp> Priority 1 Path eventlog, internal_mod => outlcp, out </Route>

I have an XML that I am trying to strip some data out so it can be pre-processed by software on another machine. XML file is being generated on a windows logging to an XML file. I am at bit of a lost I have tried too many things to list here. What I am trying to do is remove our domain name and our domain email address before it is sent to the machine to be pre-processed. I only want the username. Any records that have host\domain.com don’t need to be sent and I figured out how to drop that data. (number 4). If it helps I am running nxlog-ce-3.0.2284. Here is an example of event: (Removed bunch of xml fields for clarity of this post) 1: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">DOMAIN\username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 2: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 3: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username@domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 4: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">host\domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> Nxlog.conf: #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension xmlparser> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "C:\LogFiles\log*.log" InputType LineBased Exec $Message = $raw_event; SavePos TRUE ReadFromLast TRUE <Exec> Discard everything that doesn't seem to be an xml event if $raw_event !~ /^<Event>/ drop(); if $raw_event =~ /^(.+)host(.+)/ drop(); parse_xml(); Convert to JSON to_json(); </Exec> </Input> <Output out> Module om_udp Host yy.xx.xx.xx Port 514 </Output> <Route 1> Path in => out </Route>

Hey All, A bit of a newbee and trying to get NXLOG working with GrayLog. It is working and I'm seeing the information. The issue is that the information I'm seeing does not seem to match the PC's event logs. Please see below Config File: <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension _gelf> Module xm_gelf </Extension> <Input win> Module im_msvistalog </Input> Converting events to Snare format and sending them out over TCP syslog <Output graylog> Module om_udp Host X.X.X.X Port 3514 OutputType GELF </Output> <Route graylog_route> Path win => graylog </Route> Any ideas about what I'm doing wrong????

HTTP ERROR: 503 Problem accessing /nxlog-manager. Reason: Service Unavailable New install. Not sure why this is fighting me :)

Hi, we are using nxlog-ce-3.0.2272 on Linux (CentOS 7), after a server reboot nxlog is not started. The error message in nxlog logfile is: ERROR: couldn't open pidfile /run/nxlog/nxlog.pid. After the reboot of the server the directory /run/nxlog is missing, which seems to cause the error. The directory is created when nxlog-ce is installed on the server and nxlog is started OK. If I manually create the directory /run/nxlog and then start nxlog it also works. Is this a known error or have missed something ? BR Joakim

I am betting I am simply missing a step. Anyone able to help? Unpacking nxlog-manager (5.6.5633) ... dpkg-deb (subprocess): decompressing archive member: lzma error: compressed data is corrupt dpkg-deb: error: <decompress> subprocess returned error exit status 2 dpkg: error processing archive nxlog-manager-5.6.5633-jdk1.7.1.deb (--install): cannot copy extracted data for './opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/spring-js-resources-2.4.2.RELEASE.jar' to '/opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/spring-js-resources-2.4.2.RELEASE.jar.dpkg-new': unexpected end of file or stream Processing triggers for systemd (245.4-4ubuntu3.17) ... Errors were encountered while processing: nxlog-manager-5.6.5633-jdk1.7.1.deb

Dears, I'm running nxlog-ce 3.0.2272 under Windows Server 2022 to search the Application event log for specific events using a custom query in order to forward them as GELF messages to some Graylog server. The connection is secured by SSL. With Server 2019, everything runs smoothly but with Server 2022, nxlog.exe keeps crashing after a few events have been collected and sent to Graylog. Worst fact is, that events are omitted and not transferred to Graylog. The related event log entries (event id 1000): Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x00000000 Faulting module name: ntdll.dll, version: 10.0.20348.681, time stamp: 0x69d3cd31 Exception code: 0xc0000374 Fault offset: 0x0000000000103ad9 Faulting process id: 0x2b1c Faulting application start time: 0x01d86901d76501a6 Faulting application path: C:\Program Files (x86)\nxlog\nxlog.exe Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll Report Id: 362167c9-9922-4158-8d56-ee4bafd21e67 Faulting package full name: Faulting package-relative application ID: Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x00000000 Faulting module name: RPCRT4.dll, version: 10.0.20348.707, time stamp: 0xd31f9dd8 Exception code: 0xc0000005 Fault offset: 0x00000000000272e3 Faulting process id: 0x1a24 Faulting application start time: 0x01d868da69310cd8 Faulting application path: C:\Program Files (x86)\nxlog\nxlog.exe Faulting module path: C:\WINDOWS\System32\RPCRT4.dll Report Id: 96a6d244-74ca-4f6f-8667-8bb5082a452a Faulting package full name: Faulting package-relative application ID: Any idea? Thanks ahead, Elix

Has anyone installed NXLog Enterprise edition on Hyper-V? Any pros/cons if so?

Team, Need your assistance on nxlog agent. I have installed nxlog agent(Community Edition) in windows server and i could see the destination folder had created under c:/Program files (x86)/nxlog but when went to restart the nxlog service, i dont see the nxlog service inside services(service.msc) not sure where is the problem and need your assistance to fix it. Version: nxlog-ce-2.11.2190 Thanks, Vignesh

I am trying to use nxlog to process some historical files and load them into graylog. It is a nested json file and I only need a few fields out of it. I am hoping for some pointers on how to get the following data out of the json file and posted into graylog. username rename text to message rename first title to method rename title_link to method_link From the fields section Severity: value Region: Value Rename Last Seen to timestamp: value I would like graylog to use this as the timestamp for the data coming in. Below is my whole nxlog.conf but the relevant input is the guardduty input. nxlog.conf User nxlog Group nxlog Panic Soft default values: define INSTALLDIR /opt/nxlog PidFile %INSTALLDIR%/var/run/nxlog/nxlog.pid CacheDir %INSTALLDIR%/var/spool/nxlog ModuleDir %INSTALLDIR%/lib/nxlog/modules SpoolDir %INSTALLDIR%/var/spool/nxlog define CERTDIR %INSTALLDIR%/var/lib/nxlog/cert define CONFDIR %INSTALLDIR%/etc/nxlog.d Note that these two lines define constants only; the log file location is ultimately set by the LogFile directive (see below). The MYLOGFILE define is also used to rotate the log file automatically (see the _fileop block). define LOGDIR %INSTALLDIR%/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log If you are not using NXLog Manager, disable the include line and enable LogLevel and LogFile. #include %CONFDIR%/*.conf LogLevel DEBUG LogFile %MYLOGFILE% <Extension _syslog> Module xm_syslog </Extension> This block rotates %MYLOGFILE% on a schedule. Note that if LogFile is changed in managed.conf via NXLog Manager, rotation of the new file should also be configured there. <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour &lt;Exec&gt; if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') &gt;= 5M) ) { file_cycle('%MYLOGFILE%', 8); } &lt;/Exec&gt; &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension json_parser> Module xm_json Flatten True PrettyPrint True </Extension> <Extension exec> Module xm_exec </Extension> <Extension gelf> Module xm_gelf </Extension> <Extension antivirus_csv> Module xm_csv Fields $Severity,$timestamp,$ip,$endtime,$User,$User Groups,$Device,$Device Group Delimiter , </Extension> <Extension xml> Module xm_xml </Extension> <Input alienvault> Module im_file File "/var/nxlog/alienvault/*.log" SavePos TRUE ReadFromLast TRUE <Exec> parse_xml(); to_json(); </Exec> </Input> <Input antivirus> Module im_file File "/var/nxlog/antivirus/*.csv" ReadFromLast TRUE SavePos True <Exec> antivirus_csv->parse_csv(); to_json(); </Exec> </Input> <Input guardduty> Module im_file File "/var/nxlog/guardduty/*.json" SavePos True InputType json_parser <Exec> #Delete some fields that aren't necessary delete($type); delete($subtype); delete($text); delete($mrkdwn_in); delete($EventReceivedTime); #Convert fields back to JSON because of the deletion to_json(); &lt;/Exec&gt; </Input> <Output udp_12202> Module om_udp Host graylog.mydomain.com:12202 OutputType GELF_UDP </Output> <Output udp_5515> Module om_udp Host graylog.mydomain.com:5515 OutputType GELF_UDP </Output> <Route antivirus_to_udp_5515> Path antivirus => udp_5515 </Route> <Route to_udp_12202> Path alienvault, guardduty => udp_12202 </Route> Sample JSON. The files are larger but I kept the first and last entry. [ { "type": "message", "subtype": "bot_message", "text": "", "ts": "1614611466.000100", "username": "GuardDuty", "icons": { "image_48": "https://s3-us-east-1.amazonaws.com/slack-files2/bot_icons/2019-07-02/671758367922_48.png" }, "bot_id": "BL5799J6A", "attachments": [ { "fallback": "Recon:IAMUser/UserPermissions - <https://console.aws.amazon.com/guardduty/home?region=us-east-1#/findings?search=id%5t45ffg59a575art6789404dr5oci5a4zz>", "text": "APIs commonly used to discover the users, groups, policies and permissions in an account, was invoked by IAM principal Some-Role under unusual circumstances. Such activity is not typically seen from this principal.", "pretext": "Finding in us-east-1 for Acct: 505986456095", "title": "Recon:IAMUser/UserPermissions", "id": 1, "title_link": "https://console.aws.amazon.com/guardduty/home?region=us-east-1#/findings?search=id%5t45ffg59a575art6789404dr5oci5a4zz", "color": "e2d43b", "fields": [ { "title": "Severity", "value": "Medium", "short": true }, { "title": "Region", "value": "us-east-1", "short": true }, { "title": "Last Seen", "value": "<!date^1614611294^{date} at {time} | 2021-03-01T15:08:14.526Z>", "short": true } ], "mrkdwn_in": [ "pretext" ] } ] }, { "type": "message", "subtype": "bot_message", "text": "", "ts": "1614629763.001600", "username": "GuardDuty", "icons": { "image_48": "https://s3-us-east-1.amazonaws.com/slack-files2/bot_icons/2019-07-02/671758367922_48.png" }, "bot_id": "BL5799J6A", "attachments": [ { "fallback": "Recon:IAMUser/UserPermissions - <https://console.aws.amazon.com/guardduty/home?region=us-east-1#/findings?search=id%5t45ffg59a575art6789404dr5oci5a4zz>", "text": "APIs commonly used to discover the users, groups, policies and permissions in an account, was invoked by IAM principal Some-Role under unusual circumstances. Such activity is not typically seen from this principal.", "pretext": "Finding in us-east-1 for Acct: 505986456095", "title": "Recon:IAMUser/UserPermissions", "id": 1, "title_link": "https://console.aws.amazon.com/guardduty/home?region=us-east-1#/findings?search=id%5t45ffg59a575art6789404dr5oci5a4zz", "color": "e2d43b", "fields": [ { "title": "Severity", "value": "Medium", "short": true }, { "title": "Region", "value": "us-east-1", "short": true }, { "title": "Last Seen", "value": "<!date^1614629482^{date} at {time} | 2021-03-01T20:11:22.426Z>", "short": true } ], "mrkdwn_in": [ "pretext" ] } ] } ]

Hi, Reaching out to you because when I try to start the NXLog, i get the below message. Error 1067: the process terminated unexpectedly

Hello, We have a the need to be able to collect a collection of files that would be our Standard template of collecte and to add another template to collecte specific files. The goal is to limite the risk for disruption of other logging on the same system and to be able to modify the standard without any impact of the specific files. Is this possible and if so how to do it. Regards Edouard MORAGUES

Hi, seems there is a bug in im_msvistalog module in nxlog-ce-3.0.2272 for Windows. After service has generated some output, stopping service sometimes give the message box "Broken pipe" and sometimes Nxlog write a malformed configcache.dat and following the start posts an ERROR failed to restore the saved position from bookmark xml (error:15008)" Tested on Windows 2012 R2 and Windows 2019 <Output out> Module om_file File '%LOGDIR%\Output' </Output> <Input _im_eventlog> Module im_msvistalog SavePos TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='Security'>*</Select> </Query> </QueryList> </QueryXML> # Query <QueryList>\ # <Query Id='0'>\ # <Select Path="Security">*</Select>\ # </Query>\ # </QueryList> </Input> #<Output outNull> # Module om_null #</Output> <Route RouteA> Path _im_eventlog => out </Route> This issue make the module not worth to use because not saving the bookmark of the processed registry record, force it to generate same logs from start over each service restart. Best.

Hi, We have this error ERROR binary header not found at position 0 in data received from xxx.xxx.xxx.xxx is input really binary? Any hint will be appreciated. Regards, K

Has anyone successfully send logs from NXLog to New Relic? I've tried using om_tcp, om_ssl, om_http to send syslogs to New Relic but I keep running into issues with NXLog. For example: ERROR [om_http|out] SSL error, SSL_ERROR_SSL: retval -1, from 162.247.243.181:443, reason: sslv3 alert handshake failure ERROR [om_http|out] [ssl.c:209/nx_ssl_error()] SSL error, SSL_ERROR_SSL: retval -1, from 162.247.243.181:443, ( reason: unsupported protocol, function: SSL23_GET_SERVER_HELLO, library: SSL routines ) I've tried both the TCP endpoint for syslogs and the headerless API log endpoint, and if I send a curl -X POST with some JSON data to the New Relic log endpoint, it works fine without any issues. Any ideas?

Hi, It is possible to use Nxlog to pull logs from ELK to the Nxlog Manager? Thanks