I'm not sure it's causing a problem, but consistently nxlog will crash on Windows when making a call to stop the nxlog service with error "System Error 109 has occurred. The pipe has ended."

I'd like to fix this if anyone knows of a way.

We are using NXLog CE's im_msvistalog module to forward Windows Event Logs from the Security log, with some filtering, to an external syslog server. Functionally this works well and does exactly what we need it to.

The problem we are having is that nxlog.exe process often consumes rather high percentages of a workstation's CPU in bursts. Between 25 and 35 percent every few minutes, for around a minute at a time. This is generally too much of a performance hit and I need to find some way to resolve it.

I have already mitigated the size of the event log file that nxlog.exe is querying from by clearing the Security log entirely, so this is happening even on a system with not more than a few dozen log entries to read from. The query itself is fairly simple, it loads all Event ID 4625 entries from the Security log (these are logon failures). It then has a single command to drop any logon failures that were initiated for a computer account instead of a user account (this is done by reading the target account trying to logon, string parsing the account username to see if the final character is a "$", which denotes a computer account, and dropping the log if the "$" is found).

What can I do to mitigate the excessive CPU usage?

What I have tried so far: clearing the Windows event log that nxlog is reading from to reduce the size of the data it needs to read from disk, using UDP syslog forwarding instead of TCP, removed the parsing that dropped Event Log ID 4625 entries where the target account being logged in was a computer account instead of a user account. None of this has helped.

Edit to add: I did try writing out text logging instead of syslog forwarding. This worked but I am still experiencing the periodic excessive CPU consumption. The problem is likely in the im_msvistalog input module, I would assume.

Hi everyone!

Has anyone ever installed or have some tips regarding a community edition package compatible with Oracle Linux 6? We've tried to use the package for REHL/CENTOS 6, installing all the dependencies, but no log is generated and sent to the syslog server.

Thanks!

Trying to get this to work on a Windows 2019 Server that's a clean build with nothing on it and has all current Windows updates applied
Is it an issue with Windows 2019 Server or an issue with the XM_SYSLOG module?

######### WORKING - Copies event log data to C:\Program Files\nxlog\data\nxlog-output.log ########
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
Module xm_syslog
</Extension>

# Monitor Windows event logs
<Input eventlog>
Module im_msvistalog
</Input>

<Output file>
Module om_file
File 'C:\Program Files\nxlog\data\nxlog-output.log'
Exec to_syslog_snare();
</Output>

<Output syslogout>
Module om_ssl
Host logsx.papertrailapp.com
Port 12345
Exec $Hostname = hostname(); to_syslog_ietf();
OutputType Syslog_TLS
CAFile %CERTDIR%/papertrail-bundle.pem
AllowUntrusted FALSE
</Output>

<Route out>
Path eventlog => file
</Route>

######### NOT WORKING - NXLOG Service will start for a moment and then stop immediately ###########
<Route out>
Path eventlog => syslogout
</Route>


nxlog.log
2022-03-16 18:10:47 WARNING not starting unused module file
2022-03-16 18:10:47 INFO nxlog-ce-3.0.2272 started
2022-03-16 18:10:47 INFO connecting to logs3.papertrailapp.com:49305
2022-03-16 18:10:47 INFO successfully connected to logx.papertrailapp.com:12345

Windows Logs, Application Events:
Source: Application Error
Event ID: 1000
Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x00000000
Faulting module name: ntdll.dll, version: 10.0.17763.2628, time stamp: 0x91ea188a
Exception code: 0xc0000374
Fault offset: 0x00000000000faad9
Faulting process id: 0xa7c
Faulting application start time: 0x01d8399bfa79f8d0
Faulting application path: C:\Program Files\nxlog\nxlog.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 7870365a-2a26-49dd-9670-7c8d889f9dda
Faulting package full name:
Faulting package-relative application ID:

Windows Logs, Application Events:
Source: Windows Error Reporting
Event ID: 1001
Fault bucket 1367701673690831831, type 4
Event Name: APPCRASH
Response: Not available
Cab Id: 0

Problem signature:
P1: nxlog.exe
P2: 0.0.0.0
P3: 00000000
P4: StackHash_2e07
P5: 10.0.17763.2628
P6: 91ea188a
P7: c0000374
P8: PCH_43_FROM_ntdll+0x00000000000A0544
P9:
P10:

Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER86A8.tmp.dmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER87D2.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER8802.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER8804.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER8843.tmp.txt

These files may be available here:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_nxlog.exe_7198d2d4b17dc7d6aaa419f8df82eecf4ad86a_e5992931_12418b0d

Analysis symbol:
Rechecking for solution: 0
Report Id: 7870365a-2a26-49dd-9670-7c8d889f9dda
Report Status: 268435456
Hashed bucket: 8cc762824f1e456172fb0d6d030c9bd7
Cab Guid: 0

Hi team, from the guide below we know that the Nxlog is able to deduplicate the log by some specific fields ("CheckFields"). https://nxlog.co/documentation/nxlog-user-guide/pm_norepeat.html

May I know if it is possible to set the interval for it? Let's take below configuration as example. Is it possible to set the interval = 10s, so that the log with the same fields (Hostname, SourceName, Message) will be suppressed for only every 10 seconds. This is doable from Logstash (throttle -> period), but we wanna confirm if it is capable in Nxlog as well. Thank you.

<Input uds> Module im_uds UDS /dev/log </Input>

<Processor norepeat> Module pm_norepeat CheckFields Hostname, SourceName, Message </Processor>

<Output file> Module om_file File "/var/log/messages" </Output>

<Route uds_to_file> Path uds => norepeat => file </Route>

Hi all.

I'm having some windows server that are subscribed to a nxlog server, who in turn sends the windows logs to a linux/syslog server.

The syslog receives all these logs as NOTICE.USER which is not too practical.

I would want the nxlog to keep the criticity of the message when forwarding them. I would want nxlog to prefix the logs with the original log sender hostname so that they appear as $PROGRAM in syslog.

Also, is there a way to use some criterions to send logs from nxlog to syslog using different facilities (USER, MAIL, LOCALn,...) according to some criterions (real PROGRAM value for instance)

Hello -

Currently using this for Event logs:

Exec $SyslogFacilityValue = 22;to_syslog_snare();

However the timestamp in the logs is local machine time and it needs to be in UTC. From searching around it looks like this is possible in EE:

DateFormat YYYY-MM-DDThh:mm:ss.sUTC

However I cannot find that this is feasible for CE.

Is there a way with Community Edition to either manually set the timestamp to UTC (without having to know the local machine time) or, worst case scenario, is it possible to forward the log from nxlog without a timestamp at all so the received log will only have a timestamp of ingest time?

Config works without <QueryXML> lines. When i add thoose lines it stops sends any data. I tested with user modification or login fail etc.

Thanks for your help

<Extension syslog> Module xm_syslog </Extension>

<Extension xml> Module xm_xml </Extension>

<Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Security">*[System[(Level=1 or Level=2 or Level=3 or Level=4 or Level=0) and (EventID=1102 or EventID=4719 or EventID=4704 or EventID=4717 or EventID=4738 or EventID=4798 or EventID=4705 or EventID=4674 or EventID=4697 or EventID=4648 or EventID=4723 or EventID=4946 or EventID=4950 or EventID=6416 or EventID=6424 or EventID=4732)]]</Select> </Query> </QueryList> </QueryXML> Exec $Message = to_xml(); to_syslog_bsd(); </Input>

<Output tcp> Module om_tcp Host 127.0.0.1 Port 514 </Output>

<Route eventlog_to_tcp> Path eventlog => tcp </Route>

Hello,

I am trying to shorten a part of my raw_field which is way too long : Host_Application="..." to only keep like 50 characters. And I noticed that NXlog is using PCRE syntax for finding a string in a field, but not for the substitution where it uses classic regex.

I tried different things:

• Substitute with a regex the part I want but I can't, since it replaces everything I write in the regex.
• Find the exact string I want to replace with my PCRE rexgex, and then try to substitute it but I can't use the $1 field as the source string to modify.

I would like to know if there is any way of doing what I want.

Malauran

Hi, Having this error below ..any ideas how to resolved ? Thanks

INFO nxlog-ce-2.11.2190 started ERROR failed to subscribe to msvistalog events using bookmark: Access is denied.
ERROR failed to subscribe to msvistalog events,access denied [error code: 5]; Access is denied.
WARNING stopping nxlog service WARNING nxlog-ce received a termination request signal, exiting... WARNING not starting unused module dns INFO nxlog-ce-2.11.2190 started ERROR failed to subscribe to msvistalog events using bookmark: Access is denied.
ERROR failed to subscribe to msvistalog events,access denied [error code: 5]; Access is denied.

Hi All,

We have configured Nxlog to send application logs to RSA Virtual log collector.

We are able to send one logfile using the below configurations in nxlog.conf

<Input AppLogs> Module im_file File 'C:\Important_Application\Logs\log1.log' SavePos FALSE Recursive TRUE ReadFromLast FALSE Exec $Message = $raw_event; </Input>

<Output out2> Module om_tcp Host "0.0.0.0" Port 514 </Output>

<Route 2> Path AppLogs => out2 </Route>

However, we are unable to send all the log files within the Logs directory.

We tried the below methods:

<Input AppLogs> Module im_file File 'C:\Important_Application\Logs*.log' or 'C:\Important_Application\Logs*.log' or 'C:\Important_Application\Logs' or 'C:\Important_Application\Logs*' SavePos FALSE Recursive TRUE ReadFromLast FALSE Exec $Message = $raw_event; </Input>

But none of the above methods are workings. We need to send all the log files within the "Logs" folder.

Please help.

I get an error in this module but the xm_python.dll is available and 35kb

ERROR Failed to load module from C:\nxlog\modules\extension\xm_python.dll

thanks

Hi, I install NXLog Enterprise Edition v5 trial And try to filter out events before send to SIEM. I can get some events and see SIEM side. But when I create fake event , cannot see all. What I want? I want to forward Windows server APP, SEC ve SYS logs that have only WARNING,ERROR and CRITICAL levels in CEF format Is that config part correct? Module xm_syslog Module xm_cef Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog # Channel Security *[System/Level<4] *[System/Level<4] *[System/Level<4]

Hello Techies,

Can someone please help me below query?

How to execute windows MSI installer to silently install nxlog agent(nxlog-ce-3.0.2272.msi) using a PowerShell script. Any code snippet or reference url will be very helpful to refer from.

Thanks in advance Anil Kr

I've been trying to figure out the best way to ingest logs from 2 different data streams and have them go to separate log files.

Heres the copy of my configuration. For my first input i have a bunch of firewall logs coming in to /syslog/firewalls.log. I now want to ingest syslog data from my isilon to a different log file. It only seems to work if I have host 0.0.0.0 setup. I'm getting the data but everything is being written to firewalls.log and not my isilon.log

Any help would be greatly appreciated.

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension _json>
Module xm_json
</Extension>

<Input udp1>
Module im_udp
Host 0.0.0.0
Port 514
Exec parse_syslog();
</Input>

<Output fwlog>
Module om_file
File "/syslog/firewalls.log"
Exec to_json();
</Output>

<Input udp2>
Module im_udp
Host 0.0.0.0
Port 514
InputType Syslog_TLS
Exec parse_syslog();
</Input>


<Output isilog>
Module om_file
File "/syslog/isilon.log"
Exec to_json();
</Output>

########################################
# Routes #
########################################

<Route udp_to_file1>
Path udp1 => fwlog
</Route>

<Route udp_to_file2>
Path udp2 => isilog
</Route>

https://docs.nxlog.co/ce/current/index.html#om_python
The document has usage information. is not it right ?

or is it only for linux ? https://gitlab.com/nxlog-public/nxlog-ce/-/blob/master/ChangeLog.txt 2021-12-13 3.0.2272 [2268] added python language integration modules

[2639] added Raijin output module - om_raijin

Thanks

Have you ever tested om_elasticsearch module with the 8th ElasticSearch version? I have no error log in nxlog.log, but no index or data are available in ES instance. I used the trial version 5.4 with ElasticSearch 8.0.

Hi. Tell me where I can download the 32-bit version of CE for Windows?

Hi Folks,

We are from a reputed service based company, We have recently tested NXLog. We are planning to deploy it more than 500 production server. We want to automate deployment process. We have below queries.

1. Is the windows setup (nxlog-ce-3.0.2272.msi) supports silent installation?
2. Can it be installed using a PowerShell script without manual intervention?

Note- If NxLog serves our purpose with better performance then we will go for enterprise edition.

Thanks Anil

Hi everyone

I'm very new to this and have searched around but couldn't find anything obvious.

I have a few Pc's with NXLog Community Edition sending data to a Loggly instance, event log data

They are picking up Security and System and Application ogs out of the box it seems, but i'm wondering if i need to do anything to add other event log sources to have them sent to Loggly?

Is it a matter of adding something to the Conf file?

For example, I have Kaspersky AV on an endpoint and i want to pick up the specific Event Log where Kaspersky sits.

Am i explaining it right? I basically want to add other event Log types into the process of log sending

Mark