I'm using NXLog CE to forward Windows event logs via the im_msvistalog module. There's about 161 event id's that I want to whitelist from the security log and not send anything else from the event logs. The following config snippet works: <Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='Security'>*[System[(EventID=4627)] or System[(EventID=4624)] or System[(EventID=4775)] or System[(EventID=4776)] or System[(EventID=4777)] or System[(EventID=4741)] or System[(EventID=4742)] or System[(EventID=4743)] or System[(EventID=4744)] or System[(EventID=4745)] or System[(EventID=4746)] or System[(EventID=4747)] or System[(EventID=4748)] or System[(EventID=4749)] or System[(EventID=4750)] or System[(EventID=4751)] or System[(EventID=4752)] or System[(EventID=4753)] or System[(EventID=4759)] or System[(EventID=4760)] or System[(EventID=4672)] or System[(EventID=4634)] or System[(EventID=4648)]] </Select> </Query> </QueryList> </QueryXML> </Input> The issue is that once I add one more line to that config, NXLog stops shipping events completely. Is there a better way for me to write this that would allow for more than 23 whitelisted event id's?

What are the current vulnerabilities associated with NXLog Common Edition, version 2.10.2150?

While evaluating the NXLOG enterprise trial edition, we faced a blocker and I need some clarification/help on the same. We are using NXLOG’s CEF module (xm_cef, xm_json ) which convert’s CEF messages into JSON. It is working properly for most of the cases but giving unexpected output for few of them. raw CEF message :- CEF:0|Himanshu Arora|Sample1|10.5.011|195|Process Sample|5|abc=Sample Data suser=XY fname= dvc= shost=10.1.1.1 dhost= duser= externalId= app= reason= cs1Label=Affected User cs1= cs2Label=Safe Name cs2=Notification Sample cs3Label=Device Sample cs3= cs4Label=Database cs4= cs5Label=&quot;Other info&quot; cs5= cn1Label=Request Id cn1= cn2Label=Ticket Id cn2= msg= JSON output :- { "EventReceivedTime": "2019-04-25T13:43:49.483942+05:30", "SourceModuleName": "cef_input", "SourceModuleType": "im_file", "SyslogFacilityValue": 1, "SyslogFacility": "USER", "SyslogSeverityValue": 5, "SyslogSeverity": "NOTICE", "SeverityValue": 3, "Severity": "WARNING", "EventTime": "2019-04-25T13:43:49.483969+05:30", "Hostname": "himanshu-VirtualBox", "SourceName": "CEF", "CEFVersion": 0, "CEFDeviceVendor": "Himanshu Arora", "CEFDeviceProduct": "Sample1", "CEFDeviceVersion": "10.5.011", "CEFSignatureID": "195", "CEFName": "Process Sample", "CEFSeverity": 5, "abc": "Sample Data", "suser": "XY", "fname": "dvc=", "shost": "10.1.1.1", "dhost": "duser=" } If you notice the raw message has some fields called cs1Label,cs2Label,cs2,cn1Label,cn2Label, cn2 . these fields are missing in the JSON output file. Moreover in JSON the fields "fname" , "dhost" should have had empty value. I would like to know If this issue exists only in the enterprise trial edition and it will be resolved if we purchase the Enterprise edition ? or is it issue being fixed and will be released soon? Is there a way to include any third party libraries into NXLOG that can convert CEF to JSON.

Hi, I have been trying to stream data and the data transfer was successful with a #015 appended to each line in my log file. This is happening to all the log types trasferred. Can you let me know what could cause that?

It appears that nxlog continually retries when trying to send a log. Is there a way to limit the amount of retries and continue on if it fails over x amount of times? My issue is this, if the body of the request is bad, I will get a 400 as a result. This means, I could get stuck trying to send a bad message over and over.

I am trying to configure to capture windows 10 logs and it is displaying the following messages below: 2019-04-19 23:40:05 WARNING nxlog-ce received a termination request signal, exiting ... 2019-04-19 23:40:07 WARNING no functional input modules! 2019-04-19 23:40:07 WARNING no routes defined! 2019-04-19 23:40:07 WARNING not starting unused module out 2019-04-19 23:40:07 INFO nxlog-ce-2.10.2150 started look how this nxlog.conf Panic Soft #NoFreeOnExit TRUE define ROOT C: \ Program Files (x86) \ nxlog define CERTDIR% ROOT% \ cert define CONFDIR% ROOT% \ conf define LOGDIR% ROOT% \ data define LOGFILE% LOGDIR% \ nxlog.log LogFile% LOGFILE% Moduledir% ROOT% \ modules CacheDir% ROOT% \ data Pidfile% ROOT% \ data \ nxlog.pid SpoolDir% ROOT% \ data <Extension _syslog> Module xm_syslog </ Extension> <Extension gelf> Module xm_gelf </ Extension> <Output out> Module om_tcp Host 192.168.1.48 Port 12201 #Exec to_syslog_snare (); OutputType GELF_TCP </ Output> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </ Extension> <Extension _exec> Module xm_exec </ Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists ('% LOGFILE%') and \ (file_size ('% LOGFILE%')&gt; = 5M)) \ file_cycle ('% LOGFILE%', 8); &lt;/ Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists ('% LOGFILE%') file_cycle ('% LOGFILE%', 8); &lt;/ Schedule&gt; </ Extension>

Hi, I'm using EE trial edition now in my machine but i need to use CE edition as well for testing. Can I use both in same machine? will I lose my EE trial if i download CE now?

Is there a way to send multiple records in an http call? I'd like to send multiple rows of my log file via an http call.

There is a bug in NXLog Community Edition 2.10.2150, with module im_ssl regarding opening CA files. NXLog seems to create a file descriptor for the CA file each time a new connection is made. That is, NXLog opens the file again and again. If NXLog runs on Linux, this can quickly hit the OS limit of maximum number of files open. The following error is seen repeatedly once the OS limit of file descriptors is reached: ERROR SSL error, failed to load ca cert from '<path_to_file>', Too many open files Raising the OS limit is only a temporary solution: eventually, the next limit can be hit How can this bug be fixed?

I am trying to set up an om_http output. I get the response of ERROR HTTP response status is not OK: 400 Bad Request I need to trouble shoot what message I am actually sending. Is there an easy way to see what message is sent? This is my in/out config. <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input in> Module im_file File '<scrubbed>/data.json' Exec $message = to_json(); </Input> <Output out> Module om_http URL <scrubbed> HTTPSCAFile <scrubbed> HTTPSCertFile <scrubbed> HTTPSCertKeyFile <scrubbed> ContentType application/vnd.kafka.v1+json </Output> <Route 1> Path in => out </Route>

Hi, i have recently setup graylog and i'm using nxlog as my collector, everything seems to be working fine except nxlog is not sending logs to the graylog server, i have checked nxlog logs and this is the error 2019-04-16 11:51:08 ERROR failed to open C:\Users\s.chimere\Desktop\GRAYLOG; Access is denied C:\Users\s.chimere\Desktop\GRAYLOG this is where i have my test logs Please can anyone help.

Hi everybody, I use NXLog EE 4.3.4308. Time to time client is crushed with this error at module libapr-1-0.dll. Windows Application log: 2019-04-14 01:00:06 ERROR 1000 Faulting application name: nxlog.exe, version: 4.3.4308.0, time stamp: 0x00000000 Faulting module name: libapr-1-0.dll, version: 0.0.0.0, time stamp: 0x00000000 Exception code: 0xc0000005 Fault offset: 0x00000000000084b9 Faulting process id: 0x197c Faulting application start time: 0x01d4f112445554b9 Faulting application path: C:\nxlog\nxlog.exe Faulting module path: C:\nxlog\libapr-1-0.dll Report Id: e1a9c09a-a1e6-432d-8ecc-12d042258be8 Faulting package full name: Faulting package-relative application ID: Any ideas, please. Thanks! NXLog's log before crushed: 2019-04-14 00:56:37 ERROR apr_stat() failed on file C:\logs\2018-12-14.log; Access is denied. 2019-04-14 00:56:39 WARNING input file was deleted: C:\logs\2018-12-14.log This is old log file was deleted in weekly log rotation. Why NXLog scanning old files? This file was not updated for 3 month.

Hi, I'm trying to parse a csv log file from my local and store it again in another location in my local. But I see it is not happening. The config has no errors. the destination file is same as source file with respect to file properties. Can you tell whether any specific check should be done? <Extension csv_parser1> Module xm_csv Fields date-time,client-ip,client-hostname,server-ip,server-hostname, source-context,connector-id,source,event-id,internal-message-id,message-id,recipient-address, recipient-status,total-bytes,recipient-count,related-recipient-address,reference,message-subject, sender-address,return-path,message-info,directionality,tenant-id, original-client-ip,original-server-ip,custom-data Delimiter , </Extension> Message Tracking log as input <Input messagetracking> Module im_file File '%BASEDIR%file.log' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser1->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec> </Input> <Output msg> Module om_file File 'location\msg.log' </Output> <Route 1> Path messagetracking => msg </Route>

Can you explain what exactly happens in this block? I don't get a clear explanation of what "xEF\xBB\xBF" means in the below code and why it is having drop ()? . <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec>

Hi, I would like to know the cost of enterprise edition. Also, I would like to know whether we could use the purchased nxlog EE package in more than one server to collect logs?

Hi, I'm using a CE where xm_w3c module is not available. So i'm getting below error: 2019-04-09 14:59:30 ERROR Failed to load module from C:\Program Files (x86)\nxlog\modules\extension\xm_w3c.dll, The specified module could not be found. ; The specified module could not be found. 2019-04-09 14:59:30 ERROR Invalid InputType 'w3c_parser' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:94 Is there a way to overcome this error in CE by downloading the particular module ? Or should I try using the Enterprise edition?

Hi, Im actually having an issue with my nxlog server. We are trying to send antivirus log from a McAfee EPO to my NX. The problem we facing is that when we try a connection test from EPO to NXLOG we get this message on our Nxlog server. 2019-04-09 19:32:54 INFO SSL connection accepted from 10.28.26.214:59126 2019-04-09 19:32:54 ERROR SSL error, SSL_ERROR_SSL: retval -1, reason: peer did not return a certificate 2019-04-09 19:32:54 WARNING SSL connection closed from 10.28.26.214:59126 Can we receive the AV log without using the certificate ? Do you know a way to bypass this ?. The certificat have been created with OPENSSL with the help of one of your technicien and the certificat looks good... we have somme difficulty to understand why this operation fail. We have also put the certificat we create for NXLOG on our Antivirus server to let them communicate. Do you have any idea of what is the problem ? Your help is very appreciated gain. Greetings,

Is the scalability / performance of community vs enterprise edition any different? We tried the community edition for WEC/WEF and it appears to be dropping logs at 2000 eps. We're wondering if there are any configuration we should be aware of. Moreover, please provide sizing recommendations: > What eps can a single nxlog agent support for WEC/WEF collection? > How many VM's of what size (CPU cores and GB memory) should we plan for to support 50,000 eps?

Is there a way to run NxLOG in a "throttled" state during certain times of the day? For instance, process x number of logs per hour from 8-5

Hi. We are facing this problem that NXLog takes a lot of memory when using it to collect logs from PostgreSQL database. I tried to modify the polling interval in the config but it did not help. Our config is like this now: <Input PostgreSQL> Module im_dbi Driver pgsql SavePos false PollInterval 5 Option host 127.0.0.1 Option username ***** Option password ************** Option dbname messagelog SQL SELECT id, discriminator, time, queryid, message, timestamprecord, response, memberclass, membercode, subsystemcode FROM logrecord Exec $SourceName = 'PostgreSQL'; Exec to_json(); </Input> <Output out> Module om_tcp Host 192.168.1.1 Port 1468 Exec to_syslog_ietf(); </Output> <Route 1> Path PostgreSQL => out </Route> I enabled debug and it produces a huge amount of these lines per second: 2019-04-09 15:00:57 DEBUG worker 0 processing event 0x7f67240a6d80 2019-04-09 15:00:57 DEBUG PROCESS_EVENT: POLL (out) 2019-04-09 15:00:57 DEBUG nx_module_pollset_poll: out 2019-04-09 15:00:57 DEBUG worker 2 got signal for new job 2019-04-09 15:00:57 DEBUG worker 2 got no event to process 2019-04-09 15:00:57 DEBUG worker 2 waiting for new event 2019-04-09 15:00:57 DEBUG [out] no poll events, pollset_poll timed out 2019-04-09 15:00:57 DEBUG nx_event_to_jobqueue: POLL (out) 2019-04-09 15:00:57 DEBUG event added to jobqueue 2019-04-09 15:00:57 DEBUG worker 0 processing event 0x7f67240a4fb0 2019-04-09 15:00:57 DEBUG PROCESS_EVENT: POLL (out) 2019-04-09 15:00:57 DEBUG nx_module_pollset_poll: out 2019-04-09 15:00:57 DEBUG worker 1 got signal for new job 2019-04-09 15:00:57 DEBUG worker 1 got no event to process 2019-04-09 15:00:57 DEBUG worker 1 waiting for new event 2019-04-09 15:00:57 DEBUG [out] no poll events, pollset_poll timed out 2019-04-09 15:00:57 DEBUG nx_event_to_jobqueue: POLL (out) 2019-04-09 15:00:57 DEBUG event added to jobqueue 2019-04-09 15:00:57 DEBUG worker 0 processing event 0x7f67240a6d80 2019-04-09 15:00:57 DEBUG PROCESS_EVENT: POLL (out) 2019-04-09 15:00:57 DEBUG nx_module_pollset_poll: out 2019-04-09 15:00:57 DEBUG worker 2 got signal for new job 2019-04-09 15:00:57 DEBUG worker 2 got no event to process 2019-04-09 15:00:57 DEBUG worker 2 waiting for new event 2019-04-09 15:00:57 DEBUG [out] no poll events, pollset_poll timed out 2019-04-09 15:00:57 DEBUG nx_event_to_jobqueue: POLL (out) 2019-04-09 15:00:57 DEBUG event added to jobqueue 2019-04-09 15:00:57 DEBUG worker 0 processing event 0x7f67240a4fb0 We are not seeing the memory usage problem on other log collecting methods. Any ideas what could cause this? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 21239 root 20 0 162120 2396 1600 R 0.3 0.1 0:00.04 top 28670 nxlog 20 0 1589652 1.3g 3676 S 0.3 34.4 14:33.41 nxlog