I have nxlog 2.10.1542 on centos 7.6 I am sending rfc 3195 events to the nxlog for forward to a siem A snippet of the log that captures the error event is seen below.

2019-05-01 08:34:32 DEBUG evaluating expression 'field' at /opt/tap-nxlog/conf/nxlog.conf:128 2019-05-01 08:34:32 DEBUG evaluating expression 'field' at /opt/tap-nxlog/conf/nxlog.conf:129 2019-05-01 08:34:32 DEBUG successfully got priority

2019-05-01 08:34:32 DEBUG date is logver=600000267 timestamp=1556692474 tz="UTC+2" devname="FW-NCH-FGT600E-1" devid="<REDACATED>" vd="root" date=2019-05-01 time=08:34:34 logid="1059028705" type="utm" subtype="app-ctrl" eventtype="app-ctrl-all" level="warning" eventtime=1556692474 appid=40169 srcip=devid="<REDACATED> dstip=216.58.211.142 srcport=57244 dstport=443 srcintf="x2" srcintfrole="dmz" dstintf="x2" dstintfrole="dmz" proto=17 service="udp/443" direction="outgoing" policyid=3 sessionid=12692393 applist="sniffer-profile" appcat="Network.Service" app="QUIC" action="block" incidentserialno=1737302560 msg="Network.Service: QUIC," apprisk="low"

2019-05-01 08:34:32 DEBUG in nx_date_parse 2019-05-01 08:34:32 DEBUG in vpn parsing 2019-05-01 08:34:32 DEBUG if loop errored and return bad date 2019-05-01 08:34:32 DEBUG parse_vpn failed 2019-05-01 08:34:32 DEBUG not correct date

From what I am seeing the date matches from the event and the debug logging. additionally the epoch time "Wednesday, May 1, 2019 6:34:34 AM" from the event also seems to match. Any thoughts would be appreciated. Thanks in advance Adam

Hi, I have updated the NXLOG installation to version 2.10.2150 hoping that it will fix my issue. I have NXlogs installed on 6 servers and they are work fine except one server. Thi sis afile server so there are no application installed on it. On this server I get the following errors. When I manually restart the NXLOG service everything runs fine for few days and then NXLOG stops reporting logs. I appreciate assistance in trying to resolve this issue.

The log file located at C:\Program Files (x86)\nxlog\data contains this:

2019-05-07 10:23:44 INFO nxlog-ce-2.10.2150 started 2019-05-10 06:35:36 ERROR EvtNext failed with error 15007: The specified channel could not be found. Check channel configuration.
2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='246'><Select Path='Microsoft-WindowsAzure-Diagnostics/Bootstrapper'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='247'><Select Path='Microsoft-WindowsAzure-Diagnostics/Heartbeat'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='248'><Select Path='Microsoft-WindowsAzure-Diagnostics/Runtime'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='249'><Select Path='Microsoft-WindowsAzure-Status/GuestAgent'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='250'><Select Path='Microsoft-WindowsAzure-Status/Plugins'>*</Select></Query> 2019-05-10 06:35:43 ERROR EvtNext failed with error 1722: The RPC server is unavailable.
2019-05-10 06:35:43 ERROR Failed to query available channels; The RPC server is unavailable.
2019-05-12 08:31:07 WARNING received a system shutdown request 2019-05-12 08:31:07 WARNING stopping nxlog service 2019-05-12 08:31:07 WARNING nxlog-ce received a termination request signal, exiting... 2019-05-12 08:31:58 INFO nxlog-ce-2.10.2150 started 2019-05-12 08:32:07 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources. 2019-05-12 08:32:07 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: WitnessClientAdmin

Hi,

I transmitted IIS logs and message tracking logs from one server to the other through nxlog and saved it in a location. After complete transmission I stopped the nxlog service. I was able to understand that the transmission is complete and I downloaded the data and was working on it. suddenly within 10 minutes few files are lost under IIS and message tracking.

I do not know the reason behind this. Is it really possible? Can the data get lost after stopping the service? or something could have caused it? I'm trying to find out the root cause. Pls help

I have a directory with multiple sub directories where I need to get logs from.

Under my original plan for getting all files types in a single directory, I can get it to work by appending "\*.log" for instance at the end of the path, but unfortunately it gets every log file and not just the ones I am looking for.

Is there a way to have NxLog Community Edition search through sub directories to find the files?

This is my lab setup:

C:\users\admin\Desktop\testfiles\Server1\w3c1\test.log \w3c2\test.log \Server2\w3c1\test.log \w3c2\test.log

I've tried using these, but they don't work "C:\users\admin\Desktop\testfiles\\w3c\*.log" "C:\users\admin\Desktop\testfiles\*\w3c*\*.log"

We need it like something close to this because Server1 also has other logs that we don't want like firewall.log files.

Any advice would be great.

Thanks

Hi, I am following the CEF documentation from here https://community.microfocus.com/t5/ArcSight-Connectors/ArcSight-Common-Event-Format-CEF-Implementation-Standard/ta-p/1645557

In this its written that Severity is a string or integer and reflects the importance of the event. The valid string values are Unknown, Low, Medium, High, and Very-High. The valid integer values are 0-3=Low, 4-6=Medium, 7- 8=High, and 9-10=Very-High.

I am using xm_cef module to parse the CEF message and according to the above description if String values are provided they should be converted to appropriate Integer values or left as it is but the xm_cef module is converting the string values to 0.

eg:- Input:- CEF:0|Product|Example|unknown|1|Account|Very-High|dvchost=localhost duser=1234567 src=1.1.1.1 the severity value is Very-High

Output:- (the field CEF Severity is 0 ) { "EventReceivedTime":"2019-05-02T04:58:30.333756-07:00", "SourceModuleName":"cef_input", "SourceModuleType":"im_file", "SyslogFacilityValue":1, "SyslogFacility":"USER", "SyslogSeverityValue":5, "SyslogSeverity":"NOTICE", "SeverityValue":1, "Severity":"DEBUG", "EventTime":"2019-05-02T04:58:30.333784-07:00", "Hostname":"ubuntu", "SourceName":"CEF", "Message":"0|Product|Example|unknown|1|Account|Very-High|dvchost=localhost duser=1234567 src=1.1.1.1", "CEFVersion":0, "CEFDeviceVendor":"Product", "CEFDeviceProduct":"Example", "CEFDeviceVersion":"unknown", "CEFSignatureID":"1", "CEFName":"Account", "CEFSeverity":0, "dvchost":"localhost", "duser":"1234567", "src":"1.1.1.1" }

I have tried with Low, Medium,High,Very-High,it converts all of them to 0. I have used the conf file same as in https://nxlog.co/question/4618/xmcef-xmjson-unexpected-behaviour-while-converting-cef-json#comment-7579 NXlog version :- nxlog-4.3.4308-trial

There is no log output with respect to this

I'm using NXLog CE to forward Windows event logs via the im_msvistalog module. There's about 161 event id's that I want to whitelist from the security log and not send anything else from the event logs.

The following config snippet works: <Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='Security'>*[System[(EventID=4627)] or System[(EventID=4624)] or System[(EventID=4775)] or System[(EventID=4776)] or System[(EventID=4777)] or System[(EventID=4741)] or System[(EventID=4742)] or System[(EventID=4743)] or System[(EventID=4744)] or System[(EventID=4745)] or System[(EventID=4746)] or System[(EventID=4747)] or System[(EventID=4748)] or System[(EventID=4749)] or System[(EventID=4750)] or System[(EventID=4751)] or System[(EventID=4752)] or System[(EventID=4753)] or System[(EventID=4759)] or System[(EventID=4760)] or System[(EventID=4672)] or System[(EventID=4634)] or System[(EventID=4648)]] </Select> </Query> </QueryList> </QueryXML> </Input>

The issue is that once I add one more line to that config, NXLog stops shipping events completely.

Is there a better way for me to write this that would allow for more than 23 whitelisted event id's?

What are the current vulnerabilities associated with NXLog Common Edition, version 2.10.2150?

While evaluating the NXLOG enterprise trial edition, we faced a blocker and I need some clarification/help on the same.

We are using NXLOG’s CEF module (xm_cef, xm_json ) which convert’s CEF messages into JSON. It is working properly for most of the cases but giving unexpected output for few of them.

raw CEF message :- CEF:0|Himanshu Arora|Sample1|10.5.011|195|Process Sample|5|abc=Sample Data suser=XY fname= dvc= shost=10.1.1.1 dhost= duser= externalId= app= reason= cs1Label=Affected User cs1= cs2Label=Safe Name cs2=Notification Sample cs3Label=Device Sample cs3= cs4Label=Database cs4= cs5Label="Other info" cs5= cn1Label=Request Id cn1= cn2Label=Ticket Id cn2= msg= JSON output :- { "EventReceivedTime": "2019-04-25T13:43:49.483942+05:30", "SourceModuleName": "cef_input", "SourceModuleType": "im_file", "SyslogFacilityValue": 1, "SyslogFacility": "USER", "SyslogSeverityValue": 5, "SyslogSeverity": "NOTICE", "SeverityValue": 3, "Severity": "WARNING", "EventTime": "2019-04-25T13:43:49.483969+05:30", "Hostname": "himanshu-VirtualBox", "SourceName": "CEF", "CEFVersion": 0, "CEFDeviceVendor": "Himanshu Arora", "CEFDeviceProduct": "Sample1", "CEFDeviceVersion": "10.5.011", "CEFSignatureID": "195", "CEFName": "Process Sample", "CEFSeverity": 5, "abc": "Sample Data", "suser": "XY", "fname": "dvc=", "shost": "10.1.1.1", "dhost": "duser=" }

If you notice the raw message has some fields called cs1Label,cs2Label,cs2,cn1Label,cn2Label, cn2 . these fields are missing in the JSON output file. Moreover in JSON the fields "fname" , "dhost" should have had empty value.

I would like to know

1. If this issue exists only in the enterprise trial edition and it will be resolved if we purchase the Enterprise edition ? or is it issue being fixed and will be released soon?
2. Is there a way to include any third party libraries into NXLOG that can convert CEF to JSON.

Hi,

I have been trying to stream data and the data transfer was successful with a #015 appended to each line in my log file. This is happening to all the log types trasferred. Can you let me know what could cause that?

It appears that nxlog continually retries when trying to send a log. Is there a way to limit the amount of retries and continue on if it fails over x amount of times?

My issue is this, if the body of the request is bad, I will get a 400 as a result. This means, I could get stuck trying to send a bad message over and over.

I am trying to configure to capture windows 10 logs and it is displaying the following messages below:

2019-04-19 23:40:05 WARNING nxlog-ce received a termination request signal, exiting ... 2019-04-19 23:40:07 WARNING no functional input modules! 2019-04-19 23:40:07 WARNING no routes defined! 2019-04-19 23:40:07 WARNING not starting unused module out 2019-04-19 23:40:07 INFO nxlog-ce-2.10.2150 started

look how this nxlog.conf

Panic Soft #NoFreeOnExit TRUE

define ROOT C: \ Program Files (x86) \ nxlog define CERTDIR% ROOT% \ cert define CONFDIR% ROOT% \ conf define LOGDIR% ROOT% \ data define LOGFILE% LOGDIR% \ nxlog.log LogFile% LOGFILE%

Moduledir% ROOT% \ modules CacheDir% ROOT% \ data Pidfile% ROOT% \ data \ nxlog.pid SpoolDir% ROOT% \ data

<Extension _syslog>

Module xm_syslog

</ Extension>

<Extension gelf> Module xm_gelf </ Extension>

<Output out> Module om_tcp Host 192.168.1.48 Port 12201 #Exec to_syslog_snare (); OutputType GELF_TCP </ Output>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </ Extension>

<Extension _exec> Module xm_exec </ Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every 1 hour
    Exec if (file_exists ('% LOGFILE%') and \
               (file_size ('% LOGFILE%')> = 5M)) \
                file_cycle ('% LOGFILE%', 8);
</ Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When @weekly
    Exec if file_exists ('% LOGFILE%') file_cycle ('% LOGFILE%', 8);
</ Schedule>

</ Extension>

Hi,

I'm using EE trial edition now in my machine but i need to use CE edition as well for testing. Can I use both in same machine? will I lose my EE trial if i download CE now?

Is there a way to send multiple records in an http call?

I'd like to send multiple rows of my log file via an http call.

I am trying to set up an om_http output. I get the response of ERROR HTTP response status is not OK: 400 Bad Request

I need to trouble shoot what message I am actually sending. Is there an easy way to see what message is sent?

This is my in/out config. <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input in> Module im_file File '<scrubbed>/data.json' Exec $message = to_json(); </Input> <Output out> Module om_http URL <scrubbed> HTTPSCAFile <scrubbed> HTTPSCertFile <scrubbed> HTTPSCertKeyFile <scrubbed> ContentType application/vnd.kafka.v1+json </Output> <Route 1> Path in => out </Route>

Hi, i have recently setup graylog and i'm using nxlog as my collector, everything seems to be working fine except nxlog is not sending logs to the graylog server, i have checked nxlog logs and this is the error 2019-04-16 11:51:08 ERROR failed to open C:\Users\s.chimere\Desktop\GRAYLOG; Access is denied

C:\Users\s.chimere\Desktop\GRAYLOG this is where i have my test logs

Please can anyone help.

Hi everybody,

I use NXLog EE 4.3.4308. Time to time client is crushed with this error at module libapr-1-0.dll. Windows Application log:

2019-04-14 01:00:06 ERROR 1000 Faulting application name: nxlog.exe, version: 4.3.4308.0, time stamp: 0x00000000
Faulting module name: libapr-1-0.dll, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x00000000000084b9
Faulting process id: 0x197c
Faulting application start time: 0x01d4f112445554b9
Faulting application path: C:\nxlog\nxlog.exe
Faulting module path: C:\nxlog\libapr-1-0.dll
Report Id: e1a9c09a-a1e6-432d-8ecc-12d042258be8
Faulting package full name: 
Faulting package-relative application ID: 

Any ideas, please. Thanks!

NXLog's log before crushed:

2019-04-14 00:56:37 ERROR apr_stat() failed on file C:\logs\2018-12-14.log; Access is denied.  
2019-04-14 00:56:39 WARNING input file was deleted: C:\logs\2018-12-14.log

This is old log file was deleted in weekly log rotation. Why NXLog scanning old files? This file was not updated for 3 month.

Hi, I'm trying to parse a csv log file from my local and store it again in another location in my local. But I see it is not happening. The config has no errors. the destination file is same as source file with respect to file properties. Can you tell whether any specific check should be done?

<Extension csv_parser1> Module xm_csv Fields date-time,client-ip,client-hostname,server-ip,server-hostname,
source-context,connector-id,source,event-id,internal-message-id,message-id,recipient-address,
recipient-status,total-bytes,recipient-count,related-recipient-address,reference,message-subject,
sender-address,return-path,message-info,directionality,tenant-id,
original-client-ip,original-server-ip,custom-data Delimiter , </Extension>

Message Tracking log as input

<Input messagetracking> Module im_file File '%BASEDIR%file.log' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser1->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec> </Input>

<Output msg> Module om_file File 'location\msg.log' </Output>

<Route 1> Path messagetracking => msg </Route>

Can you explain what exactly happens in this block?
I don't get a clear explanation of what "xEF\xBB\xBF" means in the below code and why it is having drop ()? .

<Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec>

Hi,

I would like to know the cost of enterprise edition. Also, I would like to know whether we could use the purchased nxlog EE package in more than one server to collect logs?