Hi all, I have this config Panic Soft define ROOT C:\Program Files (x86)\nxlog define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension multilines> Module xm_multiline FixedLineCount 2 </Extension> <Input InputData> Module im_file File "C:\\txt\\event.txt" InputType multilines </Input> <Output OutputData> Module om_file File "C:\\txt\\txt1.log" </Output> <Route 1> Path InputData => OutputData </Route> And this input log file event1 Data1 event2 Data2 event3 Data3 event4 Data4 event5 Data5 event6 Data6 But output file is always empty and nxlog.log is without errors or warnings. I want to merge two lines into single line.

When I use im_file and om_odbc,from the log,I get this message:WARNING SQLExecDirect failed; HY000:1:0:[Devart][ODBC][PostgreSQL]在字段 "timestamp" 中空值违反了非空约束 <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log1.txt" ReadFromLast False SavePos False <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec> Exec parse_syslog(); </Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1; UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> #<Output out> Module om_file File "C:\Users\jiang.dengjie\Desktop\logtest.txt" Exec to_json(); #</Output> <Route r> Path in => out </Route> In my file: %Feb 5 15:47:32:118 2015 trust-access IFNET/5/LINK_UPDOWN: Line protocol on the interface GigabitEthernet1/0/41 is down. %Feb 5 15:47:35:367 2015 trust-access IFNET/3/PHY_UPDOWN: GigabitEthernet1/0/40 link status is up. And I want to use nxlog to save this file to my pgsql which has the table that has five colum:id,timestamp,hostname,eventname,event. Also if any viedo about how to use nxlog? Thank you very much.

I use im_file and om_file on windows,But throgh om_file,I get a file that is empty <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log.txt" Exec parse_syslog(); </Input> <Output out> Module om_file File "C:\Users\jiang.dengjie\Desktop\logtest.txt" Exec to_json(); </Output> <Route r> Path in => out </Route>

I use nxlog in windows,from the log,I get this question:ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib Then I find I do not have the agent-ca.pem. And in my environment,my pgsql do not get any data. below is my config. <Input in> Module im_file File "C:\Users\xxx\Desktop\log.txt" <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec> </Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1; UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> <Route r> Path in => out </Route>

Hi everyone! You many help me, thanks a lot. I hope you kind to help me now. My NXLog clients don't collect Windows System logs. And now I often see in my logs this message: 2019-06-04 17:49:50 INFO nxlog-4.3.4308 started 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. <QueryList> <Query Id='1'> <Select Path='System'>*</Select> </Query> </QueryList> <QueryList> <Query Id='1'> <Select Path='Application'>*</Select> </Query> </QueryList> 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events [error code: 1717]; The interface is unknown. My config: define ROOT C:\nxlog define NXLOGLOGFILE %ROOT%\data\nxlog.log define CERTDIR %ROOT%\cert PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval 0 CacheSync TRUE <Input winapp> Module im_msvistalog ReadFromLast TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='Application'>*</Select> </Query> </QueryList> </QueryXML> Exec $FileName = 'winapp.log'; Exec $EventTime = $EventReceivedTime; </Input> <Input winsys> Module im_msvistalog ReadFromLast TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='System'>*</Select> </Query> </QueryList> </QueryXML> Exec $FileName = 'winsys.log'; Exec $EventTime = $EventReceivedTime; </Input> <Output out> BufferSize 9500000 Module om_batchcompress Host 192.168.100.100 Port 1514 UseSSL true AllowUntrusted TRUE CAFile %CERTDIR%\cacert.pem CertFile %CERTDIR%\clientcert.pem CertKeyFile %CERTDIR%\clientkey.pem </Output> <Route client> Path winapp, winsys => out </Route> After restart service nothing new. Any ideas, please!

Hello, I am trying to send some logs into ELK and I am running into a bit of a snag. The logs are delimited by space and there doesn't seem to be an options to change that easily. I am not really sure how to go about getting the logs sent to my monitoring solution in a formatted way, preferably JSON. These are apache error logs: [Fri May 31 14:21:38 2019] [error] [client 1.1.1.1] File does not exist: /home/test/test.xml, referer: https://www.test-group.com/ NxLog conf: define REGEX /(?x)^[\S+\ ([^]]+)]\ [(\S+):(\S+)]\ ([client\ (\S+)]\ )?(.+)$/ <Extension multiline> Module xm_multiline HeaderLine %REGEX% </Extension> <Input in> Module im_file File &quot;C:\\path\\\*.log&quot; InputType multiline SavePos FALSE ReadFromLast FALSE &lt;Exec&gt; if $raw_event =~ /(?x)^\[\S+\ ([^\]]+)\]\ \[(\S+):(\S+)\]\ (\[client\ (\S+)\]\ )?(.+)$/ { $EventTime = parsedate($1); $ApacheModule = $2; $ApacheLogLevel = $3; $Message = $4; } &lt;/Exec&gt; It's sending the logs to ELK, but the data isn't in a usable format there. Everything looks just like it does in plain text, no fields or values. If I add the "exec to_json();" option, then I have empty logs in ELK. Maybe something is wrong with my regex, but I copied what I could from the reference manual for this log, though this log is missing data from the example in the guide. Thanks for your time

Hi, I'm trying to send json log files to a syslog server but it doesn't works. I can see the the tcp connection established with the syslog server but nothing is sent. Seems like the json file is not read. I will appreciate if someone can help me. Below, my nxlog.conf: define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogLevel DEBUG LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension json> Module xm_json </Extension> #locoal Server <Input in> Module im_file File 'k:\data\json\log\log2019.json*' Exec parse_json(); </Input> To syslog server <Output out> Module om_tcp Host x.x.x.x Port 514 </Output> <Route 1> Path in => out </Route>

I have a fresh install of CentOS 7 and NXLog EE. The Config File was restored from a previous install. The logs are coming into the system but do not appear to be sending out. The only events I see in the NXLog.Log file are related to the SSL Cert not being available for agent-ca.pem. 2019-05-30 16:51:27 INFO nxlog-4.4.4347 started 2019-05-30 16:51:27 ERROR SSL error, failed to load ca cert from '/opt/nxlog/var/lib/nxlog/cert/agent-ca.pem', reason: No such file or directory, no such file, system lib 2019-05-30 16:53:11 INFO configuration OK I would assume this is at the system level but am not sure what it might be. Anyone have any experience setting up and configuring NXLog on CentOS?

System Requirements: what are hardware and software requirements for Nxlog Enterprise edition. Available Integrations: what are the inputs supported for the tool(file, database, API, LDAP, etc...) Kafka Output Available: can we forward output to Kafka Agent Mechanism: With one agent how many logs we can transfer. is it one per system? Customizable: can we customize the code according to the requirements Price: Pricing details Can anyone help me to find these details anyone of the above?

Over the past couple weeks we've noticed high CPU usage on the nxlog service of up to 40%, average is around 25%. As a resolution, we have been stopping the nxlog service as well as the sysmon service, then starting them back up, and after about 10 minutes it finally drops back down to the normal cpu usage. We are using nxlog CE 2.10.2102 - has anyone else seen this issue? Will updating to the most current version resolve the issue? I appreciate any feedback/input! Thanks

Hi All, Im trying to get Microsoft NPS text file logs sent on to our firewall. I can get them forwarded if they are in .xml format as DTS compatible logs however I need them as IAS type files. I understand that there is a native parser for the Enterprise Edition and I have tried to obtain a quote form NXLog but as of yet not received any communication. Has anyone else out there achieved this with the CE version of nxlog and could they offer some advise on what the config would look like. Thanks :-)

Hi, is it possibe to resolve GUIDS in EventID:4624 Saw an other post from last year talking about it would be implemented i EE and after that CE. We currently running ce version and the option for ResolveSID dont work. Does anyone have a solution for this or a workaround as it looks ok in the Eventviewer where it's resolved correctly. We currently run nxlog-ce-2.9.1716.

I'm on an enterprise edition trial, we've current got logs being pushed into a time series database using fluentd but I'm also wanting to push the logs into IBM QRadar, but I'm struggling. Has anyone achieved this? Any assistance with config would be perfect. Thanks in advance.

There are many convenient string functions and procedures in the nxlog language, but I did not see one for urldecode. I have some data flowing through the web logs that the client is encoding, and it would be useful to have nxlog decode it. For example, something like: $data = urldecode("%61%6a%61%78%5f%66%6f%72%6d"); # $data = "ajax_form" In addition to requesting that functionality, is there any workarounds you could suggest?

Hi, We are planning to do a migration for nxlog manager, which involve a IP change for the nxlog manager. Are there any prerequisites to be done on the Manager before this can be done? I know that the agent's configurations will have to point to the new IP. So the plan would be to send them a new config with the manager's new IP, then change the nxlog manager to the new IP The nxlog manager is installed on Linux 7.5, running latest nxlog manager version 5.4.5209. Thank you.

I have nxlog 2.10.1542 on centos 7.6 I am sending rfc 3195 events to the nxlog for forward to a siem A snippet of the log that captures the error event is seen below. 2019-05-01 08:34:32 DEBUG evaluating expression 'field' at /opt/tap-nxlog/conf/nxlog.conf:128 2019-05-01 08:34:32 DEBUG evaluating expression 'field' at /opt/tap-nxlog/conf/nxlog.conf:129 2019-05-01 08:34:32 DEBUG successfully got priority 2019-05-01 08:34:32 DEBUG date is logver=600000267 timestamp=1556692474 tz="UTC+2" devname="FW-NCH-FGT600E-1" devid="<REDACATED>" vd="root" date=2019-05-01 time=08:34:34 logid="1059028705" type="utm" subtype="app-ctrl" eventtype="app-ctrl-all" level="warning" eventtime=1556692474 appid=40169 srcip=devid="<REDACATED> dstip=216.58.211.142 srcport=57244 dstport=443 srcintf="x2" srcintfrole="dmz" dstintf="x2" dstintfrole="dmz" proto=17 service="udp/443" direction="outgoing" policyid=3 sessionid=12692393 applist="sniffer-profile" appcat="Network.Service" app="QUIC" action="block" incidentserialno=1737302560 msg="Network.Service: QUIC," apprisk="low" 2019-05-01 08:34:32 DEBUG in nx_date_parse 2019-05-01 08:34:32 DEBUG in vpn parsing 2019-05-01 08:34:32 DEBUG if loop errored and return bad date 2019-05-01 08:34:32 DEBUG parse_vpn failed 2019-05-01 08:34:32 DEBUG not correct date From what I am seeing the date matches from the event and the debug logging. additionally the epoch time "Wednesday, May 1, 2019 6:34:34 AM" from the event also seems to match. Any thoughts would be appreciated. Thanks in advance Adam

Hi, I have updated the NXLOG installation to version 2.10.2150 hoping that it will fix my issue. I have NXlogs installed on 6 servers and they are work fine except one server. Thi sis afile server so there are no application installed on it. On this server I get the following errors. When I manually restart the NXLOG service everything runs fine for few days and then NXLOG stops reporting logs. I appreciate assistance in trying to resolve this issue. The log file located at C:\Program Files (x86)\nxlog\data contains this: 2019-05-07 10:23:44 INFO nxlog-ce-2.10.2150 started 2019-05-10 06:35:36 ERROR EvtNext failed with error 15007: The specified channel could not be found. Check channel configuration. 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='246'><Select Path='Microsoft-WindowsAzure-Diagnostics/Bootstrapper'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='247'><Select Path='Microsoft-WindowsAzure-Diagnostics/Heartbeat'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='248'><Select Path='Microsoft-WindowsAzure-Diagnostics/Runtime'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='249'><Select Path='Microsoft-WindowsAzure-Status/GuestAgent'></Select></Query> 2019-05-10 06:35:37 WARNING ignoring source as it cannot be subscribed to (error code: 15007): <Query Id='250'><Select Path='Microsoft-WindowsAzure-Status/Plugins'>*</Select></Query> 2019-05-10 06:35:43 ERROR EvtNext failed with error 1722: The RPC server is unavailable. 2019-05-10 06:35:43 ERROR Failed to query available channels; The RPC server is unavailable. 2019-05-12 08:31:07 WARNING received a system shutdown request 2019-05-12 08:31:07 WARNING stopping nxlog service 2019-05-12 08:31:07 WARNING nxlog-ce received a termination request signal, exiting... 2019-05-12 08:31:58 INFO nxlog-ce-2.10.2150 started 2019-05-12 08:32:07 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources. 2019-05-12 08:32:07 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: WitnessClientAdmin

Hi, I transmitted IIS logs and message tracking logs from one server to the other through nxlog and saved it in a location. After complete transmission I stopped the nxlog service. I was able to understand that the transmission is complete and I downloaded the data and was working on it. suddenly within 10 minutes few files are lost under IIS and message tracking. I do not know the reason behind this. Is it really possible? Can the data get lost after stopping the service? or something could have caused it? I'm trying to find out the root cause. Pls help

I have a directory with multiple sub directories where I need to get logs from. Under my original plan for getting all files types in a single directory, I can get it to work by appending "\*.log" for instance at the end of the path, but unfortunately it gets every log file and not just the ones I am looking for. Is there a way to have NxLog Community Edition search through sub directories to find the files? This is my lab setup: C:\users\admin\Desktop\testfiles\Server1\w3c1\test.log \w3c2\test.log \Server2\w3c1\test.log \w3c2\test.log I've tried using these, but they don't work "C:\users\admin\Desktop\testfiles\\w3c\*.log" "C:\users\admin\Desktop\testfiles\*\w3c*\*.log" We need it like something close to this because Server1 also has other logs that we don't want like firewall.log files. Any advice would be great. Thanks

Hi, I am following the CEF documentation from here https://community.microfocus.com/t5/ArcSight-Connectors/ArcSight-Common-Event-Format-CEF-Implementation-Standard/ta-p/1645557 In this its written that Severity is a string or integer and reflects the importance of the event. The valid string values are Unknown, Low, Medium, High, and Very-High. The valid integer values are 0-3=Low, 4-6=Medium, 7- 8=High, and 9-10=Very-High. I am using xm_cef module to parse the CEF message and according to the above description if String values are provided they should be converted to appropriate Integer values or left as it is but the xm_cef module is converting the string values to 0. eg:- Input:- CEF:0|Product|Example|unknown|1|Account|Very-High|dvchost=localhost duser=1234567 src=1.1.1.1 the severity value is Very-High Output:- (the field CEF Severity is 0 ) { "EventReceivedTime":"2019-05-02T04:58:30.333756-07:00", "SourceModuleName":"cef_input", "SourceModuleType":"im_file", "SyslogFacilityValue":1, "SyslogFacility":"USER", "SyslogSeverityValue":5, "SyslogSeverity":"NOTICE", "SeverityValue":1, "Severity":"DEBUG", "EventTime":"2019-05-02T04:58:30.333784-07:00", "Hostname":"ubuntu", "SourceName":"CEF", "Message":"0|Product|Example|unknown|1|Account|Very-High|dvchost=localhost duser=1234567 src=1.1.1.1", "CEFVersion":0, "CEFDeviceVendor":"Product", "CEFDeviceProduct":"Example", "CEFDeviceVersion":"unknown", "CEFSignatureID":"1", "CEFName":"Account", "CEFSeverity":0, "dvchost":"localhost", "duser":"1234567", "src":"1.1.1.1" } I have tried with Low, Medium,High,Very-High,it converts all of them to 0. I have used the conf file same as in https://nxlog.co/question/4618/xmcef-xmjson-unexpected-behaviour-while-converting-cef-json#comment-7579 NXlog version :- nxlog-4.3.4308-trial There is no log output with respect to this