i want to send my atp log to syslog server with the help of nxlog . i am storing atplog to atp.log file and the data is in json format. I am using below configuration but i am not getting anything on my syslog server which is AV. configuration: # Configuration for converting and sending Windows logs # to AlienVault USM Anywhere. # define ROOT C:\Program Files (x86)\nxlog define OUTPUT_DESTINATION_ADDRESS 10.0.2.4 define OUTPUT_DESTINATION_PORT 514 Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> <input atplogfile> Module im_atplogfile </input> <Input ATPLogs_IN> Module im_atplogfile File "C:\temp\log\atplogs-.log" Exec $Message = to_json(); to_syslog_bsd(); </Input> <Output ATPLogs_OUT> Module om_atplogfile File "/var/log/nxlog/syslog.log" Exec to_syslog_bsd(); </Output> ######################################## # Routes # ######################################## <Route 1> Path ATPLogs_IN => ATPLogs_OUT </Route> <Input eventlog> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security"></Select> </Query> </QueryList> Exec if ($EventID == 5156) OR ($EventID == 5158) drop(); </Input> <Output out> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route 1> Path eventlog, internal => out </Route>

Hello, I have recently been trying up a syslog-ng server for various devices and have tried a couple of things for sending Windows Events to the server. Finally decieded that NXLog will do what I need and I have gotten sent some events over without much configuration, but when trying filter within the .conf file, it always fails. I can't really find much good information as to why it might be failing, as it seems that it should be correct.(to me anyway) # Windows Event Log, <Input s_eventlog> Module im_msvistalog Exec if $EventID == 4734 or $EventID == 4624 drop(); Exec $Message = to_json(); </Input> I have narrowed it down to this block, since the log says nxlog failed to start: </Input> without matching <Input> section at C:\Program Files (x86)\nxlog\conf\nxlog.conf:43 Which is where this block ends? I can't really make sense of this, so if anyone has some guidance please tell me.

Getting error: Opsec error. rc=-1 err=-96 Connection error step 4 on the user guide page 706 anyone run into issues pulling certs or suggestions? I can telnet using the port no issues, but not able to pull certs. steps prior were all completed.

Hi, I have the following windows log message which i am trying to parse and remove the \r,\t and \n from the log. with i am using i am unable to parse it and it still shows me the log content as is? input: ACBDEFG 12/03/2015 09:05:13 AM\r\nLogName=Security\r\nSourceName=Microsoft Windows security auditing.\r\nEventCode=4672\r\nEventType=0\r\nType=Information\r\nComputerName=VDEEXCP01.teckcominco.loc\r\nTaskCategory=Special Logon\r\nOpCode=Info\r\nRecordNumber=3259542776\r\nKeywords=Audit Success\r\nMessage=msg_somethinh.\r\r\n\r\r\nSubject:\r\r\n\tSecurity ID:\t\tABCDEFFHG-12345\r\r\n\tAccount Name:\t\ABEDCEDDDD$\r\r\n\tAccount Domain:\t\tABCXDDFFEEFFFF\r\r\n\tLogon ID:\t\t98665svdvdvdv\r\r\n\r\r\nPrivileges:\t\tSeSecurityPrivilege\r\r\n\t\t\tSeBackupPrivilege\r\r\n\t\t\tSeRestorePrivilege\r\r\n\t\t\tSeTakeOwnershipPrivilege\r\r\n\t\t\tSeDebugPrivilege\r\r\n\t\t\tSeSystemEnvironmentPrivilege\r\r\n\t\t\tSeLoadDriverPrivilege\r\r\n\t\t\tSeImpersonatePrivilege\r\r\n\t\t\tSeDelegateSessionUserImpersonatePrivilege output: ACBDEFG 12/03/2018 10:15:13 AM\r\nLogName=Security\r\nSourceName=Microsoft Windows security auditing.\r\nEventCode=4672\r\nEventType=0\r\nType=Information\r\nComputerName=VDEEXCP01.teckcominco.loc\r\nTaskCategory=Special Logon\r\nOpCode=Info\r\nRecordNumber=3259542776\r\nKeywords=Audit Success\r\nMessage=msg_somethinh.\r\r\n\r\r\nSubject:\r\r\n\tSecurity ID:\t\tABCDEFFHG-12345\r\r\n\tAccount Name:\t\ABEDCEDDDD$\r\r\n\tAccount Domain:\t\tABCXDDFFEEFFFF\r\r\n\tLogon ID:\t\t98665svdvdvdv\r\r\n\r\r\nPrivileges:\t\tSeSecurityPrivilege\r\r\n\t\t\tSeBackupPrivilege\r\r\n\t\t\tSeRestorePrivilege\r\r\n\t\t\tSeTakeOwnershipPrivilege\r\r\n\t\t\tSeDebugPrivilege\r\r\n\t\t\tSeSystemEnvironmentPrivilege\r\r\n\t\t\tSeLoadDriverPrivilege\r\r\n\t\t\tSeImpersonatePrivilege\r\r\n\t\t\tSeDelegateSessionUserImpersonatePrivilege I am using the community edition and trying out to parse using the following configuration of nxlog. can you please suggest any changes needed in the conf? Global section User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO #Extension section <Extension multi> Module xm_multiline HeaderLine /^================/ EndLine /^---------------/ </Extension> Modules section <Input in> Module im_tcp HOST 0.0.0.0 Port 1532 #InputType multi # Remove the boundary markers Exec if $raw_event =~ s/========[=]+//g {} Make a single line Exec if $raw_event =~ s/[\r\n]/ /g {} #remove the end directive Exec if $raw_event =~ s/--------[-]+//g {} Exec if $raw_event =~ /^ *$/ drop(); Exec $raw_event = replace($raw_event, &quot;\t&quot;, &quot; &quot;); </Input> <Output out> Module om_file File '/tmp/output' #Exec $raw_event = "--------------------------------------\n" + $raw_event; </Output> Route section <Route r> Path in => out </Route>

I'm trying to set up nxlog with om_ssl using a 3rd party cert. I can connect without issue using openssl s_client, but am getting a certificate error in nxlog. local:~$ openssl s_client -connect logs.notmyrealdomain.com:514 CONNECTED(00000003) depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority verify return:1 depth=1 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Domain Validation Secure Server CA verify return:1 depth=0 OU = Domain Control Validated, OU = PositiveSSL Wildcard, CN = *.notmyrealdomain.com verify return:1 Certificate chain 0 s:/OU=Domain Control Validated/OU=PositiveSSL Wildcard/CN=*.notmyrealdomain.com i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA 1 s:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA i:/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority If I don't specify a CAfile in nxlog.conf, I get one error: 2019-02-11 12:19:55 ERROR SSL certificate verification failed: unable to get local issuer certificate (err: 20) Module om_ssl Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% CAFile %ROOT%\cert\ca.pem If I do specify a CAfile with the same intermediate cert as is on the server, I get a different error: 2019-02-11 12:22:24 ERROR SSL certificate verification failed: unable to get issuer certificate (err: 2) Module om_ssl Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% CAFile %ROOT%\cert\ca.pem "AllowUntrusted TRUE" does not help, but shouldn't be needed Thanks in advance for any help!

hi all, not sure if you can help me, but I recently installed the latest (beta) version of nxlog 4-99.4527 to test the multiple evtx files support. my config looks like this: &lt;Input eventlog&gt; Module im_msvistalog File &quot;C:\Users\test\Desktop\logs\*.evtx&quot; &lt;/Input&gt; &lt;Output file_from_eventlog&gt; Module om_file File &quot;C:\logs\evtx_new.log&quot; Exec to_json(); &lt;/Output&gt; while it perfectly outputs to the local file in JSON output :), i still get the following error in the nxlog.log 2019-02-01 15:33:01 ERROR failed to query msvistalog events from file (C:\Users\test\Desktop\logs\Microsoft-Windows-SettingSyncOperational.evtx),[error code: 1287]; Zur Bestimmung der Fehlerursache stehen nicht gen&uuml;gend Informationen zur Verf&uuml;gung. I'm sorry that the error message is in german but roughly translated it means "Not enough information is available to determine the cause of the error." any ideas what could cause this error? many thanks in advance, theresa

I am trying to build NxLog Comminity edition on Centos 6.10 w/ X64_86 target and found there are five dependencies packages - pr-util.x86_64 apr-util-devel.x86_64 pcre-devel.x86_64 openssl-devel.x86_64 perl-ExtUtils-Embed. Does anyone know where to download the source code for these 5 dependent packages and how to build under centos 6.10 environment? Thanks.

I am trying to process the logs collected from MacOX kernel using the method described on page 251 of the guide, using /usr/bin/log. I pretty much copied the entire code block, including xm_multiline and xm_json modules. The problem I am running into is that parse_json throws an error for a very specific message and I don't understand what it doesn't like about it. It ONLY complains about this event, while successfully processing others. First the error: 2019-02-06 11:47:19 ERROR failed to parse json string, parse error: invalid object key (must be a string); pple.duetactivityscheduler", }; (right here) ------^; [{; "category" : "scoring",; "processImageUUID" : "C8A8C46F-C86F-3DCE-AA7F-5128FEC80F95",; "processUniqueID" : 104,; "threadID" : 322877,; "timestamp" : "2019-02-06 11:47:19.133462-0500",; "traceID" : 1264507192737796,; "messageType" : "Default",; "senderProgramCounter" : 143258,; "processID" : 104,; "machTimestamp" : 10284775415304436,; "timezoneName" : "",; "subsystem" : "com.apple.duetactivityscheduler",;}] Seems like it only complains about the events with subsystem of "com.apple.duetactivityscheduler. I was able to view one of those events and nothing looks out of the ordinary for me: { "category" : "lifecycle", "processImageUUID" : "C8A8C46F-C86F-3DCE-AA7F-5128FEC80F95", "processUniqueID" : 104, "threadID" : 324554, "timestamp" : "2019-02-06 11:54:38.540442-0500", "traceID" : 1258391159308292, "messageType" : "Default", "senderProgramCounter" : 49785, "processID" : 104, "machTimestamp" : 10285214841060059, "timezoneName" : "", "subsystem" : "com.apple.duetactivityscheduler", "eventMessage" : "CANCELED: 0:com.apple.timed.ntp.needed:B67ADB <private>!", "senderImageUUID" : "B94808C8-E8F6-349C-9CCF-2C2C3EF30A24", "processImagePath" : "/usr/libexec/dasd", "senderImagePath" : "/System/Library/DuetActivityScheduler/Scheduler/DuetActivitySchedulerDaemon.bundle/Contents/MacOS/DuetActivitySchedulerDaemon" }

Hi, I am monitoring few log file on a windows 2012 server using im_file Send them to an ELK cluster When checking the indexed logs we noticed delays ( 30 sec or more) for some ( not all) The only clue I have found is a bunch of this type of error 2019-01-28 14:50:01 DEBUG future event, event thread sleeping 34361ms in cond_timedwait Any idea what that means and how to troubleshoot it ?? Many thanks Steven

I setup nxlog on windows 2008 , windows 2008 R2 , windows 2012 and windows 2016. windows 2008 and 2008 R2 nxlog have some issue with connection with syslog server while 2012 and 2016 works perfectly fine. > nxlog log file 2019-01-31 22:06:31 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host. 2019-01-31 22:06:32 INFO connecting to &lt;some loadbalancer IP&gt; 2019-01-31 22:24:57 INFO reconnecting in 1 seconds 2019-01-31 22:24:58 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-01-31 22:41:51 INFO reconnecting in 1 seconds 2019-01-31 22:41:52 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-02-01 00:45:43 INFO reconnecting in 1 seconds 2019-02-01 00:45:44 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-02-01 01:00:56 INFO reconnecting in 1 seconds 2019-02-01 01:00:56 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host. 2019-02-01 01:00:56 INFO reconnecting in 2 seconds 2019-02-01 01:00:57 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-02-01 01:19:06 WARNING received a system shutdown request 2019-02-01 01:19:06 WARNING stopping nxlog service 2019-02-01 01:19:06 WARNING nxlog-ce received a termination request signal, exiting... 2019-02-01 01:19:42 INFO nxlog-ce-2.10.2150 started 2019-02-01 01:19:42 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-02-01 01:20:09 INFO reconnecting in 1 seconds 2019-02-01 01:20:09 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host. 2019-02-01 01:20:10 INFO connecting to &lt;some loadbalancer IP&gt;:514 2019-02-01 01:20:13 WARNING received a system shutdown request 2019-02-01 01:20:13 WARNING stopping nxlog service 2019-02-01 01:20:13 WARNING nxlog-ce received a termination request signal, exiting... 2019-02-01 01:20:47 INFO nxlog-ce-2.10.2150 started 2019-02-01 01:20:47 INFO connecting to &lt;some loadbalancer IP&gt;o:514 2019-02-01 02:03:05 INFO reconnecting in 1 seconds 2019-02-01 02:03:05 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host. 2019-02-01 02:03:06 INFO connecting to &lt;some loadbalancer IP&gt;:514 > Configuration file define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogLevel INFO &lt;Extension _syslog&gt; Module xm_syslog &lt;/Extension&gt; &lt;Input eventlog&gt; Module im_msvistalog &lt;QueryXML&gt; &lt;QueryList&gt; &lt;Query Id='0'&gt; &lt;Select Path='Application'&gt;*&lt;/Select&gt; &lt;Select Path='Security'&gt;*&lt;/Select&gt; &lt;Select Path='System'&gt;*&lt;/Select&gt; &lt;/Query&gt; &lt;/QueryList&gt; &lt;/QueryXML&gt; &lt;/Input&gt; &lt;Output tcp&gt; Module om_tcp Host &lt;Load Balancer IP&gt; Port 514 Exec to_syslog_snare(); &lt;/Output&gt; &lt;Route 1&gt; Path eventlog =&gt; tcp &lt;/Route&gt; What could be the issue? Is there anything more to be added in 2008 and 2008R2?

So we have several systems/appliances that only send to one location. However, we have a need to send logs to more than one location the issue is that the logs are sent in LEEF format and one system uses LEEF and the other system uses CEF. I know NXLog will do the multiple sending however, will it also convert the logs it is sending? I am asking this because we were told the Snare Central Server could do it and found out that it cannot do it without the agents installed. Clearly you cannot install agents on an appliance so before we go to the trouble of trying to setup and build out an NXLog server we need to know if this type of thing is possible.

I want to send the window event log generated from the normal PC to the graylog. My first plan was to install sysmon and send it to graylog, but I had difficulty with the transport part So I got to know nxlog. I need the Windows event log from sysmon, which is the Windows security log. Can I check this in nxlog? Thank you for your guide. And I want to know the difference between nxlog and sysmon log

I am attempting to use xm_ifileop to rotate some logfiles I am collecting with nxlog. I can see that rotation works as expected if I specify the file path but can I use the same logic to rotate all files in directory. Example: /var/log/osquery/ on linux/mac and C:\ProgramData\osquery\log on windows has 3 files in it that start with osdqueryd. and I want to watch those and rotate them if they get over 3M. I have tried on windows and Mac to use a * in the file path to specify the directory: define OSQLOGFILE C:\\ProgramData\\osquery\\log\\osqueryd.\* but that doesn't rotate the log. If I specify each file by name then it works as expected but then I need 3 xm_fileop sections. Is there an easy way to tell nxlog to rotate all files matching a pattern? Here is my logic so far: <Extension osquery_fileop> Module xm_fileop # Check the log file size every hour and rotate if larger than 3 MB <Schedule> Every 1 hour Exec if (file_exists('%OSQLOGFILE%') and (file_size('%OSQLOGFILE%') >= 1M)) file_cycle('%OSQLOGFILE%', 4); </Schedule> </Extension>

Is it possible to manipulate the ID in SQL queries done by NXLog agent? The documentation states this in im_dbi module: The module automatically appends a WHERE id > ? LIMIT 10 clause to the statement. The result set returned by the SELECT statement must contain an id column which is then stored and used for the next query. I have a database that does not get new reqords very often so i want to reset that id for testing and development purposes in the receiving end.

Hi, I have installed community version on Linux 18.04 AMD 64 and I encountered problem with stopping NX Log service. When I hit "sudo service nxlog stop", the command process but the service still persist. When I hit "sudo service nxlog status", the result is: ● nxlog.service - LSB: logging daemon Loaded: loaded (/etc/init.d/nxlog; generated) Active: failed (Result: exit-code) since Sun 2019-01-27 16:50:34 CET; 33s ago Docs: man:systemd-sysv-generator(8) Process: 8744 ExecStart=/etc/init.d/nxlog start (code=exited, status=1/FAILURE) Tasks: 7 (limit: 2321) CGroup: /system.slice/nxlog.service └─30012 /usr/bin/nxlog led 27 16:50:34 logstash-VirtualBox systemd[1]: Starting LSB: logging daemon... led 27 16:50:34 logstash-VirtualBox nxlog[8744]: * Starting nxlog daemon... led 27 16:50:34 logstash-VirtualBox nxlog[8744]: 2019-01-27 16:50:34 ERROR Another instance is already running (pid 30012);Resource temporarily unavailable led 27 16:50:34 logstash-VirtualBox nxlog[8744]: Failed to start nxlog! led 27 16:50:34 logstash-VirtualBox nxlog[8744]: ...fail! led 27 16:50:34 logstash-VirtualBox systemd[1]: nxlog.service: Control process exited, code=exited status=1 led 27 16:50:34 logstash-VirtualBox systemd[1]: nxlog.service: Failed with result 'exit-code'. led 27 16:50:34 logstash-VirtualBox systemd[1]: Failed to start LSB: logging daemon. Could you please advice me how I can stop it and run it again? Is the service working right when I do "sudo service nxlog status"? Thank you for all your advice. Kind regards, Marek

I have my NXlog server successfully receiving rsyslog messages from client Linux boxes. Now I'm trying to have NXlog forward those messages to my Graylog server using GELF. NXlog and Graylog are both running on CentOS 7 If anyone can point me in the right direction config file wise I'd be greatly appreciative.

Hello all, I'm currently running NXLog Enterprise in Version nxlog-4.0.3550-x64 with the following config: <Input eventlog> Module im_msvistalog File C:\logs\Security.evtx </Input> <Input application> Module im_msvistalog File C:\logs\Application.evtx </Input> Trying to read-in from 2 local evtx files. In the nxlog.log I see the following error: 2019-01-21 14:34:33 ERROR ### ASSERTION FAILED at line 1945 in im_msvistalog.c/im_msvistalog_start(): "((nx_im_msvistalog_subscr_t **)(imconf->q_subs->elts))[imconf->q_subs->nelts-1]->query = imconf->_query" ### 2019-01-21 14:34:36 ERROR last message repeated 4 times 2019-01-21 14:34:36 ERROR ### ASSERTION FAILED at line 1945 in im_msvistalog.c/im_msvistalog_start(): "((nx_im_msvistalog_subscr_t **)(imconf->q_subs->elts))[imconf->q_subs->nelts-1]->query = imconf->_query" ### Do you know what I'm doing wrong here? From what I've read in the manual, the enterprise edition should be able to read evtx files. best regards, micsnare

With the following config file I am capturing the entire MS DNS logs. This includes the DNS header info which I want to filter out. I need help figuring out what I can add that will allow me to filter out the DNS header information. define TAP_Sender_IP XXX.XXX.XXX.XXX define TAP_Sender_Port XXX define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO <Extension _syslog> Module xm_syslog </Extension> <Input DNS> Module im_file File "C:\DNSlogs.txt" SavePos True <Exec> if ($raw_event =~ /^#/) OR ($raw_event == '') drop(); else { to_syslog_bsd(); } </Exec> </Input> <Output Tap> Module om_udp Host %TAP_Sender_IP% Port %TAP_Sender_Port% </Output> <Route primary> Path DNS => Tap </Route> This config removes the blank spaces between DNS entries but leaves the file header. I'm Not sure what I need to change to prevent this from capturing the DNS header information. Does anyone have any suggestions Thanks in Advance Dags

Hello, I am still very new with nxlog, and currently, I am very stuck. I need to configure nxlog to pick up application files (currently we are only receiving standard windows logs). For testing purposes, I have installed and configured nxlog on my test machine. We are using extra.conf to avoid making changes in nxlog config itself, which we install on the server automatically. Here is my configuration which works fine on my test machine, but on the real application server, it only produces an empty file. Can it be related to the server itself, or am I just missing something? Logs are not showing any error messages. <Extension xmlparser> Module xm_xml </Extension> <Extension multiline_1> Module xm_multiline HeaderLine /^<Message>/ EndLine /^</Message>/ </Extension> <Input timmsg> Module im_file File 'C:\Users\Administrator\Desktop\msg.log' SavePos FALSE ReadFromLast FALSE InputType multiline_1 <Exec> Parse the xml event parse_xml(); # Rewrite some fields $EventTime = parsedate($timestamp); delete($timestamp); delete($EventReceivedTime); # Convert to JSON to_json(); &lt;/Exec&gt; </Input> Define the output that goes to LogPoint for analysis <Output timout> Module om_file File "C:\Users\administrator\Desktop\1.txt" </Output> Tie together inputs to outputs <Route 2> Path timmsg => timout </Route>

Here at ABB Facts we have been using solarwinds forwarding client with Kiwi but the client is showing its age and no longer works properly. I have set up nxlog as a replacement forwarder and it works fine but the priority is now set to Debug on all the windows events instead of Notice or Info as it was with the Solarwinds forwarder. My conf file is very simple and shouldnt cause this problem. I have searched for a solution but without any luck. Here is my conf file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> <Input eventlog> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="System"></Select> <Select Path="Security"></Select> </Query> </QueryList> </Input> <Output out> Module om_udp Host 10.250.254.19 Port 514 </Output>