Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
nxlog windows direct channel log capture issue
navdeepsingh83 created
Hi folks,
I am trying to capture analytics and debug logs on windows server 2012 r2. The logs are under applications and services log and the log path is Microsoft-Windows-DNSServer/Analytical
When I enable it, i get the following error
2018-09-19 16:52:23 ERROR failed to subscribe to msvistalog events using bookmark: The caller is trying to subscribe to a direct channel which is not allowed. The events for a direct channel go directly to a logfile and cannot be subscribed to.
2018-09-19 16:52:23 ERROR failed to subscribe to msvistalog events,the Query is invalid: [error code: 50]
I have tried few methods but none of them are working.
any idea how can we capture windows debug and analytics logs using nxlog?
navdeepsingh83 created
NXLOG Service failed
Shahmiri created
Dear all,
I'm trying to get hold of the IIS logs and I get the following issue when I try to restart the service.... we are working on a extra.conf file and I know that it is the one that hinders the service to start.... I just cant see where in the code I mess up.
Here's the code.
Created by NXlog Configuration AT 04-07-2018 08:20:12
NXlog Configuration Version 2018-05-14
Created On HOSTNAMEWEB03
OS INFO 2008 - nxlogserver: 10.233.26.20
dnsloginfo $Undefined DHCPLOGINFO $Undefined###
Start off with Definitions
Rootdir defined from: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\nxlog,installdir;HKEY_LOCAL_MACHINE\SOFTWARE\nxlog,installdir
define ROOT <C:\Program Files\nxlog>
Generic Settings for ALL installations
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %LOGFILE%
LogLevel INFO
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Extension _json>
Module xm_json
</Extension>
Define our inputs
Start ISS created by # 18-09-2018###
<Input IIS>
Module im_file
File C:\inetpub\logs\LogFiles\W3SVC1\*
SavePos True
InputType LineBased
</Input>
END ISS Inserted by # 18-09-2018###
<Input winlog>
Module im_msvistalog
ReadFromLast TRUE
ResolveSID TRUE
<QueryXML>
<QueryList>
<Query Id='1'>
<Select Path='Application'></Select>
<Select Path='Security'></Select>
<Select Path='System'>*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
Define the output that goes to LP for analysis
<Output syslogout>
Module om_tcp
Host 10.2XX.26.2X
Port 514
Exec to_syslog_bsd();
</Output>
<Output winout>
Module om_tcp
Host 10.2XX.26.2X
Port 514
Exec to_json(); $Message = $raw_event;to_syslog_bsd();
</Output>
Tie together inputs to outputs
<Route 1>
Path winlog => winout
</Route>
include %CONFDIR%\extra.conf
Configuration Completed
The following is taken out of the nxlog.log
2018-09-19 09:28:10 WARNING nxlog received a termination request signal, exiting...
nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86
Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf
The filename, directory name, or volume label syntax is incorrect.
2018-09-19 09:41:15 INFO nxlog-4.0.3735 started
2018-09-19 09:41:15 WARNING not starting unused module syslogout
2018-09-19 09:41:15 INFO connecting to 10.233.26.20:514
2018-09-19 09:44:00 WARNING stopping nxlog service
2018-09-19 09:44:00 WARNING nxlog received a termination request signal, exiting...
nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86
Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf
The filename, directory name, or volume label syntax is incorrect.
nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86
Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf
The filename, directory name, or volume label syntax is incorrect.
Shahmiri created
IM_ODBC always executes same query
w.schmitt@evidos.nl created
Hi i am trying to use the im_odbc module with nxlog-4.1.4016 running on a windows 2016 server. but it seems like i am in an loop.
I looked at the documentation and took notice of the required id field, but the output is always the same ID, as if the position (savepos ) is not saved.
sqlserver : 14.0.2002.14
this is my minimalized input config
<Input database>
Module im_odbc
SavePos TRUE
ConnectionString DSN=dbserver;database=db;UID=sa;Pwd=password;
SQL select Id as id from table where id = ?
Exec $Level = "INFO";
</Input>
this is my output config
<Output test>
Module om_file
File "D:\_Data\log_null_output.log"
</Output>
this is the output of the file
2018-09-18 16:39:49 adss-dbserver INFO
id: 26335
2018-09-18 16:39:50 adss-dbserver INFO
id: 26335
2018-09-18 16:39:51 adss-dbserver INFO
id: 26335
2018-09-18 16:39:52 adss-dbserver INFO
id: 26335
debug logging nxlog
2018-09-18 17:26:14 DEBUG CONFIG: adss_sql
2018-09-18 17:26:14 DEBUG SQL: select Id as id from SigningTransactionLogs where id = ?
2018-09-18 17:26:14 DEBUG nx_expr_new_field: Level
2018-09-18 17:26:14 DEBUG field declared at line 9, character 12 in C:\Program Files\nxlog\conf\inputs\adss_sql.in
2018-09-18 17:26:14 DEBUG left_value expr
2018-09-18 17:26:14 DEBUG adding string [INFO]
2018-09-18 17:26:14 DEBUG string literal declared at line 9, character 21 in C:\Program Files\nxlog\conf\inputs\adss_sql.in
2018-09-18 17:26:14 DEBUG literal
2018-09-18 17:26:14 DEBUG assignment declared at line 9, character 22 in C:\Program Files\nxlog\conf\inputs\adss_sql.in
2018-09-18 17:26:14 DEBUG assignment: left_value = expr
2018-09-18 17:26:14 DEBUG statement: assignment
2018-09-18 17:26:14 DEBUG finished parsing statements
....
2018-09-18 17:26:16 DEBUG new event in event_thread [adss_sql:READ]
2018-09-18 17:26:16 DEBUG nx_event_to_jobqueue: READ (adss_sql)
2018-09-18 17:26:16 DEBUG event added to jobqueue
2018-09-18 17:26:16 DEBUG future event, event thread sleeping -697014233us in cond_timedwait
2018-09-18 17:26:16 DEBUG worker 0 got signal for new job
2018-09-18 17:26:16 DEBUG worker 0 processing event 0x164caf0
2018-09-18 17:26:16 DEBUG PROCESS_EVENT: READ (adss_sql)
2018-09-18 17:26:16 DEBUG odbc read
2018-09-18 17:26:16 DEBUG im_odbc_execute with last id 0
2018-09-18 17:26:16 DEBUG resultset column: id (bigint identity)
2018-09-18 17:26:16 DEBUG fetching from id 0
2018-09-18 17:26:16 DEBUG NO DATA
2018-09-18 17:26:16 DEBUG worker 0 waiting for new event
2018-09-18 17:26:16 DEBUG new event in event_thread [_fileop:SCHEDULE]
2018-09-18 17:26:16 DEBUG new event in event_thread [_fileop:SCHEDULE]
2018-09-18 17:26:16 DEBUG new event in event_thread [adss_sql:READ]
2018-09-18 17:26:16 DEBUG future event, event thread sleeping 1000000us in cond_timedwait
2018-09-18 17:26:17 DEBUG event thread wait time is up (after 1000000us)
2018-09-18 17:26:17 DEBUG new event in event_thread [_fileop:SCHEDULE]
2018-09-18 17:26:17 DEBUG new event in event_thread [_fileop:SCHEDULE]
2018-09-18 17:26:17 DEBUG new event in event_thread [adss_sql:READ]
2018-09-18 17:26:17 DEBUG nx_event_to_jobqueue: READ (adss_sql)
2018-09-18 17:26:17 DEBUG event added to jobqueue
2018-09-18 17:26:17 DEBUG future event, event thread sleeping -698029889us in cond_timedwait
2018-09-18 17:26:17 DEBUG worker 1 got signal for new job
2018-09-18 17:26:17 DEBUG worker 1 processing event 0x164c910
2018-09-18 17:26:17 DEBUG PROCESS_EVENT: READ (adss_sql)
2018-09-18 17:26:17 DEBUG odbc read
2018-09-18 17:26:17 DEBUG im_odbc_execute with last id 0
2018-09-18 17:26:17 DEBUG resultset column: id (bigint identity)
2018-09-18 17:26:17 DEBUG fetching from id 0
2018-09-18 17:26:17 DEBUG NO DATA
w.schmitt@evidos.nl created
Update to TLS 1.3
dk created
Hello!
Are there plans for an update of NXLog Community Edition to integrate TLS 1.3 (or rather OpenSSL 1.1.1)?
Unfortunately I didn't find any information about this. If this question has already been asked, I would be thankful if anyone could show me this thread.
Regards
dk created
NXLOG CE running on Windows 2016
omoanth created
Running NXLOG on Windows 2016
Error failed to subscribe to msvistalog events, the channel was not found [error code: 15007], the specific channel could not be found. check channel configuration
NXLOG config file as requested.
Sample of NXLOG configuration file
This is a sample configuration file. See the nxlog reference manual about the
configuration options. It should be installed locally and is also available
online at http://nxlog.org/docs/
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_msvistalog
For windows 2003 and earlier use the following:
Module im_mseventlog
Query <QueryList>
<Query Id="0">
<Select Path="Application"></Select>
<Select Path="System"></Select>
<Select Path="Security"></Select>
<Select Path="ForwardedEvents"></Select>
<Select Path="Setup"></Select>
<Select Path="Microsoft-Windows-Sysmon/Operational"></Select>
</Query>
</QueryList>
</Input>
<Output out>
Module om_tcp
Host x.x.x.x
Port 514
Exec to_syslog_snare();
</Output>
<Route 1>
Path in => out
</Route>
omoanth created
Adding custom header (om_http)
piro314 created
Hi,
According to this documentation https://nxlog.co/documentation/nxlog-user-guide#om_http
I can use 'AddHeader' to put my custom headers to http request.
However, when I put it in config i get: ERROR invalid keyword: AddHeader
I also tried
Exec add_http_header('Application-Id', 'b1f8b7a0-5cc5-11e8-8230-0db3d3bfb10d');
This time error is: procedure 'add_http_header()' does not exist or takes different arguments.
What's the correct way to add a http header"?
piro314 created
XML file giving "no element found" and not well-formed" errors
Deleted user created
Hello,
As the name entices, I am getting multiple errors with XML files.
Here is my conf:
<Extension xml>
Module xm_xml
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in>
Module im_file
File "C:\Users\administrator\Desktop\2016.xml"
SavePos FALSE
ReadFromLast FALSE
Exec parse_xml();
Exec to_json();
</Input>
<Output out>
Module om_file
File "C:\Users\administrator\Desktop\testxml.txt"
</Output>
Here are the errors:
2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20
in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
orted; XML parse error at line 1: no element found
2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20
in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab
orted; XML parse error at line 1: not well-formed (invalid token)
Thanks for your time.
Deleted user created
NXLog with Logstash using custom TAGS
NXlog_monitoring created
Hi,
I'm trying to send messages from NXLog into Logstash with a custom TAG. Logstash would filter those messages and then send them into specific topics in Kafka. For example my current Logstash + Filebeats works like that:
filebeat.yml has:
paths:
- /var/log/*.log
tags: ["EXAMPLE_1"]
Logstash.yml has :
output {
if "EXAMPLE_1" in [tags]{
kafka {
bootstrap_servers => "example_dns:9092"
topic_id => "example_1_topic_kafka"
}
}
}
Is it possible to recreate such simple config with NXLog?
NXlog_monitoring created
Some Windows events are missing their EventData section
AmirG created
Hello,
I'm using NXlog CE 2.10.2102 on a Win 2012 R2 x64 server to collect both the four default Windows logs and the Forwarded Events snd send to a Syslog server as Snare formatted. However, some events only contains their System segment, missing their entire EventData. For example, all of events 1000 and 1001 and all 4624 events with Kerberos login. 4624 with Advapi are passed just fine. I've no idea why is that, every idea would be welcomed.
Here's my configuration:
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
LogLevel INFO
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="ForwardedEvents"></Select>
<Select Path="Application"></Select>
<Select Path="System"></Select>
<Select Path="Security"></Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
$Message =~ s/(\t|\R)/ /g; to_syslog_snare();
</Exec>
</Input>
<Output out>
Module om_udp
Host 1.2.3.4
</Output>
<Route 66>
Path in => out
</Route>
AmirG created
output module file - CacheSize
luke.taucher created
In https://nxlog.co/documentation/nxlog-user-guide#om_file_config it mentions;
"In case of dynamic filenames, a cache can be utilized to keep files open. This increases performance by reducing the overhead caused by many open/close operations. It is recommended to set this to the number of expected files to be written. Note that this should not be set to more than the number of open files allowed by the system. This caching provides performance benefits on Windows only. Caching is disabled by default."
However in https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#om_file - there is no mention of this function, and I can't seem to configure it on my current nxlog.conf as it throws
2018-09-04 01:44:13 ERROR invalid om_file keyword: CacheSize at C:\Program Files (x86)\nxlog\conf\nxlog.conf:131
2018-09-04 01:44:13 ERROR module 'out' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:144
2018-09-04 01:44:13 ERROR route 1 is not functional without output modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:144
Any ideas? is this something only available in enterprise edition?
luke.taucher created
Nxlog taking MC Afee event and send it to another monitoring server
Maxime.Tremblay created
Hi, i have an NXLOG server installed and i want it to take the antivirus event to another monitoring server. Do you know wich command i had to enter to take this log and push it to another server ?
Maxime.Tremblay created
NXLOG im_ODBC module to collect data from table of MSSQL
Sam_wang created
Hello,
I have set up windows 2012R2+SQL2014 environment to verify the im_obdc function, now it is working to gather data from database, but there are two problems encountered.
1. How can I limit the rows to fetch from table every time ? if there is no limitation, it will affect the database performance at the first time .
2. I have a ID in SQL statement , and followed the instruction to have "where id > ? " in SQL statement. but nxlog always execute the same SQL statement and return same data to nxlog agent.
The following are configuration of input and SQL statement from SQL profiler
Configuration of input
==================
<Input in>
Module im_odbc
ConnectionString DSN=SEPM;uid=sem5;pwd=Admin123;database=sem5
#MaxIdSQL SELECT MAX(TIME_STAMP) from V_ALERTS
#ReadFromLast True
SQL SELECT a.USN, a.ALERT_IDX, a.SOURCE, a.VIRUSNAME_IDX, a.NOOFVIRUSES, a.FILEPATH, a.DESCRIPTION, a.ACTUALACTION_IDX, a.REQUESTEDACTION_IDX, a.ALERTDATETIME, a.USER_NAME, a.SOURCE_COMPUTER_NAME, a.SOURCE_COMPUTER_IP, a.TIME_STAMP, a.SOURCE_COMPUTER_IP_TEXT, v.VIRUSNAME, v.TYPE, PAT.VERSION, PAT.SEQUENCE, LOWER(S.NAME), LOWER(G.NAME), LOWER(P.NAME), Q.NAME, I.COMPUTER_DOMAIN_NAME, I.COMPUTER_NAME, I.CURRENT_LOGIN_USER, I.CURRENT_LOGIN_DOMAIN, I.IP_ADDR1_TEXT, I.MAC_ADDR1, I.OS_LANG, I.DISK_TOTAL, I.MEMORY, I.OPERATION_SYSTEM, I.SERVICE_PACK, I.BIOS_VERSION, SA.AGENT_VERSION, SA.AGENT_TYPE, SA.PROFILE_VERSION, SA.STATUS, SA.LAST_UPDATE_TIME, SA.INFECTED, SA.WORSTINFECTION_IDX, SA.LAST_VIRUS_TIME, SA.LAST_SCAN_TIME, SA.LAST_DOWNLOAD_TIME, SA.CONTENT_UPDATE, SA.PROFILE_SERIAL_NO, SA.MAJOR_VERSION, SA.LICENSE_STATUS, SA.LICENSE_EXPIRY FROM V_ALERTS a with (NOLOCK) LEFT JOIN VIRUS v ON a.VIRUSNAME_IDX = v.VIRUSNAME_IDX LEFT JOIN V_SEM_COMPUTER I ON I.COMPUTER_ID = a.COMPUTER_IDX LEFT JOIN SEM_AGENT SA ON I.COMPUTER_ID = SA.COMPUTER_ID LEFT JOIN IDENTITY_MAP S ON SA.DOMAIN_ID = S.ID LEFT JOIN IDENTITY_MAP G ON SA.GROUP_ID = G.ID LEFT JOIN IDENTITY_MAP P ON SA.LAST_SERVER_ID = P.ID LEFT JOIN IDENTITY_MAP Q ON SA.LAST_SITE_ID = Q.ID LEFT JOIN PATTERN PAT ON SA.PATTERN_IDX = PAT.PATTERN_IDX WHERE a.TIME_STAMP > ? ORDER BY a.TIME_STAMP
#SavePos True
PollInterval 30
</Input>
SQL Statement from Profiler
=======================
exec sp_executesql N'SELECT a.USN, a.ALERT_IDX, a.SOURCE, a.VIRUSNAME_IDX, a.NOOFVIRUSES, a.FILEPATH, a.DESCRIPTION, a.ACTUALACTION_IDX, a.REQUESTEDACTION_IDX, a.ALERTDATETIME, a.USER_NAME, a.SOURCE_COMPUTER_NAME, a.SOURCE_COMPUTER_IP, a.TIME_STAMP, a.SOURCE_COMPUTER_IP_TEXT, v.VIRUSNAME, v.TYPE, PAT.VERSION, PAT.SEQUENCE, LOWER(S.NAME), LOWER(G.NAME), LOWER(P.NAME), Q.NAME, I.COMPUTER_DOMAIN_NAME, I.COMPUTER_NAME, I.CURRENT_LOGIN_USER, I.CURRENT_LOGIN_DOMAIN, I.IP_ADDR1_TEXT, I.MAC_ADDR1, I.OS_LANG, I.DISK_TOTAL, I.MEMORY, I.OPERATION_SYSTEM, I.SERVICE_PACK, I.BIOS_VERSION, SA.AGENT_VERSION, SA.AGENT_TYPE, SA.PROFILE_VERSION, SA.STATUS, SA.LAST_UPDATE_TIME, SA.INFECTED, SA.WORSTINFECTION_IDX, SA.LAST_VIRUS_TIME, SA.LAST_SCAN_TIME, SA.LAST_DOWNLOAD_TIME, SA.CONTENT_UPDATE, SA.PROFILE_SERIAL_NO, SA.MAJOR_VERSION, SA.LICENSE_STATUS, SA.LICENSE_EXPIRY FROM V_ALERTS a with (NOLOCK) LEFT JOIN VIRUS v ON a.VIRUSNAME_IDX = v.VIRUSNAME_IDX LEFT JOIN V_SEM_COMPUTER I ON I.COMPUTER_ID = a.COMPUTER_IDX LEFT JOIN SEM_AGENT SA ON I.COMPUTER_ID = SA.COMPUTER_ID LEFT JOIN IDENTITY_MAP S ON SA.DOMAIN_ID = S.ID LEFT JOIN IDENTITY_MAP G ON SA.GROUP_ID = G.ID LEFT JOIN IDENTITY_MAP P ON SA.LAST_SERVER_ID = P.ID LEFT JOIN IDENTITY_MAP Q ON SA.LAST_SITE_ID = Q.ID LEFT JOIN PATTERN PAT ON SA.PATTERN_IDX = PAT.PATTERN_IDX WHERE a.TIME_STAMP > @P1 ORDER BY a.TIME_STAMP',N'@P1 bigint',2664642240
Thanks in advanced.
Sam_wang created
Unable to send csv logs to graylog udp using nxlog
navdeepsingh83 created
Hi Everyone,
I have copied jira access logs, filter it using csv and put under a directory as csv file.
I am using below nxconf which gets loaded and nxlog service gets started, however no logs are send to graylog server.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension json>
Module xm_json
</Extension>
<Extension fileop>
Module xm_fileop
</Extension>
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension gelf>
Module xm_gelf
</Extension>
<Extension jira>
Module xm_csv
Fields $IPAddress,$UserName,$DateTime,$HTTPAction,$ResponseCode,$Column10,$Column11
FieldTypes string,string,string,string,string,string,string
Delimiter","
</Extension>
<Input in>
Module im_file
File "C:\Users\jira\Documents\TempOut\JiraAccessLogs\accessLog.csv"
#ReadFromLast False
#Recursive True
#SavePos True
<Exec>
if $raw_event =~ /^#/ drop();
else
{
jira->parse_csv();
to_json();
}
</Exec>
</Input>
<Output out>
Module om_udp
Host 172.17.1.87
Port 5046
OutputType GELF_UDP
Exec $short_message = $raw_event; # Avoids truncation of the short_message field.
Exec $collector_node_id = 'SINNB0094';
Exec $Hostname = hostname_fqdn();
#Use the following line for debugging (uncomment the fileop extension above as well)
Exec file_write("C:\Users\\jira\\Documents\\TempOut\\JiraAccessLogs\\nxlog-debug.log", $raw_event);
</Output>
<Route 1>
Path in => out
</Route>
if i replace, output section with below and send the data to txt/log file, i can see the logs converted to json and written.
Module om_file
File "C:\nxlog-debug.txt"
Ii suspect there is some issue with out with om_udp but since i don’t see any error or warning, it’s difficult to troubleshoot.
can you spot what is wrong with the out?
navdeepsingh83 created
Output is literally in another language.
Deleted user created
lol so yeah my output is in another language??
I am running an XML input of data and trying to get it into an easy format to use for Elasitc. I followed the manual the best I could here https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_multiline_example_5 but my output is crazy.
##NxLog conf file##
<Extension multiline>
Module xm_multiline
HeaderLine /^\s*<Obj RefId="[0-9][0-9]?[0-9]?[0-9]?">/
</Extension>
<Extension _xml>
Module xm_xml
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Input in3>
Module im_file
File "C:\Users\administrator\Desktop\newtest.xml"
InputType multiline
SavePos FALSE
ReadFromLast FALSE
Exec parse_xml();
Exec to_json();
</Input>
<Output out3>
Module om_file
File "C:\Users\administrator\Desktop\testxml.txt"
</Output>
<Route>
Path in3 => out3
</Route>
##End conf##
##Data sample##
<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
<Obj RefId="12">
<TN RefId="4">
<T>System.Diagnostics.EventLogEntry#System/Microsoft-Windows-Kernel-General/16</T>
<T>System.Diagnostics.EventLogEntry#System/Microsoft-Windows-Kernel-General</T>
<T>System.Diagnostics.EventLogEntry</T>
<T>System.ComponentModel.Component</T>
<T>System.MarshalByRefObject</T>
<T>System.Object</T>
</TN>
<ToString>System.Diagnostics.EventLogEntry</ToString>
<Props>
<S N="MachineName">testserver</S>
<BA N="Data" />
<I32 N="Index">23749</I32>
<S N="Category">(0)</S>
<I16 N="CategoryNumber">0</I16>
<I32 N="EventID">16</I32>
<Obj N="EntryType" RefId="13">
<TNRef RefId="1" />
<ToString>Information</ToString>
<I32>4</I32>
</Obj>
<S N="Message">The description for Event ID '16' in Source 'Microsoft-Windows-Kernel-General' cannot be found. The local computer may not have the necessary registry information or message DLL files to display the message, or you may not have permission to access them. The following information is part of the event:'109', '??\C:\Users\testaccount\AppData\Local\Packages\Microsoft.Windows.Cortana_cw5n1h2txyewy\Settings\settings.dat', '12', '4'</S>
<S N="Source">Microsoft-Windows-Kernel-General</S>
<Obj N="ReplacementStrings" RefId="14">
<TNRef RefId="2" />
<LST>
<S>109</S>
<S>??\C:\Users\testaccount\AppData\Local\Packages\Microsoft.Windows.Cortana_cw5n1h2txyewy\Settings\settings.dat</S>
<S>12</S>
<S>4</S>
</LST>
</Obj>
<I64 N="InstanceId">16</I64>
<DT N="TimeGenerated">2018-08-14T08:32:50-04:00</DT>
<DT N="TimeWritten">2018-08-14T08:32:50-04:00</DT>
<S N="UserName">testaccount</S>
<Nil N="Site" />
<Nil N="Container" />
</Props>
<MS>
<I32 N="EventID">16</I32>
</MS>
</Obj>
</Objs>
##End Sample##
##Output##
<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">
㰀伀戀樀 刀攀昀䤀搀㴀∀㈀∀㸀ഀഀ
<TN RefId="4">
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀⼀吀一㸀ഀഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索ഀ
<Props>
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀伀戀樀 一㴀∀䔀渀琀爀礀吀礀瀀攀∀ 刀攀昀䤀搀㴀∀㌀∀㸀ഀഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索ഀ
</Obj>
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀伀戀樀 一㴀∀刀攀瀀氀愀挀攀洀攀渀琀匀琀爀椀渀最猀∀ 刀攀昀䤀搀㴀∀㐀∀㸀ഀഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索ഀ
<LST>
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀⼀䰀匀吀㸀ഀഀ
</Obj>
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀⼀倀爀漀瀀猀㸀ഀഀ
<MS>
≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ
㰀⼀䴀匀㸀ഀഀ
</Obj>
㰀⼀伀戀樀猀㸀ഀ
##End Output##
Deleted user created
Can Nxlog Enterprise Edition 3.1.1930 work with MSSQL 2014 ?
Sam_wang created
Hello,
We have Nxlog Enterprise Edition 3.1.1930 to collect log from MSSQL 2014 via im_odbc module, but it return the following error.
"ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)"
The related configuration as below:
<Input in>
Module im_odbc
ConnectionString DSN=SymantecEndpointSecurityDSN;database=sem5;
SQL SELECT IDX as ID,ALERT_IDX as AlertID, COMPUTER_IDX as ComputerID,SOURCE as SRC,VIRUSNAME_IDX as virusname FROM V_ALERTS WHERE IDX > ?
</Input>
Thanks for any help~
Sam_wang created
Filter type in windows event log
anvers created
Hi All, I wanted to see if I could get help with a filter for windows events. I want to get all the "security" events except those that have a word with .tmp inside, For example:
ObjectName
F:\Personal\Battista\14FC4253.tmp
With the user help in chat i try with this config but doesnt work:
This is a sample configuration file. See the nxlog reference manual about the
configuration options. It should be installed locally and is also available
online at http://nxlog.org/docs/
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _gelf>
#Module xm_syslog
Module xm_gelf
#Module xm_json
</Extension>
<Input in1>
Module im_msvistalog
Uncomment the following to collect specific event logs only
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
<Exec>
if $Channel == 'Security' and $ObjectName =~ /.tmp/ drop();
</Exec>
</Input>
<Output out>
Module om_udp
Host 172.20.5.32
Port 12201
#Exec to_syslog_snare();
OutputType GELF
</Output>
<Route 1>
Path in1 => out
</Route>
THanks !
anvers created
Does Nxlog Enterprise Edition 3.1.1930 work on windows 2012R2 ?
Sam_wang created
I tried to install nxlog enterprise edition 3.1.1930 on windows 2012 R2,this agent can be installed with error message, but the nxlog could not read anything. any suggestion? many thanks!
Sam_wang created
Does the enterprise edition nxlog keep working when the licence is expired.
Sam_wang created
Our company had nxlog enterprise license , but we only paid for one year, Can we use nxlog with this old version when the license is expired.? many thanks!
Sam_wang created
Upgrade from nxlog-ce-2.9 to nxlog-ce-2.10
micsnare created
Hi,
I would like to upgrade from version nxlog-ce-2.9.1716-1.x86_64 to the current nxlog version 2.10
my biggest worry is that there would be massive syntax changes so that my current config/setup is no longer working with the new version.
so, could you please tell me what has changed from 2.9.1716-1 to the current version?
best regards,
micsnare
micsnare created
NXlog CE Seems to be Dropping many messages related to Microsoft DNS Debug log.
BenC created
I have NXLog CE latest version monitoring both Windows Event Logs and the DNS Debug log file on Server 2012 R2 and sending to TCP GELF format to a Graylog server.
I'm seeing periodic significant discrepancies between the actual amount of logs generated vs the logs that are being sent and received in my central logging platform (Graylog). from 10,000 messages per minute to 150 messages per minute, when it happens. I have verified this by getting a local copy of the dnsdebug log and checking the amount of lives vs running a query for the same time period in Graylog.
I see this drops specifically against the DNS_Debug file not against the Windows EVTX file, they come through at a normal rate.
Does anyone see anything wrong with the configuration below?
Are there Debug sources for NXlog that can be reviewed to see if internal errors are being generated?
I'm also seeing NXlog send messages with blank short_message and full_message.
Example
{"version":"1.1","_EventReceivedTime":"2018-08-16 16:36:51","_SourceModuleName":"DNS_Debug","_SourceModuleType":"im_file","host":"<Hostname>","short_message":"","full_message":"","timestamp":1534401411,"level":6}
This generates errors on the Graylog as well as an invalid input against a mandatory field in the GELF specification.
In terms of volume we're talking approximately 500,000 messages per 30mins.
Version : nxlog-ce-2.10.2102.msi
NB: We had to use [Exec $ShortMessage = $raw_event;] because by default the short_message field was coming through as 64 character truncated, which appears to be a default configuration.
Any help community would be greatly appreciated.
Next course of action is to set the internal logging to debug and check out what is happening.
Title: nxlog_winsrv
Version: 0.1
OS: Server 2012R2
For any concerns please contact [REDACTED]
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension gelf>
Module xm_gelf
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Input evtx_in>
Module im_msvistalog
Query <QueryList>
<Query Id="0">
<Select Path="Setup"></Select>
<Select Path="System"></Select>
<Select Path="Security">*</Select>
</Query>
</QueryList>
</Input>
<Input dnsDebug_in>
Module im_file
File "C:\dns_debug.txt"
SavePos TRUE
Exec $ShortMessage = $raw_event;
</Input>
<Output default_out>
Module om_tcp
Host [HOSTNAME}
Port [Host_Port]
OutputType GELF_TCP
</Output>
<Route default_route>
Path evtx_in, dnsDebug_in => default_out
</Route>
BenC created