Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.

Problems sending Windows Eventlog to graylog
Hello everybody, I'm sorry to bother you with another question concerning Windows Eventlog forwarding to graylog. Unfortunately I'm not able to figure this out on my own. used versions: nxlog 2.10.2102 (running on Windows Server 2016) graylog 2.4.6 (running on Debian 9) I have two nxlog setups. One using syslog and another one using GELF. Both do not work as I would expect. **1. Syslog** ``` Module xm_syslog Module im_msvistalog Exec delete($Keywords); Exec if ($EventType == "VERBOSE") drop(); Module om_tcp Host graylog Port 5140 Exec $raw_event = replace($raw_event, "\n", " "); Exec $raw_event = replace($raw_event, "\r", " "); Exec $raw_event = replace($raw_event, "\t", " "); Exec to_syslog_ietf(); Path eventlog => out_graylog ``` The problem is that there are eventlog entries containing line breaks. Unfortunately they are not removed by the replace commands. So in graylog one message is split into many messages with every linebreak. Using wireshark I can observe that the linebreaks consist of LF characters (Unix line endings). **2. Gelf** ``` Module xm_gelf Module im_msvistalog Exec delete($Keywords); Exec if ($EventType == "VERBOSE") drop(); Module om_tcp Host graylog Port 12201 OutputType GELF Path eventlog => out_graylog ``` Unfortunately this setup does not work at all. No messages are showing up in Graylog (of course I've activated the correspnding input). Using wireshark I can observe that a lot of TCP packets are sent to graylog but none of them contain readable messages. Can anybody help me with either setup? Thanks and Regards, Carsten

c.scharfenberg created
Replies: 1
View post »
last updated
input file does not exist

hi,

I'm working on monitoring a log file using nxlog. I have the File set to "C:\Program Files\test1.log" but it's saying that the "input file does not exist". I tried running a python script to check the file using the os module

import os

test = os.listdir('C:\Program Files\test1.log') print(test)

This will return an error "FileNotFoundError: The system cannot find the path specified"

I noticed that this error has been encountered before but none of the solutions I tried work.

any help is much appreciated.

Thanks, skawt


skawt created
Replies: 1
View post »
last updated
How to setting om_http custom timeout?

Hello,

is the any way to set custom timeout in om_http? or custom retry mechanism?

Thanks


wisnu.sudarmadi created
Replies: 1
View post »
last updated
Windows event ID not forwarded and problem with control characters

Hi. I am having an issue with forwarding event logs from a centralized server to an rsyslog and indexed in splunk. The logs are forwarded but the Event ID (the most important part) is missing. I am also having an issue with control characters on , this however could be blamed on rsyslog, but as I understand it the issue with control characters could be solved in the nxlog config.

Anyone care to give me a nudge in the correct way here?

//Thx


ryssland created
Replies: 2
View post »
last updated
convert field containig xml to json

i am getting data from a database, one of these fields containts an xml, is it possible to convert this single field to json?

sample data { "id": 27101, "ResponseStatus": "SUCCESS", "RequestTime": "2018-09-19 14:21:48", "ResponseXml": "<?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?>\r\n<Envelope xmlns=&quot;http://schemas.xmlsoap.org/soap/envelope/&quot;><Header /><Body><from>Jani</from></Body></Envelope>\r\n", "RequestMode": "DSS", "ErrorCode": null, }

i want the ResponseXml field to be converted to json aswell, i also want to keep the other fields

or any other sollution to parse the xml so i have access to the data inside the xml

thx!


w.schmitt@evidos.nl created
Replies: 1
View post »
last updated
nxlog windows direct channel log capture issue

Hi folks,

I am trying to capture analytics and debug logs on windows server 2012 r2. The logs are under applications and services log and the log path is Microsoft-Windows-DNSServer/Analytical

When I enable it, i get the following error

2018-09-19 16:52:23 ERROR failed to subscribe to msvistalog events using bookmark: The caller is trying to subscribe to a direct channel which is not allowed. The events for a direct channel go directly to a logfile and cannot be subscribed to. 2018-09-19 16:52:23 ERROR failed to subscribe to msvistalog events,the Query is invalid: [error code: 50]

I have tried few methods but none of them are working.

any idea how can we capture windows debug and analytics logs using nxlog?


navdeepsingh83 created
Replies: 1
View post »
last updated
NXLOG Service failed

Dear all,

I'm trying to get hold of the IIS logs and I get the following issue when I try to restart the service.... we are working on a extra.conf file and I know that it is the one that hinders the service to start.... I just cant see where in the code I mess up.

Here's the code.

Created by NXlog Configuration AT 04-07-2018 08:20:12

NXlog Configuration Version 2018-05-14

Created On HOSTNAMEWEB03

OS INFO 2008 - nxlogserver: 10.233.26.20

dnsloginfo $Undefined DHCPLOGINFO $Undefined###

Start off with Definitions

Rootdir defined from: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\nxlog,installdir;HKEY_LOCAL_MACHINE\SOFTWARE\nxlog,installdir

define ROOT <C:\Program Files\nxlog>

Generic Settings for ALL installations

define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %LOGFILE% LogLevel INFO

<Extension _syslog> Module xm_syslog </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _json> Module xm_json </Extension>

Define our inputs

Start ISS created by # 18-09-2018###

<Input IIS> Module im_file File C:\inetpub\logs\LogFiles\W3SVC1\* SavePos True InputType LineBased </Input>

END ISS Inserted by # 18-09-2018###

<Input winlog> Module im_msvistalog ReadFromLast TRUE ResolveSID TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='Application'></Select> <Select Path='Security'></Select> <Select Path='System'>*</Select> </Query> </QueryList> </QueryXML> </Input>

Define the output that goes to LP for analysis

<Output syslogout> Module om_tcp Host 10.2XX.26.2X Port 514 Exec to_syslog_bsd(); </Output>

<Output winout> Module om_tcp Host 10.2XX.26.2X Port 514 Exec to_json(); $Message = $raw_event;to_syslog_bsd(); </Output>

Tie together inputs to outputs

<Route 1> Path winlog => winout </Route> include %CONFDIR%\extra.conf

Configuration Completed

The following is taken out of the nxlog.log

2018-09-19 09:28:10 WARNING nxlog received a termination request signal, exiting... nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86 Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf The filename, directory name, or volume label syntax is incorrect.

2018-09-19 09:41:15 INFO nxlog-4.0.3735 started 2018-09-19 09:41:15 WARNING not starting unused module syslogout 2018-09-19 09:41:15 INFO connecting to 10.233.26.20:514 2018-09-19 09:44:00 WARNING stopping nxlog service 2018-09-19 09:44:00 WARNING nxlog received a termination request signal, exiting... nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86 Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf The filename, directory name, or volume label syntax is incorrect.

nxlog failed to start: Invalid 'include' directive at C:\Program Files\nxlog\conf\extra.conf:86 Failed to open config file <C:\Program Files\nxlog>\conf\extra.conf The filename, directory name, or volume label syntax is incorrect.


Shahmiri created
Replies: 1
View post »
last updated
IM_ODBC always executes same query
Hi i am trying to use the im_odbc module with nxlog-4.1.4016 running on a windows 2016 server. but it seems like i am in an loop. I looked at the documentation and took notice of the required id field, but the output is always the same ID, as if the position (savepos ) is not saved. sqlserver : 14.0.2002.14 **this is my minimalized input config** Module im_odbc SavePos TRUE ConnectionString DSN=dbserver;database=db;UID=sa;Pwd=password; SQL select Id as id from table where id = ? Exec $Level = "INFO"; **this is my output config** Module om_file File "D:\\_Data\\log_null_output.log" **this is the output of the file** 2018-09-18 16:39:49 adss-dbserver INFO id: 26335 2018-09-18 16:39:50 adss-dbserver INFO id: 26335 2018-09-18 16:39:51 adss-dbserver INFO id: 26335 2018-09-18 16:39:52 adss-dbserver INFO id: 26335 **debug logging nxlog** 2018-09-18 17:26:14 DEBUG CONFIG: adss_sql 2018-09-18 17:26:14 DEBUG SQL: select Id as id from SigningTransactionLogs where id = ? 2018-09-18 17:26:14 DEBUG nx_expr_new_field: Level 2018-09-18 17:26:14 DEBUG field declared at line 9, character 12 in C:\Program Files\nxlog\conf\inputs\adss_sql.in 2018-09-18 17:26:14 DEBUG left_value expr 2018-09-18 17:26:14 DEBUG adding string [INFO] 2018-09-18 17:26:14 DEBUG string literal declared at line 9, character 21 in C:\Program Files\nxlog\conf\inputs\adss_sql.in 2018-09-18 17:26:14 DEBUG literal 2018-09-18 17:26:14 DEBUG assignment declared at line 9, character 22 in C:\Program Files\nxlog\conf\inputs\adss_sql.in 2018-09-18 17:26:14 DEBUG assignment: left_value = expr 2018-09-18 17:26:14 DEBUG statement: assignment 2018-09-18 17:26:14 DEBUG finished parsing statements .... 2018-09-18 17:26:16 DEBUG new event in event_thread [adss_sql:READ] 2018-09-18 17:26:16 DEBUG nx_event_to_jobqueue: READ (adss_sql) 2018-09-18 17:26:16 DEBUG event added to jobqueue 2018-09-18 17:26:16 DEBUG future event, event thread sleeping -697014233us in cond_timedwait 2018-09-18 17:26:16 DEBUG worker 0 got signal for new job 2018-09-18 17:26:16 DEBUG worker 0 processing event 0x164caf0 2018-09-18 17:26:16 DEBUG PROCESS_EVENT: READ (adss_sql) 2018-09-18 17:26:16 DEBUG odbc read 2018-09-18 17:26:16 DEBUG im_odbc_execute with last id 0 2018-09-18 17:26:16 DEBUG resultset column: id (bigint identity) 2018-09-18 17:26:16 DEBUG fetching from id 0 2018-09-18 17:26:16 DEBUG NO DATA 2018-09-18 17:26:16 DEBUG worker 0 waiting for new event 2018-09-18 17:26:16 DEBUG new event in event_thread [_fileop:SCHEDULE] 2018-09-18 17:26:16 DEBUG new event in event_thread [_fileop:SCHEDULE] 2018-09-18 17:26:16 DEBUG new event in event_thread [adss_sql:READ] 2018-09-18 17:26:16 DEBUG future event, event thread sleeping 1000000us in cond_timedwait 2018-09-18 17:26:17 DEBUG event thread wait time is up (after 1000000us) 2018-09-18 17:26:17 DEBUG new event in event_thread [_fileop:SCHEDULE] 2018-09-18 17:26:17 DEBUG new event in event_thread [_fileop:SCHEDULE] 2018-09-18 17:26:17 DEBUG new event in event_thread [adss_sql:READ] 2018-09-18 17:26:17 DEBUG nx_event_to_jobqueue: READ (adss_sql) 2018-09-18 17:26:17 DEBUG event added to jobqueue 2018-09-18 17:26:17 DEBUG future event, event thread sleeping -698029889us in cond_timedwait 2018-09-18 17:26:17 DEBUG worker 1 got signal for new job 2018-09-18 17:26:17 DEBUG worker 1 processing event 0x164c910 2018-09-18 17:26:17 DEBUG PROCESS_EVENT: READ (adss_sql) 2018-09-18 17:26:17 DEBUG odbc read 2018-09-18 17:26:17 DEBUG im_odbc_execute with last id 0 2018-09-18 17:26:17 DEBUG resultset column: id (bigint identity) 2018-09-18 17:26:17 DEBUG fetching from id 0 2018-09-18 17:26:17 DEBUG NO DATA

w.schmitt@evidos.nl created
Replies: 1
View post »
last updated
Update to TLS 1.3

Hello!

Are there plans for an update of NXLog Community Edition to integrate TLS 1.3 (or rather OpenSSL 1.1.1)?

Unfortunately I didn't find any information about this. If this question has already been asked, I would be thankful if anyone could show me this thread.

Regards


dk created
Replies: 1
View post »
last updated
NXLOG CE running on Windows 2016

Running NXLOG on Windows 2016

Error failed to subscribe to msvistalog events, the channel was not found [error code: 15007], the specific channel could not be found. check channel configuration

NXLOG config file as requested.

Sample of NXLOG configuration file

This is a sample configuration file. See the nxlog reference manual about the

configuration options. It should be installed locally and is also available

online at http://nxlog.org/docs/

Please set the ROOT to the folder your nxlog was installed into,

otherwise it will not start.

#define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension _syslog> Module xm_syslog </Extension>

<Input in> Module im_msvistalog

For windows 2003 and earlier use the following:

Module im_mseventlog

Query <QueryList>
<Query Id="0">
<Select Path="Application"></Select>
<Select Path="System">
</Select>
<Select Path="Security"></Select>
<Select Path="ForwardedEvents">
</Select>
<Select Path="Setup"></Select>
<Select Path="Microsoft-Windows-Sysmon/Operational">
</Select>
</Query>
</QueryList> </Input>

<Output out> Module om_tcp Host x.x.x.x Port 514 Exec to_syslog_snare(); </Output>

<Route 1> Path in => out </Route>


omoanth created
Replies: 1
View post »
last updated
Adding custom header (om_http)

Hi, According to this documentation https://nxlog.co/documentation/nxlog-user-guide#om_http I can use 'AddHeader' to put my custom headers to http request. However, when I put it in config i get: ERROR invalid keyword: AddHeader

I also tried

 Exec 				add_http_header('Application-Id', 'b1f8b7a0-5cc5-11e8-8230-0db3d3bfb10d');

This time error is: procedure 'add_http_header()' does not exist or takes different arguments.

What's the correct way to add a http header"?


piro314 created
Replies: 1
View post »
last updated
XML file giving "no element found" and not well-formed" errors

Hello,

As the name entices, I am getting multiple errors with XML files.

Here is my conf: <Extension xml> Module xm_xml </Extension>

<Extension json> Module xm_json </Extension>

<Input in> Module im_file File "C:\Users\administrator\Desktop\2016.xml" SavePos FALSE ReadFromLast FALSE Exec parse_xml(); Exec to_json(); </Input>

<Output out> Module om_file File "C:\Users\administrator\Desktop\testxml.txt" </Output>

Here are the errors: 2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab orted; XML parse error at line 1: no element found 2018-09-10 15:58:04 ERROR procedure 'parse_xml' failed at line 33, character 20 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been ab orted; XML parse error at line 1: not well-formed (invalid token)

Thanks for your time.


Deleted user created
Replies: 2
View post »
last updated
NXLog with Logstash using custom TAGS

Hi, I'm trying to send messages from NXLog into Logstash with a custom TAG. Logstash would filter those messages and then send them into specific topics in Kafka. For example my current Logstash + Filebeats works like that:

filebeat.yml has:

paths: - /var/log/*.log

tags: ["EXAMPLE_1"]

Logstash.yml has :

output { if "EXAMPLE_1" in [tags]{ kafka { bootstrap_servers => "example_dns:9092" topic_id => "example_1_topic_kafka" } } }

Is it possible to recreate such simple config with NXLog?


NXlog_monitoring created
Replies: 1
View post »
last updated
Some Windows events are missing their EventData section

Hello,

I'm using NXlog CE 2.10.2102 on a Win 2012 R2 x64 server to collect both the four default Windows logs and the Forwarded Events snd send to a Syslog server as Snare formatted. However, some events only contains their System segment, missing their entire EventData. For example, all of events 1000 and 1001 and all 4624 events with Kerberos login. 4624 with Advapi are passed just fine. I've no idea why is that, every idea would be welcomed.

Here's my configuration:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO

<Extension _syslog> Module xm_syslog </Extension>

<Input in> Module im_msvistalog <QueryXML> <QueryList>
<Query Id="0"> <Select Path="ForwardedEvents"></Select> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security"></Select> </Query> </QueryList> </QueryXML> <Exec> $Message =~ s/(\t|\R)/ /g; to_syslog_snare(); </Exec> </Input>

<Output out> Module om_udp Host 1.2.3.4 </Output>

<Route 66> Path in => out </Route>


AmirG created
Replies: 4
View post »
last updated
output module file - CacheSize

In https://nxlog.co/documentation/nxlog-user-guide#om_file_config it mentions;

"In case of dynamic filenames, a cache can be utilized to keep files open. This increases performance by reducing the overhead caused by many open/close operations. It is recommended to set this to the number of expected files to be written. Note that this should not be set to more than the number of open files allowed by the system. This caching provides performance benefits on Windows only. Caching is disabled by default."

However in https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#om_file - there is no mention of this function, and I can't seem to configure it on my current nxlog.conf as it throws

2018-09-04 01:44:13 ERROR invalid om_file keyword: CacheSize at C:\Program Files (x86)\nxlog\conf\nxlog.conf:131 2018-09-04 01:44:13 ERROR module 'out' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:144 2018-09-04 01:44:13 ERROR route 1 is not functional without output modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:144

Any ideas? is this something only available in enterprise edition?


luke.taucher created
Replies: 1
View post »
last updated
Nxlog taking MC Afee event and send it to another monitoring server

Hi, i have an NXLOG server installed and i want it to take the antivirus event to another monitoring server. Do you know wich command i had to enter to take this log and push it to another server ?


Maxime.Tremblay created
Replies: 1
View post »
last updated
NXLOG im_ODBC module to collect data from table of MSSQL
Hello,

I have set up windows 2012R2+SQL2014 environment to verify the im_obdc function, now it is working to gather data from database, but there are two problems encountered.

1. How can I limit the rows to fetch from table every time ? if there is no limitation, it will affect the database performance at the first time .
2. I have a ID in SQL statement , and followed the instruction to have "where id > ? " in SQL statement. but nxlog always execute the same SQL statement and return same data to nxlog agent.

The following are configuration of input and SQL statement from SQL profiler

Configuration of input
==================
<Input in>
Module im_odbc
ConnectionString DSN=SEPM;uid=sem5;pwd=Admin123;database=sem5
#MaxIdSQL SELECT MAX(TIME_STAMP) from V_ALERTS
#ReadFromLast True
SQL SELECT a.USN, a.ALERT_IDX, a.SOURCE, a.VIRUSNAME_IDX, a.NOOFVIRUSES, a.FILEPATH, a.DESCRIPTION, a.ACTUALACTION_IDX, a.REQUESTEDACTION_IDX, a.ALERTDATETIME, a.USER_NAME, a.SOURCE_COMPUTER_NAME, a.SOURCE_COMPUTER_IP, a.TIME_STAMP, a.SOURCE_COMPUTER_IP_TEXT, v.VIRUSNAME, v.TYPE, PAT.VERSION, PAT.SEQUENCE, LOWER(S.NAME), LOWER(G.NAME), LOWER(P.NAME), Q.NAME, I.COMPUTER_DOMAIN_NAME, I.COMPUTER_NAME, I.CURRENT_LOGIN_USER, I.CURRENT_LOGIN_DOMAIN, I.IP_ADDR1_TEXT, I.MAC_ADDR1, I.OS_LANG, I.DISK_TOTAL, I.MEMORY, I.OPERATION_SYSTEM, I.SERVICE_PACK, I.BIOS_VERSION, SA.AGENT_VERSION, SA.AGENT_TYPE, SA.PROFILE_VERSION, SA.STATUS, SA.LAST_UPDATE_TIME, SA.INFECTED, SA.WORSTINFECTION_IDX, SA.LAST_VIRUS_TIME, SA.LAST_SCAN_TIME, SA.LAST_DOWNLOAD_TIME, SA.CONTENT_UPDATE, SA.PROFILE_SERIAL_NO, SA.MAJOR_VERSION, SA.LICENSE_STATUS, SA.LICENSE_EXPIRY FROM V_ALERTS a with (NOLOCK) LEFT JOIN VIRUS v ON a.VIRUSNAME_IDX = v.VIRUSNAME_IDX LEFT JOIN V_SEM_COMPUTER I ON I.COMPUTER_ID = a.COMPUTER_IDX LEFT JOIN SEM_AGENT SA ON I.COMPUTER_ID = SA.COMPUTER_ID LEFT JOIN IDENTITY_MAP S ON SA.DOMAIN_ID = S.ID LEFT JOIN IDENTITY_MAP G ON SA.GROUP_ID = G.ID LEFT JOIN IDENTITY_MAP P ON SA.LAST_SERVER_ID = P.ID LEFT JOIN IDENTITY_MAP Q ON SA.LAST_SITE_ID = Q.ID LEFT JOIN PATTERN PAT ON SA.PATTERN_IDX = PAT.PATTERN_IDX WHERE a.TIME_STAMP > ? ORDER BY a.TIME_STAMP
#SavePos True
PollInterval 30
</Input>

SQL Statement from Profiler
=======================
exec sp_executesql N'SELECT a.USN, a.ALERT_IDX, a.SOURCE, a.VIRUSNAME_IDX, a.NOOFVIRUSES, a.FILEPATH, a.DESCRIPTION, a.ACTUALACTION_IDX, a.REQUESTEDACTION_IDX, a.ALERTDATETIME, a.USER_NAME, a.SOURCE_COMPUTER_NAME, a.SOURCE_COMPUTER_IP, a.TIME_STAMP, a.SOURCE_COMPUTER_IP_TEXT, v.VIRUSNAME, v.TYPE, PAT.VERSION, PAT.SEQUENCE, LOWER(S.NAME), LOWER(G.NAME), LOWER(P.NAME), Q.NAME, I.COMPUTER_DOMAIN_NAME, I.COMPUTER_NAME, I.CURRENT_LOGIN_USER, I.CURRENT_LOGIN_DOMAIN, I.IP_ADDR1_TEXT, I.MAC_ADDR1, I.OS_LANG, I.DISK_TOTAL, I.MEMORY, I.OPERATION_SYSTEM, I.SERVICE_PACK, I.BIOS_VERSION, SA.AGENT_VERSION, SA.AGENT_TYPE, SA.PROFILE_VERSION, SA.STATUS, SA.LAST_UPDATE_TIME, SA.INFECTED, SA.WORSTINFECTION_IDX, SA.LAST_VIRUS_TIME, SA.LAST_SCAN_TIME, SA.LAST_DOWNLOAD_TIME, SA.CONTENT_UPDATE, SA.PROFILE_SERIAL_NO, SA.MAJOR_VERSION, SA.LICENSE_STATUS, SA.LICENSE_EXPIRY FROM V_ALERTS a with (NOLOCK) LEFT JOIN VIRUS v ON a.VIRUSNAME_IDX = v.VIRUSNAME_IDX LEFT JOIN V_SEM_COMPUTER I ON I.COMPUTER_ID = a.COMPUTER_IDX LEFT JOIN SEM_AGENT SA ON I.COMPUTER_ID = SA.COMPUTER_ID LEFT JOIN IDENTITY_MAP S ON SA.DOMAIN_ID = S.ID LEFT JOIN IDENTITY_MAP G ON SA.GROUP_ID = G.ID LEFT JOIN IDENTITY_MAP P ON SA.LAST_SERVER_ID = P.ID LEFT JOIN IDENTITY_MAP Q ON SA.LAST_SITE_ID = Q.ID LEFT JOIN PATTERN PAT ON SA.PATTERN_IDX = PAT.PATTERN_IDX WHERE a.TIME_STAMP > @P1 ORDER BY a.TIME_STAMP',N'@P1 bigint',2664642240



Thanks in advanced.

Sam_wang created
Replies: 1
View post »
last updated
Unable to send csv logs to graylog udp using nxlog

Hi Everyone, I have copied jira access logs, filter it using csv and put under a directory as csv file. I am using below nxconf which gets loaded and nxlog service gets started, however no logs are send to graylog server.

#define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension json> Module xm_json </Extension>

<Extension fileop> Module xm_fileop </Extension>

<Extension _syslog> Module xm_syslog </Extension>

<Extension gelf> Module xm_gelf </Extension>

<Extension jira> Module xm_csv Fields $IPAddress,$UserName,$DateTime,$HTTPAction,$ResponseCode,$Column10,$Column11 FieldTypes string,string,string,string,string,string,string Delimiter"," </Extension>

<Input in> Module im_file File "C:\Users\jira\Documents\TempOut\JiraAccessLogs\accessLog.csv" #ReadFromLast False #Recursive True #SavePos True

<Exec> if $raw_event =~ /^#/ drop(); else { jira->parse_csv();
to_json(); } </Exec>

</Input>

<Output out>

Module om_udp
Host 172.17.1.87
Port 5046
OutputType  GELF_UDP
Exec $short_message = $raw_event; # Avoids truncation of the short_message field.
Exec $collector_node_id = 'SINNB0094';
Exec $Hostname = hostname_fqdn();
#Use the following line for debugging (uncomment the fileop extension above as well)
Exec file_write(&quot;C:\Users\\jira\\Documents\\TempOut\\JiraAccessLogs\\nxlog-debug.log&quot;, $raw_event);

</Output>

<Route 1> Path in => out </Route>

if i replace, output section with below and send the data to txt/log file, i can see the logs converted to json and written.

Module om_file File "C:\nxlog-debug.txt"

Ii suspect there is some issue with out with om_udp but since i don’t see any error or warning, it’s difficult to troubleshoot. can you spot what is wrong with the out?


navdeepsingh83 created
Replies: 3
View post »
last updated
Output is literally in another language.

lol so yeah my output is in another language??

I am running an XML input of data and trying to get it into an easy format to use for Elasitc. I followed the manual the best I could here https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_multiline_example_5 but my output is crazy.

##NxLog conf file##

<Extension multiline> Module xm_multiline HeaderLine /^\s*<Obj RefId="[0-9][0-9]?[0-9]?[0-9]?">/ </Extension> <Extension _xml> Module xm_xml </Extension> <Extension _json> Module xm_json </Extension> <Input in3> Module im_file File "C:\Users\administrator\Desktop\newtest.xml" InputType multiline SavePos FALSE ReadFromLast FALSE Exec parse_xml(); Exec to_json(); </Input> <Output out3> Module om_file File "C:\Users\administrator\Desktop\testxml.txt" </Output> <Route> Path in3 => out3 </Route>

##End conf##

##Data sample##

<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"> <Obj RefId="12"> <TN RefId="4"> <T>System.Diagnostics.EventLogEntry#System/Microsoft-Windows-Kernel-General/16</T> <T>System.Diagnostics.EventLogEntry#System/Microsoft-Windows-Kernel-General</T> <T>System.Diagnostics.EventLogEntry</T> <T>System.ComponentModel.Component</T> <T>System.MarshalByRefObject</T> <T>System.Object</T> </TN> <ToString>System.Diagnostics.EventLogEntry</ToString> <Props> <S N="MachineName">testserver</S> <BA N="Data" /> <I32 N="Index">23749</I32> <S N="Category">(0)</S> <I16 N="CategoryNumber">0</I16> <I32 N="EventID">16</I32> <Obj N="EntryType" RefId="13"> <TNRef RefId="1" /> <ToString>Information</ToString> <I32>4</I32> </Obj> <S N="Message">The description for Event ID '16' in Source 'Microsoft-Windows-Kernel-General' cannot be found. The local computer may not have the necessary registry information or message DLL files to display the message, or you may not have permission to access them. The following information is part of the event:'109', '??\C:\Users\testaccount\AppData\Local\Packages\Microsoft.Windows.Cortana_cw5n1h2txyewy\Settings\settings.dat', '12', '4'</S> <S N="Source">Microsoft-Windows-Kernel-General</S> <Obj N="ReplacementStrings" RefId="14"> <TNRef RefId="2" /> <LST> <S>109</S> <S>??\C:\Users\testaccount\AppData\Local\Packages\Microsoft.Windows.Cortana_cw5n1h2txyewy\Settings\settings.dat</S> <S>12</S> <S>4</S> </LST> </Obj> <I64 N="InstanceId">16</I64> <DT N="TimeGenerated">2018-08-14T08:32:50-04:00</DT> <DT N="TimeWritten">2018-08-14T08:32:50-04:00</DT> <S N="UserName">testaccount</S> <Nil N="Site" /> <Nil N="Container" /> </Props> <MS> <I32 N="EventID">16</I32> </MS> </Obj> </Objs>

##End Sample##

##Output##

<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04">഍ ਍  㰀伀戀樀 刀攀昀䤀搀㴀∀㄀㈀∀㸀ഀ਍ഀ <TN RefId="4">഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍    㰀⼀吀一㸀ഀ਍ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢㄰ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ഀ <Props>഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍      㰀伀戀樀 一㴀∀䔀渀琀爀礀吀礀瀀攀∀ 刀攀昀䤀搀㴀∀㄀㌀∀㸀ഀ਍ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢㄰ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢㄰ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢㄰ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ഀ </Obj>഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍      㰀伀戀樀 一㴀∀刀攀瀀氀愀挀攀洀攀渀琀匀琀爀椀渀最猀∀ 刀攀昀䤀搀㴀∀㄀㐀∀㸀ഀ਍ഀ笊䔢敶瑮敒散癩摥楔敭㨢㈢㄰ⴸ㠰㌭‰ㄱ〺㨱㐴Ⱒ匢畯捲䵥摯汵乥浡≥∺湩∳∬潓牵散潍畤敬祔数㨢椢彭楦敬索਍ഀ <LST>഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍        㰀⼀䰀匀吀㸀ഀ਍ഀ </Obj>഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍    㰀⼀倀爀漀瀀猀㸀ഀ਍ഀ <MS>഍ ਍≻癅湥剴捥楥敶呤浩≥∺〲㠱〭ⴸ〳ㄠ㨱㄰㐺∴∬潓牵散潍畤敬慎敭㨢椢㍮Ⱒ匢畯捲䵥摯汵呥灹≥∺浩晟汩≥ൽ ਍    㰀⼀䴀匀㸀ഀ਍ഀ </Obj>഍ ਍ 㰀⼀伀戀樀猀㸀ഀ

##End Output##


Deleted user created
Replies: 1
View post »
last updated
Can Nxlog Enterprise Edition 3.1.1930 work with MSSQL 2014 ?

Hello,

We have Nxlog Enterprise Edition 3.1.1930 to collect log from MSSQL 2014 via im_odbc module, but it return the following error. "ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)"

The related configuration as below:

<Input in> Module im_odbc ConnectionString DSN=SymantecEndpointSecurityDSN;database=sem5; SQL SELECT IDX as ID,ALERT_IDX as AlertID, COMPUTER_IDX as ComputerID,SOURCE as SRC,VIRUSNAME_IDX as virusname FROM V_ALERTS WHERE IDX > ? </Input>

Thanks for any help~


Sam_wang created
Replies: 2
View post »
last updated