Hello NXlog world! I'm having some challenges to pull data from a SQL 2012 (running on W2K16) database using the im-odbc connector. This is my input:

> <Input MSSQL_IN> > Module im_odbc > ConnectionString Driver={ODBC Driver 17 for SQL Server}; Server=XXXXXXXX;
> Trusted_Connection=yes; DATABASE=XXXXX; > PollInterval 5 > IdType timestamp > SQL SELECT event_time AS 'id', f., a.name AS action_name
> FROM fn_get_audit_file('C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\XXXXX\Log\Audit-File.sqlaudit', default,
> default) AS f
> INNER JOIN sys.dm_audit_actions AS a
> ON f.action_id = a.action_id
> WHERE event_time > ? > <Exec> > delete($id); > rename_field($event_time, $EventTime); > </Exec> > > </Input>

The error I'm getting when starting NXlog is as follows:

2019-10-16 13:51:03 INFO nxlog-4.4.4431 started 2019-10-16 13:51:03 INFO im_odbc successfully connected to the database 2019-10-16 13:51:03 ERROR SQLExecDirect failed, 42000:2:300:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]CONTROL SERVER permission was denied on object 'server', database 'master'.; 42000:3:297:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The user does not have permission to perform this action. (odbc error code: -1)`

I have tried to enable of the permissions on the database (GRANT) but the CONTROL SERVER is throwing me off. I'm pretty sure is a stupid thing I haven't thought about, but since I am very new to NXlog, and obviously not a SQL admin, I am defeated! for now, so I'm hopeful someone, can help me with some guidance so I can troubleshoot further. TIA and happy hump day!

JC

Hi!

We have nxlog ce running in a Windows machine. It works ok.

• If time is changed to the future, it continues forwarding logs.
• However, if time is changed to the past, logs are not forwarded anymore. This affects to logs from windows events, from a text file, etc.

It seems that nxlog is filtering the logs and that logs with a previous time than others received are discarded. Logs are forwarded again if nxlog service is restarted (this seems to be doing a 'reset' on expected time) Do you know how could we avoid this?

I am trying to us pm_pattern to filter messages that are approved for a 'limited' feed, and still have the full feed go to the admin feed. I have set up the config to send the input to two feeds, that works fine. I've then tried to use pm_pattern to match certain strings and DROP them from the restricted feed. So far no luck. I'm sure I'm missing something really simple here, and would really appreciate if anyone had the time to check the configs for me... in nxlog.conf (edited for brevity)... Module pm_pattern PatternFile "/data/conf/nxlog-patternmatch.yaml" Module om_http URL https:// destination string #Batchmode none HTTPSAllowUntrusted TRUE HTTPSCADir /etc/ssl/certs/ HTTPSCertFile /etc/ssl/certs/ca-certificates.crt Exec if defined $PatternID drop(); Path client2001 => client_filter => out2001,client-test and this is the patternmatch yaml (some matchfields removed)... 2021-06-03 01:02:03 1 Client 1 client input raw_event regexp rsyslogd raw_event regexp NetworkManager raw_event regexp Systemd raw_event regexp dnf raw_event regexp dbus raw_event regexp chrony raw_event regexp CRON raw_event regexp motd raw_event regexp snapd raw_event regexp promtail raw_event regexp nxlog raw_event regexp kernel raw_event regexp loki

Hi support,

1. We are trying to send logs from a server with NXLog agent installed to SIEM, with SSL/TLS (encrypted traffic) Would it be possible to only use ONE certificate for each agent?

2. For NXLog manager and the agents is it possible to use only ONE CA for One agent?

Hi, I have this type of input in nxlog.conf: ``` Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); to_json(); perl_call("process"); ``` My question is, how should I include that JSON output that I get from to_json() to my perl code? Should I write like this?: ``` my ( $event ) = @_; ``` Or it's only the output of parse_syslog_bsd() (as in example for xm_perl https://nxlog.co/documentation/nxlog-user-guide/xm_perl.html)? **More generally, my question is how to include JSON output that i get from to_json() to perl code of xm_perl module?**

Hi,

I transmitted IIS logs and message tracking logs from one server to the other through nxlog and saved it in a location. After complete transmission I stopped the nxlog service. I was able to understand that the transmission is complete and I downloaded the data and was working on it. suddenly within 10 minutes few files are lost under IIS and message tracking.

I do not know the reason behind this. Is it really possible? Can the data get lost after stopping the service? or something could have caused it? I'm trying to find out the root cause. Pls help

Hi, I'm trying to parse a csv log file from my local and store it again in another location in my local. But I see it is not happening. The config has no errors. the destination file is same as source file with respect to file properties. Can you tell whether any specific check should be done?

<Extension csv_parser1> Module xm_csv Fields date-time,client-ip,client-hostname,server-ip,server-hostname,
source-context,connector-id,source,event-id,internal-message-id,message-id,recipient-address,
recipient-status,total-bytes,recipient-count,related-recipient-address,reference,message-subject,
sender-address,return-path,message-info,directionality,tenant-id,
original-client-ip,original-server-ip,custom-data Delimiter , </Extension>

Message Tracking log as input

<Input messagetracking> Module im_file File '%BASEDIR%file.log' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser1->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec> </Input>

<Output msg> Module om_file File 'location\msg.log' </Output>

<Route 1> Path messagetracking => msg </Route>

Can you explain what exactly happens in this block?
I don't get a clear explanation of what "xEF\xBB\xBF" means in the below code and why it is having drop ()? .

<Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec>

Hi,

I would like to know the cost of enterprise edition. Also, I would like to know whether we could use the purchased nxlog EE package in more than one server to collect logs?

i want to send my atp log to syslog server with the help of nxlog . i am storing atplog to atp.log file and the data is in json format.

I am using below configuration but i am not getting anything on my syslog server which is AV.

configuration:
# Configuration for converting and sending Windows logs
# to AlienVault USM Anywhere.
#

define ROOT C:\Program Files (x86)\nxlog
define OUTPUT_DESTINATION_ADDRESS 10.0.2.4
define OUTPUT_DESTINATION_PORT 514

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log


<Extension json>
Module xm_json
</Extension>

<Extension syslog>
Module xm_syslog
</Extension>

<Input internal>
Module im_internal
</Input>
<input atplogfile>
Module im_atplogfile
</input>

<Input ATPLogs_IN>
Module im_atplogfile
File "C:\\temp\\log\\atplogs-*.log"
Exec $Message = to_json(); to_syslog_bsd();
</Input>

<Output ATPLogs_OUT>
Module om_atplogfile
File "/var/log/nxlog/syslog.log"
Exec to_syslog_bsd();
</Output>


########################################
# Routes #
########################################
<Route 1>
Path ATPLogs_IN => ATPLogs_OUT
</Route>

<Input eventlog>
Module im_msvistalog
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
Exec if ($EventID == 5156) OR ($EventID == 5158) drop();
</Input>


<Output out>

Module om_udp
Host %OUTPUT_DESTINATION_ADDRESS%
Port %OUTPUT_DESTINATION_PORT%
Exec $EventTime = integer($EventTime) / 1000000;
Exec $Message = to_json(); to_syslog_bsd();

</Output>

<Route 1>
Path eventlog, internal => out
</Route>

Hi, Would like to know whether there is an option to input the files generated by log4net and push it to loggly. The file format from log4net could be with same extension or on rolling numbers. For example, the file name could be like samplelog-10102018.txt, samplelog-10102018.txt.1, samplelog-10102018.txt.2 or samplelog-10102018.1.txt, samplelog-10102018.2.txt, samplelog-10102018.3.txt. Tried with filename in input block (with in nxlog.conf file) as samplelog*.txt and samplelog*.txt.*, but could not get the details in loggly. How to read multiple files in a location with filenames in rotation based on dateformat.

Thanks in advance!

Im not sure why im get this access denied in the nxlog logs 2018-11-01 10:17:45 ERROR failed to subscribe to msvistalog events,access denied [error code: 5]; Access is denied. Im using nxlog EE v4 on windows 2012 server and it is run as default system user Any suggestions where to look in the windows configuration or nxlog configuration itself ?

hi,

I'm working on monitoring a log file using nxlog. I have the File set to "C:\Program Files\test1.log" but it's saying that the "input file does not exist". I tried running a python script to check the file using the os module

import os

test = os.listdir('C:\Program Files\test1.log') print(test)

This will return an error "FileNotFoundError: The system cannot find the path specified"

I noticed that this error has been encountered before but none of the solutions I tried work.

any help is much appreciated.

Thanks, skawt

Hi, I'm trying to send messages from NXLog into Logstash with a custom TAG. Logstash would filter those messages and then send them into specific topics in Kafka. For example my current Logstash + Filebeats works like that:

filebeat.yml has:

paths: - /var/log/*.log

tags: ["EXAMPLE_1"]

Logstash.yml has :

output { if "EXAMPLE_1" in [tags]{ kafka { bootstrap_servers => "example_dns:9092" topic_id => "example_1_topic_kafka" } } }

Is it possible to recreate such simple config with NXLog?

We are currently using Nxlog enterprise 4.0.3735 to send events logs(windows events - > json out) from windows server to central log server.

1. I want to estimate system requirements(cpu,memory,disk) needed by nxlog agent to handle 4000 to 5000 events per seconds. I m also concerned about what happens when the agent gets high (unexpected) number of events more than expected during peak load times
2. Are logs buffered some where on disk and sent later or something similar when link between the nxlog agent and central logserver is broken for some reason ?

Hi, I have a problem with nxlog. Try to start service nxlog with kafka configuration (including installation librdkafka) and unfortunately i have an error with starting nxlog:

error: "Unit nxlog.service has begun starting up.

Jan 03 17:24:12 Kafka4 nxlog[19220]: 2018-01-03 17:24:12 ERROR Failed to load module from /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so, /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so: undefined symbol: rd_kafka_last_error;DSO load failed Jan 03 17:24:12 Kafka4 systemd[1]: nxlog.service: control process exited, code=exited status=1 Jan 03 17:24:12 Kafka4 systemd[1]: Failed to start NXLog daemon. -- Subject: Unit nxlog.service has failed -- Defined-By: systemd"

Hello, we’re looking at using multiple instances of nxlog.exe on the same Windows machine. What is the supported method for this please?

If I create a new process we receive the ERROR Service is already running log entry.

Thanks

Hi!

I just want to ask if when I update my nxlog to the current version, do I need to revise my code?

Is there a list of versions with release notes available? Upon saerching in the website, I couldn't find
the list of versions. Only the latesd version is available.

Thank you in advance. 

 

 

hello I had an error  while execting nxlog on windows server 

2017-05-11 15:34:51 ERROR invalid keyword: { at C:\Program Files (x86)\nxlog\conf\nxlog.conf:38

 

NXLOG.CONF

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Input in>

Module  im_file

File 'C:\DNSlogs\DNSlogs.log'
SavePos TRUE
ReadFromLast TRUE
PollInterval 5
CloseWhenIdle TRUE
# The following line is the pre-parsing of DNS log. It turns the normal DNS log line into a more readable and easy for parsing with OSSIM format
 Exec if $raw_event =~/(\d+\/\d+\/\d+\s+\d+\:\d+\:\d+)\s+.*(Snd|Rcv)\s+(\d+\.\d+\.\d+\.\d+)\s+\S+\s+\w+\s+\w+\s+\S+\s+\w+\s+\S+\s+(\w+)\s+\(\d+\)(\S+)\(\d+\)/\

 { \
                  $EventTime = parsedate($1); \
                  $SourceName = $3; \
                  $ProcessID = $2; \
                  $Message = $5; \
                  if $Message =~ s/\(\d+\)/./g log_info("removed all whitespace in SourceName"); \
                }
</Input>

<Output out>
    Module      om_udp
    Host        80.176.253.71
    Port        514
    Exec        to_syslog_bsd();
</Output>

<Route 1>
    Path        in => out
</Route>

thx for help