Hi! We have nxlog ce running in a Windows machine. It works ok. If time is changed to the future, it continues forwarding logs. However, if time is changed to the past, logs are not forwarded anymore. This affects to logs from windows events, from a text file, etc. It seems that nxlog is filtering the logs and that logs with a previous time than others received are discarded. Logs are forwarded again if nxlog service is restarted (this seems to be doing a 'reset' on expected time) Do you know how could we avoid this?

I am trying to us pm_pattern to filter messages that are approved for a 'limited' feed, and still have the full feed go to the admin feed. I have set up the config to send the input to two feeds, that works fine. I've then tried to use pm_pattern to match certain strings and DROP them from the restricted feed. So far no luck. I'm sure I'm missing something really simple here, and would really appreciate if anyone had the time to check the configs for me... in nxlog.conf (edited for brevity)... Module pm_pattern PatternFile "/data/conf/nxlog-patternmatch.yaml" Module om_http URL https:// destination string #Batchmode none HTTPSAllowUntrusted TRUE HTTPSCADir /etc/ssl/certs/ HTTPSCertFile /etc/ssl/certs/ca-certificates.crt Exec if defined $PatternID drop(); Path client2001 => client_filter => out2001,client-test and this is the patternmatch yaml (some matchfields removed)... 2021-06-03 01:02:03 1 <group> <name>Client</name> <pattern> <id>1</id> <name>client input</name> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>rsyslogd</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>NetworkManager</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>Systemd</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>dnf</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>dbus</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>chrony</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>CRON</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>motd</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>snapd</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>promtail</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>nxlog</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>kernel</value> </matchfield> <matchfield> <Name>raw_event</Name> <type>regexp</type> <value>loki</value> </matchfield> </pattern> </group>

Hi support, We are trying to send logs from a server with NXLog agent installed to SIEM, with SSL/TLS (encrypted traffic) Would it be possible to only use ONE certificate for each agent? For NXLog manager and the agents is it possible to use only ONE CA for One agent?

Hi, I have this type of input in nxlog.conf: <Input udp> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); to_json(); perl_call("process"); </Input> My question is, how should I include that JSON output that I get from to_json() to my perl code? Should I write like this?: my ( $event ) = @_; Or it's only the output of parse_syslog_bsd() (as in example for xm_perl https://nxlog.co/documentation/nxlog-user-guide/xm_perl.html)? More generally, my question is how to include JSON output that i get from to_json() to perl code of xm_perl module?

Hello NXlog world! I'm having some challenges to pull data from a SQL 2012 (running on W2K16) database using the im-odbc connector. This is my input: > <Input MSSQL_IN> > Module im_odbc > ConnectionString Driver={ODBC Driver 17 for SQL Server}; Server=XXXXXXXX; > Trusted_Connection=yes; DATABASE=XXXXX; > PollInterval 5 > IdType timestamp > SQL SELECT event_time AS 'id', f., a.name AS action_name > FROM fn_get_audit_file('C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\XXXXX\Log\Audit-File.sqlaudit', default, > default) AS f > INNER JOIN sys.dm_audit_actions AS a > ON f.action_id = a.action_id > WHERE event_time > ? > <Exec> > delete($id); > rename_field($event_time, $EventTime); > </Exec> > > </Input> The error I'm getting when starting NXlog is as follows: 2019-10-16 13:51:03 INFO nxlog-4.4.4431 started 2019-10-16 13:51:03 INFO im_odbc successfully connected to the database 2019-10-16 13:51:03 ERROR SQLExecDirect failed, 42000:2:300:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]CONTROL SERVER permission was denied on object 'server', database 'master'.; 42000:3:297:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The user does not have permission to perform this action. (odbc error code: -1)` I have tried to enable of the permissions on the database (GRANT) but the CONTROL SERVER is throwing me off. I'm pretty sure is a stupid thing I haven't thought about, but since I am very new to NXlog, and obviously not a SQL admin, I am defeated! for now, so I'm hopeful someone, can help me with some guidance so I can troubleshoot further. TIA and happy hump day! JC

Hi, I transmitted IIS logs and message tracking logs from one server to the other through nxlog and saved it in a location. After complete transmission I stopped the nxlog service. I was able to understand that the transmission is complete and I downloaded the data and was working on it. suddenly within 10 minutes few files are lost under IIS and message tracking. I do not know the reason behind this. Is it really possible? Can the data get lost after stopping the service? or something could have caused it? I'm trying to find out the root cause. Pls help

Hi, I'm trying to parse a csv log file from my local and store it again in another location in my local. But I see it is not happening. The config has no errors. the destination file is same as source file with respect to file properties. Can you tell whether any specific check should be done? <Extension csv_parser1> Module xm_csv Fields date-time,client-ip,client-hostname,server-ip,server-hostname, source-context,connector-id,source,event-id,internal-message-id,message-id,recipient-address, recipient-status,total-bytes,recipient-count,related-recipient-address,reference,message-subject, sender-address,return-path,message-info,directionality,tenant-id, original-client-ip,original-server-ip,custom-data Delimiter , </Extension> Message Tracking log as input <Input messagetracking> Module im_file File '%BASEDIR%file.log' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser1->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec> </Input> <Output msg> Module om_file File 'location\msg.log' </Output> <Route 1> Path messagetracking => msg </Route>

Can you explain what exactly happens in this block? I don't get a clear explanation of what "xEF\xBB\xBF" means in the below code and why it is having drop ()? . <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate(${date-time}); } </Exec>

Hi, I would like to know the cost of enterprise edition. Also, I would like to know whether we could use the purchased nxlog EE package in more than one server to collect logs?

i want to send my atp log to syslog server with the help of nxlog . i am storing atplog to atp.log file and the data is in json format. I am using below configuration but i am not getting anything on my syslog server which is AV. configuration: # Configuration for converting and sending Windows logs # to AlienVault USM Anywhere. # define ROOT C:\Program Files (x86)\nxlog define OUTPUT_DESTINATION_ADDRESS 10.0.2.4 define OUTPUT_DESTINATION_PORT 514 Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> <input atplogfile> Module im_atplogfile </input> <Input ATPLogs_IN> Module im_atplogfile File "C:\temp\log\atplogs-.log" Exec $Message = to_json(); to_syslog_bsd(); </Input> <Output ATPLogs_OUT> Module om_atplogfile File "/var/log/nxlog/syslog.log" Exec to_syslog_bsd(); </Output> ######################################## # Routes # ######################################## <Route 1> Path ATPLogs_IN => ATPLogs_OUT </Route> <Input eventlog> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security"></Select> </Query> </QueryList> Exec if ($EventID == 5156) OR ($EventID == 5158) drop(); </Input> <Output out> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route 1> Path eventlog, internal => out </Route>

Hi, Would like to know whether there is an option to input the files generated by log4net and push it to loggly. The file format from log4net could be with same extension or on rolling numbers. For example, the file name could be like samplelog-10102018.txt, samplelog-10102018.txt.1, samplelog-10102018.txt.2 or samplelog-10102018.1.txt, samplelog-10102018.2.txt, samplelog-10102018.3.txt. Tried with filename in input block (with in nxlog.conf file) as samplelog*.txt and samplelog*.txt.*, but could not get the details in loggly. How to read multiple files in a location with filenames in rotation based on dateformat. Thanks in advance!

Im not sure why im get this access denied in the nxlog logs 2018-11-01 10:17:45 ERROR failed to subscribe to msvistalog events,access denied [error code: 5]; Access is denied. Im using nxlog EE v4 on windows 2012 server and it is run as default system user Any suggestions where to look in the windows configuration or nxlog configuration itself ?

hi, I'm working on monitoring a log file using nxlog. I have the File set to "C:\Program Files\test1.log" but it's saying that the "input file does not exist". I tried running a python script to check the file using the os module import os test = os.listdir('C:\Program Files\test1.log') print(test) This will return an error "FileNotFoundError: The system cannot find the path specified" I noticed that this error has been encountered before but none of the solutions I tried work. any help is much appreciated. Thanks, skawt

Hi, I'm trying to send messages from NXLog into Logstash with a custom TAG. Logstash would filter those messages and then send them into specific topics in Kafka. For example my current Logstash + Filebeats works like that: filebeat.yml has: paths: - /var/log/*.log tags: ["EXAMPLE_1"] Logstash.yml has : output { if "EXAMPLE_1" in [tags]{ kafka { bootstrap_servers => "example_dns:9092" topic_id => "example_1_topic_kafka" } } } Is it possible to recreate such simple config with NXLog?

We are currently using Nxlog enterprise 4.0.3735 to send events logs(windows events - > json out) from windows server to central log server. I want to estimate system requirements(cpu,memory,disk) needed by nxlog agent to handle 4000 to 5000 events per seconds. I m also concerned about what happens when the agent gets high (unexpected) number of events more than expected during peak load times Are logs buffered some where on disk and sent later or something similar when link between the nxlog agent and central logserver is broken for some reason ?

Hi, I have a problem with nxlog. Try to start service nxlog with kafka configuration (including installation librdkafka) and unfortunately i have an error with starting nxlog: error: "Unit nxlog.service has begun starting up. Jan 03 17:24:12 Kafka4 nxlog[19220]: 2018-01-03 17:24:12 ERROR Failed to load module from /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so, /opt/nxsec/libexec/nxlog/modules/output/om_kafka.so: undefined symbol: rd_kafka_last_error;DSO load failed Jan 03 17:24:12 Kafka4 systemd[1]: nxlog.service: control process exited, code=exited status=1 Jan 03 17:24:12 Kafka4 systemd[1]: Failed to start NXLog daemon. -- Subject: Unit nxlog.service has failed -- Defined-By: systemd"

Hi , Trying to create an ODBC connect for NXLog to connect to.   NXLog is installed on the same Windows 2012 server as the SQL Server 2008R2 instance.   Scenario 1: 32-bit ODBC is setup as a System DSN with a SQL Server account that has DBO access to the desired database NXLog service is setup to run under the System account.      - I've tried both drivers available on the system ("SQL Server Native Client 10.0"  and "SQL Server") - get the same result in the error log for each: ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][SQL Server Native Client 10.0][SQL Server]Login failed for user ''. (odbc error code: -1)and ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)   Scenario 2: Same ODBC, but with a Windows account that has full Admin access to the desired databases, and is the same account logged into Windows NXLog service is setup to run under this same account. Goal is to have the same user account accessing everything, in the hope of getting it to connect.   Same error messages as above.   Login failed for user ' '.     Since the error messages don't show the user that is failing to login, I'm having trouble narrowing down where the failure is at.   NXLOG.conf file: <Input call_logs> Module im_odbc ConnectionString DSN=SIEM_NXLog;database=recorder; SQL SELECT ident as id ,at.audit_time as EventTime ,am.audit_module_name as Message FROM mytables...  WHERE at.ident>? SavePos TRUE </Input>   There's one line in the documentation that has me scratching my head: SECTION 6.2.18 (ODBC) The data source must be accessible by the user which nxlog is running under.   I'm not sure if this means that the NTService account needs database access? Or, if the service must be under a Windows account user that has database access? Or, by using a ODBC->System DSN , shouldn't the ODBC already be accessible to all users on the system?   Any thoughts or insight would be helpful. Thanks in advance.   Cheers,  Peter          

Hello, we’re looking at using multiple instances of nxlog.exe on the same Windows machine. What is the supported method for this please? If I create a new process we receive the ERROR Service is already running log entry. Thanks

Hi! I just want to ask if when I update my nxlog to the current version, do I need to revise my code? Is there a list of versions with release notes available? Upon saerching in the website, I couldn't find the list of versions. Only the latesd version is available. Thank you in advance.   

  hello I had an error  while execting nxlog on windows server  2017-05-11 15:34:51 ERROR invalid keyword: { at C:\Program Files (x86)\nxlog\conf\nxlog.conf:38   NXLOG.CONF define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>     Module      xm_syslog </Extension> <Input in> Module  im_file File 'C:\DNSlogs\DNSlogs.log' SavePos TRUE ReadFromLast TRUE PollInterval 5 CloseWhenIdle TRUE # The following line is the pre-parsing of DNS log. It turns the normal DNS log line into a more readable and easy for parsing with OSSIM format  Exec if $raw_event =~/(\d+\/\d+\/\d+\s+\d+\:\d+\:\d+)\s+.*(Snd|Rcv)\s+(\d+\.\d+\.\d+\.\d+)\s+\S+\s+\w+\s+\w+\s+\S+\s+\w+\s+\S+\s+(\w+)\s+\(\d+\)(\S+)\(\d+\)/\  { \                   $EventTime = parsedate($1); \                   $SourceName = $3; \                   $ProcessID = $2; \                   $Message = $5; \                   if $Message =~ s/\(\d+\)/./g log_info("removed all whitespace in SourceName"); \                 } </Input> <Output out>     Module      om_udp     Host        80.176.253.71     Port        514     Exec        to_syslog_bsd(); </Output> <Route 1>     Path        in => out </Route> thx for help