if I start nxlog manul from the command line like

/opt/nxlog/bin/nxlog -c      # RedHat7

everything works fine ; it reads the input logs from the positions when stopped. if I start it as service

systemctl start nxlog

then it ignores all the records of the input logs written while it was stopped. at my input definitions I have

SavePos          TRUE
ReadFromLast     TRUE

any help is welcomed, thanks, G. Bouras

Is there a way that I could write a python script and make it execute through the im_exec module

Hi there, I've attempted to get NXlog CE installed onto a clean CentOS 7 host, following the documentation, ran 'rpm -ivh nxlog-ce-2.10.2150-1_rhel7.x86_64.rpm' ... whilst the RPM command completes, it appears to do absolutely nothing .. I search the host for any files with 'nxlog' in the filename.. nothing!

Have I got a dud RPM file ? (Downloaded direct via the website)

Any suggestions, or known other versions of NXlog CE that I can use?

Hello, I'm both new here and new at nxlog so excuse my question if it sounds awkward. I'm trying to configure nxlog for an environment with multiple intermediary loghosts which have different IP addresses. The only pattern is that the machine that is sending the log and the loghost always have a similar first three octets (same subnet). So the computer 192.168.0.10 will send logs to 192.168.0.100 and the computer 10.10.10.30 will send its logs to 10.10.10.100. The last octet of all loghosts are similar as well.

My goal is to be able to call the computer IP with HostIP, match it with a regex [0-9]{1,3}[.][0-9]{1,3}[.][0-9]{1,3}[.][0-9]{1,3} and transform it to $1.$2.$3.100 which will be the loghost IP. My output module may look like this:

<Output loghost> Module om_udp Host $loghost Port 514 </output>

Why am I doing this? I'm deploying nxlog via GPO and wanted to send a single nxlog.conf to all the domain computers which will find the corresponding loghost based on their own IP.

At this time, none of my attempts to add a regex to an Exec directive in the output module were successful. If any one had come across the need for adding a variable as Host or similar issue, I will appreciate your help. Any other directions are much appreciated.

Thank you, Mikal

Hi, I'm looking for a mean to move logs files from one folder to another folder after processed them. I want to know if nxlog has a Procedure like file_copy to do that. Thank you

Hi,

There was an issue in nxlog CE 2.5 edition, when remote and local logging both are enabled and for some reason if remote logging is stopped, both loggings are stopped, my understanding is that it has been fixed in latest edition, I would like to know exact versions in which it got fixed and does it fixed in CE or EE?

Error I am referring to in nxlog CE 2.5.1089 is as following: ERROR om_udp apr_socket_send failed Connection refused

Please let me know if it is fixed in CE subsequent versions and if yes, can you please provide exact version in which it got fixed, that would help.

In case of query, please do let me know.

Thanks, Chandrashekhar

Hi, I'm testing Nxlog EE trial. nxlog-trial-4.4.4347_windows_x64.msi

And configured nxlog on windows host, but om_kafka output module won't work. with error

2019-06-14 18:28:47 ERROR Failed to load module from C:\Program Files\nxlog\modules\output\om_kafka.dll, The specified module could not be found.  ; The specified module could not be found.  
2019-06-14 18:28:47 ERROR module 'kafka' is not declared at C:\Program Files\nxlog\conf\nxlog.conf:84

However om_kafka.dll100% persist in folder C:\Program Files\nxlog\modules\output\ I'm trying to reinstall, repair, install on x32 and x64 same as Windows Server 2012R2 and Windows Server 2016

Similar module om_kafka works fine on Centos7

Seems that “out of the box” this module is not working on OS windows , I mean, what am I missing? maybe any additional librdkafka installation required?

Hi, I'm testing Nxlog EE trial And configured nxlog as WEC with im_wseventing module, but for some reason `$EventType` field Parsed to simple "`AUDIT`" not `AUDIT_SUCCESS` or `AUDIT_FAILURE` In doc Possible values are: `CRITICAL, ERROR, AUDIT_FAILURE, AUDIT_SUCCESS, INFO, WARNING, and VERBOSE`. Example of Event: ``` json Jun 14 15:13:33 SRVTEST-00.test Microsoft-Windows-Security-Auditing[648]: { "MessageID": "uuid:1DB8B636-E34C-4DB5-951D-EEE30FD8F837", "SourceName": "Microsoft-Windows-Security-Auditing", "ProviderGuid": "{54849625-5478-4994-A5BA-3E3B0328C30D}", "EventID": 4634, "Version": 0, "LevelValue": 0, "EventType": "AUDIT", "SeverityValue": 2, "Severity": "INFO", "OpcodeValue": 0, "Keywords": "0x8020000000000000", "EventTime": "2019-06-14 15:13:33", "RecordNumber": 3437460, "ExecutionProcessID": 648, "ExecutionThreadID": 4980, "Channel": "Security", "Hostname": "SRVTEST-00.test", "TargetUserSid": "S-1-5-18", "TargetUserName": "SRVTEST-00$", "TargetDomainName": "TEST", "TargetLogonId": "0x2b06461", "LogonType": "3", "Message": "An account was logged off.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tSRVTEST-00$\n\tAccount Domain:\t\tTEST\n\tLogon ID:\t\t0x2B06461\n\nLogon Type:\t\t\t3\n\nThis event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.", "Level": "Information", "Task": "Logoff", "Opcode": "Info", "EventReceivedTime": "2019-06-14 15:13:35", "SourceModuleName": "wseventin", "SourceModuleType": "im_wseventing", "HostIP": "192.168.5.5" } ``` My nxlog config: ``` User nxlog Group nxlog Panic Soft # default values: PidFile /opt/nxlog/var/run/nxlog/nxlog.pid CacheDir /opt/nxlog/var/spool/nxlog ModuleDir /opt/nxlog/libexec/nxlog/modules SpoolDir /opt/nxlog/var/spool/nxlog define LOGDIR /opt/nxlog/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log LogFile %MYLOGFILE% Module xm_syslog Module xm_json Module xm_resolver Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') >= 5M) ) { file_cycle('%MYLOGFILE%', 8); } # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); Module im_wseventing Address http://srvtest-12.test:80/wsman ListenAddr 0.0.0.0 Port 80 SubscriptionName testing Exec $HostIP = name_to_ipaddr($Hostname); Exec log_info(to_json()); * * * Module om_file File '/opt/nxlog/var/log/nxlog/winevent.log' CreateDir TRUE Exec $Message = to_json(); to_syslog_bsd(); Path wseventin => tofile ``` Is it bug or trial restrictions?

Hello I've been trying to the use linuxaudit system to work but I'm stuck.

--- Nxlog-agent setup --- OS: SUSE Tumbleweed 20190512 Agent-Version: 4.4.4347 Module: im_linuxaudit

--- Configuration --- <Extension _json> Module xm_json </Extension>

<Extension audit_parser> Module xm_kvp KVPDelimiter ' ' KVDelimiter = EscapeChar '' </Extension>

<Input audit> Module im_linuxaudit FlowControl FALSE <Rules> -D -b 320 -w /etc/passwd -p wa -k etcpasswd -w /bin/cat -p wxa -k cat_exection -e 1 </Rules> <Exec> audit_parser->parse_kvp(); $Hostname = hostname(); $FQDN = hostname_fqdn(); $Tag = "audit"; $SourceName = "auditd_nxlog"; </Exec> </Input>

<Output tcp> Module om_tcp Host 192.168.4.58 Port 1337 Exec to_json(); to_syslog_bsd(); </Output>

<Route audit_to_tcp> Path audit => tcp </Route>

I've been trying to run the above mentioned audit config however whenever I'm doing /bin/cat/ /etc/passwd I don't get any kind of alerts on my receiver box. HOWEVER I will get logs when I change user (e.g. su otheruser). Additionally I was wondering if the log format 'ENRICHED' from the normal audit daemon is supported. https://github.com/linux-audit/audit-documentation/wiki/SPEC-Audit-Event-Enrichment

Best regards Florian Reiter

Hi everybody! Today I found in the logs of NXLog 4.3.4308 Server very odd error: ``` 2019-06-12 11:22:04 ERROR failed to open file when trying to truncate: Too many open files ``` The service was not working at this time until I has restarted it. Could you be please so kind tell me what is the limit of opened files? What's the number simultaneous connections can the service hold? Thanks!

I see in the documentation that Nano is supported but I don't see Server Core mentioned explicitly.

Thanks,

Hi all, I have this config ``` Panic Soft define ROOT C:\Program Files (x86)\nxlog define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_multiline FixedLineCount 2 Module im_file File "C:\\txt\\event.txt" InputType multilines Module om_file File "C:\\txt\\txt1.log" Path InputData => OutputData ``` And this input log file ``` event1 Data1 event2 Data2 event3 Data3 event4 Data4 event5 Data5 event6 Data6 ``` But output file is always empty and nxlog.log is without errors or warnings. I want to merge two lines into single line.

When I use im_file and om_odbc,from the log,I get this message:WARNING SQLExecDirect failed; HY000:1:0:[Devart][ODBC][PostgreSQL]在字段 "timestamp" 中空值违反了非空约束 <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log1.txt" ReadFromLast False SavePos False <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec>

Exec parse_syslog();

</Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1;
UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> #<Output out>

Module om_file

File "C:\Users\jiang.dengjie\Desktop\logtest.txt"

Exec to_json();

#</Output> <Route r> Path in => out </Route> In my file: %Feb 5 15:47:32:118 2015 trust-access IFNET/5/LINK_UPDOWN: Line protocol on the interface GigabitEthernet1/0/41 is down. %Feb 5 15:47:35:367 2015 trust-access IFNET/3/PHY_UPDOWN: GigabitEthernet1/0/40 link status is up. And I want to use nxlog to save this file to my pgsql which has the table that has five colum:id,timestamp,hostname,eventname,event. Also if any viedo about how to use nxlog? Thank you very much.

I use im_file and om_file on windows,But throgh om_file,I get a file that is empty <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log.txt" Exec parse_syslog(); </Input> <Output out> Module om_file File "C:\Users\jiang.dengjie\Desktop\logtest.txt" Exec to_json(); </Output> <Route r> Path in => out </Route>

I use nxlog in windows,from the log,I get this question:ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib Then I find I do not have the agent-ca.pem. And in my environment,my pgsql do not get any data. below is my config. <Input in> Module im_file File "C:\Users\xxx\Desktop\log.txt" <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec> </Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1;
UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> <Route r> Path in => out </Route>

Hi everyone! You many help me, thanks a lot. I hope you kind to help me now. My NXLog clients don't collect Windows System logs. And now I often see in my logs this message: ``` 2019-06-04 17:49:50 INFO nxlog-4.3.4308 started 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. * * 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events [error code: 1717]; The interface is unknown. ``` My config: ``` define ROOT C:\nxlog define NXLOGLOGFILE %ROOT%\data\nxlog.log define CERTDIR %ROOT%\cert PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval 0 CacheSync TRUE Module im_msvistalog ReadFromLast TRUE * Exec $FileName = 'winapp.log'; Exec $EventTime = $EventReceivedTime; Module im_msvistalog ReadFromLast TRUE * Exec $FileName = 'winsys.log'; Exec $EventTime = $EventReceivedTime; BufferSize 9500000 Module om_batchcompress Host 192.168.100.100 Port 1514 UseSSL true AllowUntrusted TRUE CAFile %CERTDIR%\cacert.pem CertFile %CERTDIR%\clientcert.pem CertKeyFile %CERTDIR%\clientkey.pem Path winapp, winsys => out ``` After restart service nothing new. Any ideas, please!

Hello,

I am trying to send some logs into ELK and I am running into a bit of a snag. The logs are delimited by space and there doesn't seem to be an options to change that easily.

I am not really sure how to go about getting the logs sent to my monitoring solution in a formatted way, preferably JSON.

These are apache error logs:

[Fri May 31 14:21:38 2019] [error] [client 1.1.1.1] File does not exist: /home/test/test.xml, referer: https://www.test-group.com/

NxLog conf:

define REGEX /(?x)^[\S+\ ([^]]+)]\ [(\S+):(\S+)]\ ([client\ (\S+)]\ )?(.+)$/

<Extension multiline>

Module xm_multiline
HeaderLine %REGEX%

</Extension>

<Input in>

Module          im_file
File            "C:\\path\\\*.log"
InputType       multiline
SavePos         FALSE
ReadFromLast    FALSE
<Exec>
    if $raw_event =~ /(?x)^\[\S+\ ([^\]]+)\]\ \[(\S+):(\S+)\]\ (\[client\ (\S+)\]\ )?(.+)$/
    {
        $EventTime = parsedate($1);
        $ApacheModule = $2;
        $ApacheLogLevel = $3;
        $Message = $4;
    }
</Exec>

It's sending the logs to ELK, but the data isn't in a usable format there. Everything looks just like it does in plain text, no fields or values. If I add the "exec to_json();" option, then I have empty logs in ELK. Maybe something is wrong with my regex, but I copied what I could from the reference manual for this log, though this log is missing data from the example in the guide.

Thanks for your time

Hi, I'm trying to send json log files to a syslog server but it doesn't works. I can see the the tcp connection established with the syslog server but nothing is sent. Seems like the json file is not read. I will appreciate if someone can help me. Below, my nxlog.conf:

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log

LogLevel DEBUG

LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension json> Module xm_json </Extension>

#locoal Server <Input in> Module im_file File 'k:\data\json\log\log2019.json*' Exec parse_json(); </Input>

To syslog server

<Output out> Module om_tcp Host x.x.x.x Port 514 </Output>

<Route 1> Path in => out </Route>

I have a fresh install of CentOS 7 and NXLog EE.

The Config File was restored from a previous install.

The logs are coming into the system but do not appear to be sending out. The only events I see in the NXLog.Log file are related to the SSL Cert not being available for agent-ca.pem.

2019-05-30 16:51:27 INFO nxlog-4.4.4347 started 2019-05-30 16:51:27 ERROR SSL error, failed to load ca cert from '/opt/nxlog/var/lib/nxlog/cert/agent-ca.pem', reason: No such file or directory, no such file, system lib 2019-05-30 16:53:11 INFO configuration OK

I would assume this is at the system level but am not sure what it might be.

Anyone have any experience setting up and configuring NXLog on CentOS?

1. System Requirements: what are hardware and software requirements for Nxlog Enterprise edition.
2. Available Integrations: what are the inputs supported for the tool(file, database, API, LDAP, etc...)
3. Kafka Output Available: can we forward output to Kafka
4. Agent Mechanism: With one agent how many logs we can transfer. is it one per system?
5. Customizable: can we customize the code according to the requirements
6. Price: Pricing details

Can anyone help me to find these details anyone of the above?