Hi, I'm testing Nxlog EE trial. nxlog-trial-4.4.4347_windows_x64.msi

And configured nxlog on windows host, but om_kafka output module won't work. with error

2019-06-14 18:28:47 ERROR Failed to load module from C:\Program Files\nxlog\modules\output\om_kafka.dll, The specified module could not be found.  ; The specified module could not be found.  
2019-06-14 18:28:47 ERROR module 'kafka' is not declared at C:\Program Files\nxlog\conf\nxlog.conf:84

However om_kafka.dll100% persist in folder C:\Program Files\nxlog\modules\output\ I'm trying to reinstall, repair, install on x32 and x64 same as Windows Server 2012R2 and Windows Server 2016

Similar module om_kafka works fine on Centos7

Seems that “out of the box” this module is not working on OS windows , I mean, what am I missing? maybe any additional librdkafka installation required?

Hi, I'm testing Nxlog EE trial And configured nxlog as WEC with im_wseventing module, but for some reason `$EventType` field Parsed to simple "`AUDIT`" not `AUDIT_SUCCESS` or `AUDIT_FAILURE` In doc Possible values are: `CRITICAL, ERROR, AUDIT_FAILURE, AUDIT_SUCCESS, INFO, WARNING, and VERBOSE`. Example of Event: ``` json Jun 14 15:13:33 SRVTEST-00.test Microsoft-Windows-Security-Auditing[648]: { "MessageID": "uuid:1DB8B636-E34C-4DB5-951D-EEE30FD8F837", "SourceName": "Microsoft-Windows-Security-Auditing", "ProviderGuid": "{54849625-5478-4994-A5BA-3E3B0328C30D}", "EventID": 4634, "Version": 0, "LevelValue": 0, "EventType": "AUDIT", "SeverityValue": 2, "Severity": "INFO", "OpcodeValue": 0, "Keywords": "0x8020000000000000", "EventTime": "2019-06-14 15:13:33", "RecordNumber": 3437460, "ExecutionProcessID": 648, "ExecutionThreadID": 4980, "Channel": "Security", "Hostname": "SRVTEST-00.test", "TargetUserSid": "S-1-5-18", "TargetUserName": "SRVTEST-00$", "TargetDomainName": "TEST", "TargetLogonId": "0x2b06461", "LogonType": "3", "Message": "An account was logged off.\n\nSubject:\n\tSecurity ID:\t\tS-1-5-18\n\tAccount Name:\t\tSRVTEST-00$\n\tAccount Domain:\t\tTEST\n\tLogon ID:\t\t0x2B06461\n\nLogon Type:\t\t\t3\n\nThis event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.", "Level": "Information", "Task": "Logoff", "Opcode": "Info", "EventReceivedTime": "2019-06-14 15:13:35", "SourceModuleName": "wseventin", "SourceModuleType": "im_wseventing", "HostIP": "192.168.5.5" } ``` My nxlog config: ``` User nxlog Group nxlog Panic Soft # default values: PidFile /opt/nxlog/var/run/nxlog/nxlog.pid CacheDir /opt/nxlog/var/spool/nxlog ModuleDir /opt/nxlog/libexec/nxlog/modules SpoolDir /opt/nxlog/var/spool/nxlog define LOGDIR /opt/nxlog/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log LogFile %MYLOGFILE% Module xm_syslog Module xm_json Module xm_resolver Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') >= 5M) ) { file_cycle('%MYLOGFILE%', 8); } # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); Module im_wseventing Address http://srvtest-12.test:80/wsman ListenAddr 0.0.0.0 Port 80 SubscriptionName testing Exec $HostIP = name_to_ipaddr($Hostname); Exec log_info(to_json()); * * * Module om_file File '/opt/nxlog/var/log/nxlog/winevent.log' CreateDir TRUE Exec $Message = to_json(); to_syslog_bsd(); Path wseventin => tofile ``` Is it bug or trial restrictions?

Hello I've been trying to the use linuxaudit system to work but I'm stuck.

--- Nxlog-agent setup --- OS: SUSE Tumbleweed 20190512 Agent-Version: 4.4.4347 Module: im_linuxaudit

--- Configuration --- <Extension _json> Module xm_json </Extension>

<Extension audit_parser> Module xm_kvp KVPDelimiter ' ' KVDelimiter = EscapeChar '' </Extension>

<Input audit> Module im_linuxaudit FlowControl FALSE <Rules> -D -b 320 -w /etc/passwd -p wa -k etcpasswd -w /bin/cat -p wxa -k cat_exection -e 1 </Rules> <Exec> audit_parser->parse_kvp(); $Hostname = hostname(); $FQDN = hostname_fqdn(); $Tag = "audit"; $SourceName = "auditd_nxlog"; </Exec> </Input>

<Output tcp> Module om_tcp Host 192.168.4.58 Port 1337 Exec to_json(); to_syslog_bsd(); </Output>

<Route audit_to_tcp> Path audit => tcp </Route>

I've been trying to run the above mentioned audit config however whenever I'm doing /bin/cat/ /etc/passwd I don't get any kind of alerts on my receiver box. HOWEVER I will get logs when I change user (e.g. su otheruser). Additionally I was wondering if the log format 'ENRICHED' from the normal audit daemon is supported. https://github.com/linux-audit/audit-documentation/wiki/SPEC-Audit-Event-Enrichment

Best regards Florian Reiter

Hi everybody! Today I found in the logs of NXLog 4.3.4308 Server very odd error: ``` 2019-06-12 11:22:04 ERROR failed to open file when trying to truncate: Too many open files ``` The service was not working at this time until I has restarted it. Could you be please so kind tell me what is the limit of opened files? What's the number simultaneous connections can the service hold? Thanks!

I see in the documentation that Nano is supported but I don't see Server Core mentioned explicitly.

Thanks,

Hi all, I have this config ``` Panic Soft define ROOT C:\Program Files (x86)\nxlog define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_multiline FixedLineCount 2 Module im_file File "C:\\txt\\event.txt" InputType multilines Module om_file File "C:\\txt\\txt1.log" Path InputData => OutputData ``` And this input log file ``` event1 Data1 event2 Data2 event3 Data3 event4 Data4 event5 Data5 event6 Data6 ``` But output file is always empty and nxlog.log is without errors or warnings. I want to merge two lines into single line.

When I use im_file and om_odbc,from the log,I get this message:WARNING SQLExecDirect failed; HY000:1:0:[Devart][ODBC][PostgreSQL]在字段 "timestamp" 中空值违反了非空约束 <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log1.txt" ReadFromLast False SavePos False <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec>

Exec parse_syslog();

</Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1;
UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> #<Output out>

Module om_file

File "C:\Users\jiang.dengjie\Desktop\logtest.txt"

Exec to_json();

#</Output> <Route r> Path in => out </Route> In my file: %Feb 5 15:47:32:118 2015 trust-access IFNET/5/LINK_UPDOWN: Line protocol on the interface GigabitEthernet1/0/41 is down. %Feb 5 15:47:35:367 2015 trust-access IFNET/3/PHY_UPDOWN: GigabitEthernet1/0/40 link status is up. And I want to use nxlog to save this file to my pgsql which has the table that has five colum:id,timestamp,hostname,eventname,event. Also if any viedo about how to use nxlog? Thank you very much.

I use im_file and om_file on windows,But throgh om_file,I get a file that is empty <Extension _syslog> Module xm_syslog </Extension> <Extension _json> Module xm_json </Extension> <Input in> Module im_file File "C:\Users\jiang.dengjie\Desktop\log.txt" Exec parse_syslog(); </Input> <Output out> Module om_file File "C:\Users\jiang.dengjie\Desktop\logtest.txt" Exec to_json(); </Output> <Route r> Path in => out </Route>

I use nxlog in windows,from the log,I get this question:ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib Then I find I do not have the agent-ca.pem. And in my environment,my pgsql do not get any data. below is my config. <Input in> Module im_file File "C:\Users\xxx\Desktop\log.txt" <Exec> if $raw_event =~ /^(\w+ \d+ \S+ \S+) (\S+) (\S+):(.+)$/ { $timestamp = $1; $hostname = $2; $eventname = $3; $event = $4; } </Exec> </Input> <Output out> Module om_odbc ConnectionString Driver={Devart ODBC Driver for PostgreSQL}; Server=127.0.0.1;
UID=qytangdbuser; PWD=Cisc0123; Database=qytangdb SQL "INSERT INTO qytdb_network_log (timestamp, hostname, eventname, event) VALUES (?,?,?,?)",$timestamp,$hostname,$eventname,$event </Output> <Route r> Path in => out </Route>

Hi everyone! You many help me, thanks a lot. I hope you kind to help me now. My NXLog clients don't collect Windows System logs. And now I often see in my logs this message: ``` 2019-06-04 17:49:50 INFO nxlog-4.3.4308 started 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. * * 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events [error code: 1717]; The interface is unknown. ``` My config: ``` define ROOT C:\nxlog define NXLOGLOGFILE %ROOT%\data\nxlog.log define CERTDIR %ROOT%\cert PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval 0 CacheSync TRUE Module im_msvistalog ReadFromLast TRUE * Exec $FileName = 'winapp.log'; Exec $EventTime = $EventReceivedTime; Module im_msvistalog ReadFromLast TRUE * Exec $FileName = 'winsys.log'; Exec $EventTime = $EventReceivedTime; BufferSize 9500000 Module om_batchcompress Host 192.168.100.100 Port 1514 UseSSL true AllowUntrusted TRUE CAFile %CERTDIR%\cacert.pem CertFile %CERTDIR%\clientcert.pem CertKeyFile %CERTDIR%\clientkey.pem Path winapp, winsys => out ``` After restart service nothing new. Any ideas, please!

Hello,

I am trying to send some logs into ELK and I am running into a bit of a snag. The logs are delimited by space and there doesn't seem to be an options to change that easily.

I am not really sure how to go about getting the logs sent to my monitoring solution in a formatted way, preferably JSON.

These are apache error logs:

[Fri May 31 14:21:38 2019] [error] [client 1.1.1.1] File does not exist: /home/test/test.xml, referer: https://www.test-group.com/

NxLog conf:

define REGEX /(?x)^[\S+\ ([^]]+)]\ [(\S+):(\S+)]\ ([client\ (\S+)]\ )?(.+)$/

<Extension multiline>

Module xm_multiline
HeaderLine %REGEX%

</Extension>

<Input in>

Module          im_file
File            "C:\\path\\\*.log"
InputType       multiline
SavePos         FALSE
ReadFromLast    FALSE
<Exec>
    if $raw_event =~ /(?x)^\[\S+\ ([^\]]+)\]\ \[(\S+):(\S+)\]\ (\[client\ (\S+)\]\ )?(.+)$/
    {
        $EventTime = parsedate($1);
        $ApacheModule = $2;
        $ApacheLogLevel = $3;
        $Message = $4;
    }
</Exec>

It's sending the logs to ELK, but the data isn't in a usable format there. Everything looks just like it does in plain text, no fields or values. If I add the "exec to_json();" option, then I have empty logs in ELK. Maybe something is wrong with my regex, but I copied what I could from the reference manual for this log, though this log is missing data from the example in the guide.

Thanks for your time

Hi, I'm trying to send json log files to a syslog server but it doesn't works. I can see the the tcp connection established with the syslog server but nothing is sent. Seems like the json file is not read. I will appreciate if someone can help me. Below, my nxlog.conf:

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log

LogLevel DEBUG

LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension json> Module xm_json </Extension>

#locoal Server <Input in> Module im_file File 'k:\data\json\log\log2019.json*' Exec parse_json(); </Input>

To syslog server

<Output out> Module om_tcp Host x.x.x.x Port 514 </Output>

<Route 1> Path in => out </Route>

I have a fresh install of CentOS 7 and NXLog EE.

The Config File was restored from a previous install.

The logs are coming into the system but do not appear to be sending out. The only events I see in the NXLog.Log file are related to the SSL Cert not being available for agent-ca.pem.

2019-05-30 16:51:27 INFO nxlog-4.4.4347 started 2019-05-30 16:51:27 ERROR SSL error, failed to load ca cert from '/opt/nxlog/var/lib/nxlog/cert/agent-ca.pem', reason: No such file or directory, no such file, system lib 2019-05-30 16:53:11 INFO configuration OK

I would assume this is at the system level but am not sure what it might be.

Anyone have any experience setting up and configuring NXLog on CentOS?

1. System Requirements: what are hardware and software requirements for Nxlog Enterprise edition.
2. Available Integrations: what are the inputs supported for the tool(file, database, API, LDAP, etc...)
3. Kafka Output Available: can we forward output to Kafka
4. Agent Mechanism: With one agent how many logs we can transfer. is it one per system?
5. Customizable: can we customize the code according to the requirements
6. Price: Pricing details

Can anyone help me to find these details anyone of the above?

Over the past couple weeks we've noticed high CPU usage on the nxlog service of up to 40%, average is around 25%. As a resolution, we have been stopping the nxlog service as well as the sysmon service, then starting them back up, and after about 10 minutes it finally drops back down to the normal cpu usage.

We are using nxlog CE 2.10.2102 - has anyone else seen this issue?

Will updating to the most current version resolve the issue?

I appreciate any feedback/input!

Thanks

Hi All,

Im trying to get Microsoft NPS text file logs sent on to our firewall. I can get them forwarded if they are in .xml format as DTS compatible logs however I need them as IAS type files. I understand that there is a native parser for the Enterprise Edition and I have tried to obtain a quote form NXLog but as of yet not received any communication. Has anyone else out there achieved this with the CE version of nxlog and could they offer some advise on what the config would look like.

Thanks

:-)

Hi, is it possibe to resolve GUIDS in EventID:4624 Saw an other post from last year talking about it would be implemented i EE and after that CE. We currently running ce version and the option for ResolveSID dont work. Does anyone have a solution for this or a workaround as it looks ok in the Eventviewer where it's resolved correctly. We currently run nxlog-ce-2.9.1716.

I'm on an enterprise edition trial, we've current got logs being pushed into a time series database using fluentd but I'm also wanting to push the logs into IBM QRadar, but I'm struggling.

Has anyone achieved this? Any assistance with config would be perfect.

Thanks in advance.

There are many convenient string functions and procedures in the nxlog language, but I did not see one for urldecode. I have some data flowing through the web logs that the client is encoding, and it would be useful to have nxlog decode it. For example, something like:

    $data = urldecode("%61%6a%61%78%5f%66%6f%72%6d");   # $data = "ajax_form"

In addition to requesting that functionality, is there any workarounds you could suggest?

Hi,

We are planning to do a migration for nxlog manager, which involve a IP change for the nxlog manager. Are there any prerequisites to be done on the Manager before this can be done? I know that the agent's configurations will have to point to the new IP. So the plan would be to send them a new config with the manager's new IP, then change the nxlog manager to the new IP The nxlog manager is installed on Linux 7.5, running latest nxlog manager version 5.4.5209.

Thank you.