Hello, I'm using nxlog community to send logs from my firewalls through syslog. My output is like that: <output log_to_file> module om_file File 'c:\datas\firewall_' + $MessageSourceAddress + '.log' <output> If my firewalls 192.168.1.1 and 192.168.1.2 are correctly configured, the result will be two files: c:\datas\firewall_192.168.1.1.log and c:\datas\firewall_192.168.1.2.log My problem is now to rotate this file on a daily basis. I've tried to apply the command rotate_to but it seems that it applies only on the first file.

How can I do rotation on multiple files with names based on a variable ?

Thank you !

nxlog-ce 2.9.1716 on Windows 10/Server 2016.

The usage of om_udp seems to cause nxlog.exe to listen on ephemeral port. om_tcp does not cause this. I can't find anything in documentation that explains this behavior.

Please help.

MK

After using kvp parser i've got variables with spaces in names. For example: "$Event Time" or "$Source Name".

I'm interested in two things:

1. How i can interact with this variable names? For example i'm trying construction "$EventTime = $Event Time;" with many shield variations: ",',),], etc, but this not work.
2. it is possible to prevent this situation? Massage format example below:

"DeviceEvent: Virus found,IP Address: 10.X.X.X,Computer name: xxx-xxx,Source: Auto-Protect scan,Risk name: Infostealer.Gampass,Occurrences: 1,File path: X:\xxxx_xxx.exe,Description: ,Actual action: Moved back,Requested action: Quarantined,Secondary action: Deleted,Event time: 2020-01-21 17:24:58,Event Insert Time: 2020-01-21 17:27:06,End Time: 2020-01-21 17:59:17,Last update time: 2020-01-21 18:01:07,Domain Name: xxxx,Group Name: XXXX,Server Name: xx-xxx,User Name: SYSTEM,Source Computer Name: ,Source Computer IP: ,Disposition: Reputation was not used in this detection.........."

Hello, I am sending a message with hostname to my syslog server, my conf is as follows:

define ROOT C: \ Program Files (x86) \ nxlog

Moduledir% ROOT% \ modules CacheDir% ROOT% \ data Pidfile% ROOT% \ data \ nxlog.pid SpoolDir% ROOT% \ data LogFile% ROOT% \ data \ nxlog.log

<Extension _syslog> Module xm_syslog </Extension>

<Input in> Module im_msvistalog

<Exec> parse_syslog (); $ Message = "hostnamexxx" + $ Message; to_syslog_ietf (); </Exec>

</Input>

<Output out> Om_udp module Host xx.xxx.xx Port 514 Exec to_syslog_ietf (); </Output>

<Route 1> Path in => out </Route>

My log is coming with the message correctly:

Feb 12 23:11:34 DESKTOP-XXXXX Microsoft-Windows-Eventlog [964] hostnamexxxxINFO 1102 The audit log was cleared. Subject: Security ID: # xxxxxxxx-1001 Account Name: Admin Domain Name: DESKTOP-XXXXX Logon ID: 0xD438A

However, the message "hostnamexxxx" is coming in the middle of the log, as you can see above. This is disturbing my parser, is there any way I can put this "hostnamexxxx" message last in my log? Example:

Feb 12 23:11:34 DESKTOP-XXXXX Microsoft-Windows-Eventlog [964] INFO 1102 The audit log was cleared. Subject: Security ID: # xxxxxxxx-1001 Account Name: Admin Domain Name: DESKTOP-XXXXX Logon ID: 0xD438A hostnamexxxx

Thanks

Hello everyone,

I'm having trouble architecturing something with hmac verification, any help of yours would be welcome.

I'm trying to setup an architecture with three clients/servers and using hmac/hmac_check to guarantee the integrity of the logs. Logs 1 are created by client 1 and sent to client 2, which check their integrity, logs 2 are created by client 2, and both logs 1 and 2 are send in the end to client 3 which finally check for integrity for both of them. Here is a "beautiful" scheme to illustrate my words:

client 1 ---hmac(logs 1)---> client 2 ---hmac_check(logs 1) + hmac(logs 2)---> client 3 ---hmac_check(logs 1) + hmac_check(logs2) + hmac(logs 3)-->...

I would have multiple routes on each clients and I would use different instances of each processor on each route to avoid having errors like "processor X already used in route A and was not load in the route B". Also, I would be using batchcompress between client 1 and 2 but UDP between client 2 and 3.

I'm wondering how you would do this thing? Would you open multiple UDP ports on client 3 to receive independantly logs coming from client 1 and client 2 and check the hmac independantly or would you send those two logs on the same network port and check them with the same hmac_check processor? And would you use multiple routes to process independantly logs coming from different clients because of the hmac integrity check?

Thank you in advance, Kind Regards,

Say I wanted to run a PowerShell script on an interval using the `im_exec` module how would I do that? ``` Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "script goes here" $output = $raw_event; ``` I can make the interval work by creating pauses in the code but then it appears as though the script process would run forever which is undesirable for efficiency and stability reasons. Is there a way to run the code on a interval, say every fifteen minutes? Thoughts? Thanks in advance!!

Hi, It's basically what the title implies. I want to run a command that produces a string and save it as a variable to be used later in various output blocks. So far the command is working and I can send the output to a om_file logfile but I don't know how to assign the value to a variable to be used elsewhere in the config. ``` Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "$IP = Get-WmiObject Win32_NetworkAdapterConfiguration | where {$_.IPEnabled};$IP.DefaultIPGateway" parse_json(); $EventTime = parsedate($EventTime); ``` Thanks!!

Hello,

The documentation about the support of SNARE format (https://nxlog.co/documentation/nxlog-user-guide/snare.html) describes how the account name should be passed.

However, the function to_syslog_snare(), puts N/A in that field instead of the username in the Windows event. This happens both in the example output (https://nxlog.co/documentation/nxlog-user-guide/snare.html#generating-snare) and with the latest nxlog community edition. Is this a bug or a paying feature of the enterprise edition?

Sincerely

Hi all,

I'm trying to deploy NXLog Enterprise to a couple of Windows domain controllers, pointed to Graylog to audit security. As part of this, we need the "ResolveSID" feature so have gone Enterprise edition. Unfortunately only got a 1 year sub approved which doesn't allow enterprise support :(

However with Enterprise edition, the only Security events that NXLog sends to Graylog are "Event log automatic backup" events when the .evtx files get rotated - nothing else from Security (all other sources seem OK).

The interesting thing here is that the Community edition doesn't have this problem - security events are forwarded just fine. I've also tried the 32bit v4 Enterprise MSI (since the Community edition is 32bit), but it exhibits the same behaviour as above. The v3 Enterprise edition seems to mostly work, but ignores "ResolveSID TRUE" (it reads the setting ok, I've tested this by changing it to a non-boolean value to test that it read it to complain about it, and it did, but when set to TRUE, it still sends unresolved numeric SIDs through for Event ID 4627 "Group membership information" events)

Does anyone have any further troubleshooting tips for either of these problems? Ideally I'd like to get v4 working.

Thanks

I am trying to use nxlog to read from a text file and send to a syslog collector from Rapid7.

I have read through Rapid7's documentation found at https://insightidr.help.rapid7.com/docs/nxlog

I have verified that the input is working by outputting to a text file and verifying there was output, but it will not output to syslog. This is traversing a firewall and I have the proper rules in place, I can also see that the traffic is not getting to the firewall as there are no packets dropped or captured at the firewall so my only logical conclusion is nxlog is not sending the output to syslog. ICMP (Ping) traffic goes through the firewall so there is connectivity.

Is there a debug log mode for nxlog where I can get more details on what is happening.

Here is my config file: Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension fileop> Module xm_fileop </Extension>

<Input Ping_Audit> Module im_file File 'C:\ping-logs\audit.log' SavePos TRUE ReadFromLast TRUE PollInterval 1 <Exec>
if $raw_event =~ /^#. drop(); else { to_syslog_bsd(); } </Exec> </Input>

<Output Rapid7_5004> Module om_udp Host 192.168.251.201 Port 5004 </Output>

<Route 1> Path Ping_Audit => Rapid7_5004 </Route>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

Hi,

I'm using NXLog on Windows to send Event Log data into Google BigQuery but I'm not seeing my data. There appears to be no errors in nxlog.log When I use this: <Output out> Module om_file File 'C:\Program Files (x86)\nxlog\data\nxlogfile.log' </Output>

I can see the data from the event log being logged.

Is there a module that logs all data as it's being transmitted?

Thanks in advance!!

I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file: ``` Module xm_csv Fields $time, $date, $host, $from, $ip, $loginfo, $color FieldTypes string, string, string, string, string, string, integer Delimiter | Module im_file File "C:\\M2PLogs\\log*" SavePos TRUE if $raw_event =~ /^#/ drop(); else { csv->parse_csv(); if ( not defined $color ) drop(); $message = $raw_event; $raw_event = to_json(); } ``` In this case, if the line doesn't have 6 fields, I understand the `$color` field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text. Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

Hi everybody!

I have a problem with collecting logs.

Сlient application logs:

2020-01-09 15:24:54 INFO connected to server OK
2020-01-09 15:25:22 INFO reconnecting in 1 seconds
2020-01-09 15:25:22 ERROR remote ssl socket was reset? (SSL_ERROR_SYSCALL with errno=9); End of file found

TCP dump at the moment error:

C: Client Hello
S: Server Hello, Certificate, Certificate Request, Server Hello Done
C: Certificate, Client Key Exchange, Certificat Verify, Change Cipher Spec, Encrypted Handshake Message
S: New Session Ticket, Change Cipher Spec, Encrypted Handshake Message
C: Application Data
S: Encrypted Alert

And part of the data segment is looped, the infinitely the same fragment of data is stored in the log file on the server side.

How I may to detect the cause of this problem? I hope you help me, please. May be, I need to correct deep parameters of network settings? Thank you!

Hi! Please help,

Is it possible to collect single line logs from file, without newline characters (SAP for example) with NxLog?

In Splunk Heavy Forwarder this function work with Line Breaker(Regex)

LINE_BREAKER = ([23])[A-Z][A-Z][A-Z0-9]\d{14}00

Hi All,

I recently upgraded a machine (used for evaluating graylog) from CentOS 7 to CentOS 8 and it turns out that nxlog was removed in the process.

Now I can't find it in the CentOS 8 dnf packages.

Is nxlog available for Centos 8? and if so, who can I install it?

Many thanks,

Oren

Hi there,

I'm just trying to find the Community Edition Windows installer MSI hashes/checksums (MD5, SHA-256 etc.) on your website but I'm struggling.

I also see that the installer itself doesn't have a digital signature.

Can you point me to where the installer hashes are if you have them? I want to be able to verify the integrity of my download before deployment.

Best regards!

I have download an eval copy of nxlog EE, and am trying to send Microsoft DNS logs to my Splunk Cloud instance. I've read thru various documentation, but am getting an error "ERROR SSL certificate verification failed: self signed certificate in certificate chain (err: 19)", which is confusing me, because I am sending this to Splunk Cloud over HEC, which should have proper cert chain, so I'm not sure where it's getting a self-signed cert from. Here is what my config file looks like (not the whole thing, just the points of interest):

<Input dns_analytical> Module im_etw Provider Microsoft-Windows-DNSServer </Input>

<Output splunk_out> Module om_http URL https://http-inputs-xxx.splunkcloud.com/services/collector AddHeader Authorization: Splunk <auth key from Splunk HEC> ... </Output>

<Route splunk> Path dns_analytical => splunk_out </Route>

I could use on my local on-prem heavy forwarder but I would really like to send directly to Splunk Cloud over HEC, since it will minimize some moving parts.

We are utilizing nxlog win with the im_msvistalog module. Fairly simplified json output file (om_file) with an hourly rotation. AWS Kinesis Tap is configured to read from this file as a source. We continue to experience strange situations where nxlog will appear to be running healthy as a service, but will be writing 0 lines to the output file as if the mswineventlog back end is just not functioning. When this occurs the nxlog configuration schedule will executed as expected, and rotate 0KB files into the destination folder. This is running on two domain controllers with wineventlog seeing anywhere from 20-30 individual logs per second. Is anyone utilizing this type of configuration and experiencing any similar issues? Strangely enough a search in the forum for "kinesis" was returned with 0 results. Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_json Module xm_exec Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); #Get logs from Windows EventLog API Module im_msvistalog #Send logs to customized file #define OUTDIR C:\Program Files (x86)\logs\data_nxlogs define OUTDIR D:\nxlog-logs Exec create_var('offset'); Module om_file File '%OUTDIR%\output.json' When @hourly if not dir_exists('%OUTDIR%\Older_Logs') dir_make('%OUTDIR%\Older_Logs'); rotate_to('%OUTDIR%\Older_Logs\' +strftime(now(), '%m_%d_%Y-%H_%M.log')); $EventTime = strftime($EventTime, '%Y-%m-%d %H:%M:%S %z'); to_json(); #Check back the previous logs and delete all logs older than 24hrs When @hourly #log_info(now()); #log_info(now() - 86400); set_var('offset', now() - 86400); if file_exists('D:\nxlog-logs\Older_Logs\' +strftime(get_var('offset'), '%m_%d_%Y-%H_%M.log')) file_remove('D:\nxlog-logs\Older_Logs\' +strftime(get_var('offset'), '%m_%d_%Y-%H_%M.log')); Path eventlog => out

I can't get NXlog to read hidden files in Windows server 2012 R2. The source files are in this hidden folder: C:\ProgramData. There are no errors in the log file even if i put the debug mode on. I've double checked the path in input module and it is correct. The NXLog version is 4.4.4347. I also tried to look at Windows logs but no sign of problems there either. Any ideas on what to look next?

Scenario: Two inputs are configured. One is incorrect and results in a 400 response from the endpoint defined in om_http. The other input can send events without generating an error.

Observed behavior: NXLog will continue to resend an event that failed which causes a loop condition that prevents any other events from being sent.

Is there a way to change this behavior?

Thanks!