Hi, It's basically what the title implies. I want to run a command that produces a string and save it as a variable to be used later in various output blocks. So far the command is working and I can send the output to a om_file logfile but I don't know how to assign the value to a variable to be used elsewhere in the config. ``` Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "$IP = Get-WmiObject Win32_NetworkAdapterConfiguration | where {$_.IPEnabled};$IP.DefaultIPGateway" parse_json(); $EventTime = parsedate($EventTime); ``` Thanks!!

Hello,

The documentation about the support of SNARE format (https://nxlog.co/documentation/nxlog-user-guide/snare.html) describes how the account name should be passed.

However, the function to_syslog_snare(), puts N/A in that field instead of the username in the Windows event. This happens both in the example output (https://nxlog.co/documentation/nxlog-user-guide/snare.html#generating-snare) and with the latest nxlog community edition. Is this a bug or a paying feature of the enterprise edition?

Sincerely

Hi all,

I'm trying to deploy NXLog Enterprise to a couple of Windows domain controllers, pointed to Graylog to audit security. As part of this, we need the "ResolveSID" feature so have gone Enterprise edition. Unfortunately only got a 1 year sub approved which doesn't allow enterprise support :(

However with Enterprise edition, the only Security events that NXLog sends to Graylog are "Event log automatic backup" events when the .evtx files get rotated - nothing else from Security (all other sources seem OK).

The interesting thing here is that the Community edition doesn't have this problem - security events are forwarded just fine. I've also tried the 32bit v4 Enterprise MSI (since the Community edition is 32bit), but it exhibits the same behaviour as above. The v3 Enterprise edition seems to mostly work, but ignores "ResolveSID TRUE" (it reads the setting ok, I've tested this by changing it to a non-boolean value to test that it read it to complain about it, and it did, but when set to TRUE, it still sends unresolved numeric SIDs through for Event ID 4627 "Group membership information" events)

Does anyone have any further troubleshooting tips for either of these problems? Ideally I'd like to get v4 working.

Thanks

I am trying to use nxlog to read from a text file and send to a syslog collector from Rapid7.

I have read through Rapid7's documentation found at https://insightidr.help.rapid7.com/docs/nxlog

I have verified that the input is working by outputting to a text file and verifying there was output, but it will not output to syslog. This is traversing a firewall and I have the proper rules in place, I can also see that the traffic is not getting to the firewall as there are no packets dropped or captured at the firewall so my only logical conclusion is nxlog is not sending the output to syslog. ICMP (Ping) traffic goes through the firewall so there is connectivity.

Is there a debug log mode for nxlog where I can get more details on what is happening.

Here is my config file: Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension fileop> Module xm_fileop </Extension>

<Input Ping_Audit> Module im_file File 'C:\ping-logs\audit.log' SavePos TRUE ReadFromLast TRUE PollInterval 1 <Exec>
if $raw_event =~ /^#. drop(); else { to_syslog_bsd(); } </Exec> </Input>

<Output Rapid7_5004> Module om_udp Host 192.168.251.201 Port 5004 </Output>

<Route 1> Path Ping_Audit => Rapid7_5004 </Route>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

Hi,

I'm using NXLog on Windows to send Event Log data into Google BigQuery but I'm not seeing my data. There appears to be no errors in nxlog.log When I use this: <Output out> Module om_file File 'C:\Program Files (x86)\nxlog\data\nxlogfile.log' </Output>

I can see the data from the event log being logged.

Is there a module that logs all data as it's being transmitted?

Thanks in advance!!

I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file: ``` Module xm_csv Fields $time, $date, $host, $from, $ip, $loginfo, $color FieldTypes string, string, string, string, string, string, integer Delimiter | Module im_file File "C:\\M2PLogs\\log*" SavePos TRUE if $raw_event =~ /^#/ drop(); else { csv->parse_csv(); if ( not defined $color ) drop(); $message = $raw_event; $raw_event = to_json(); } ``` In this case, if the line doesn't have 6 fields, I understand the `$color` field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text. Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

Hi everybody!

I have a problem with collecting logs.

Сlient application logs:

2020-01-09 15:24:54 INFO connected to server OK
2020-01-09 15:25:22 INFO reconnecting in 1 seconds
2020-01-09 15:25:22 ERROR remote ssl socket was reset? (SSL_ERROR_SYSCALL with errno=9); End of file found

TCP dump at the moment error:

C: Client Hello
S: Server Hello, Certificate, Certificate Request, Server Hello Done
C: Certificate, Client Key Exchange, Certificat Verify, Change Cipher Spec, Encrypted Handshake Message
S: New Session Ticket, Change Cipher Spec, Encrypted Handshake Message
C: Application Data
S: Encrypted Alert

And part of the data segment is looped, the infinitely the same fragment of data is stored in the log file on the server side.

How I may to detect the cause of this problem? I hope you help me, please. May be, I need to correct deep parameters of network settings? Thank you!

Hi! Please help,

Is it possible to collect single line logs from file, without newline characters (SAP for example) with NxLog?

In Splunk Heavy Forwarder this function work with Line Breaker(Regex)

LINE_BREAKER = ([23])[A-Z][A-Z][A-Z0-9]\d{14}00

Hi All,

I recently upgraded a machine (used for evaluating graylog) from CentOS 7 to CentOS 8 and it turns out that nxlog was removed in the process.

Now I can't find it in the CentOS 8 dnf packages.

Is nxlog available for Centos 8? and if so, who can I install it?

Many thanks,

Oren

Hi there,

I'm just trying to find the Community Edition Windows installer MSI hashes/checksums (MD5, SHA-256 etc.) on your website but I'm struggling.

I also see that the installer itself doesn't have a digital signature.

Can you point me to where the installer hashes are if you have them? I want to be able to verify the integrity of my download before deployment.

Best regards!

I have download an eval copy of nxlog EE, and am trying to send Microsoft DNS logs to my Splunk Cloud instance. I've read thru various documentation, but am getting an error "ERROR SSL certificate verification failed: self signed certificate in certificate chain (err: 19)", which is confusing me, because I am sending this to Splunk Cloud over HEC, which should have proper cert chain, so I'm not sure where it's getting a self-signed cert from. Here is what my config file looks like (not the whole thing, just the points of interest):

<Input dns_analytical> Module im_etw Provider Microsoft-Windows-DNSServer </Input>

<Output splunk_out> Module om_http URL https://http-inputs-xxx.splunkcloud.com/services/collector AddHeader Authorization: Splunk <auth key from Splunk HEC> ... </Output>

<Route splunk> Path dns_analytical => splunk_out </Route>

I could use on my local on-prem heavy forwarder but I would really like to send directly to Splunk Cloud over HEC, since it will minimize some moving parts.

We are utilizing nxlog win with the im_msvistalog module. Fairly simplified json output file (om_file) with an hourly rotation. AWS Kinesis Tap is configured to read from this file as a source. We continue to experience strange situations where nxlog will appear to be running healthy as a service, but will be writing 0 lines to the output file as if the mswineventlog back end is just not functioning. When this occurs the nxlog configuration schedule will executed as expected, and rotate 0KB files into the destination folder. This is running on two domain controllers with wineventlog seeing anywhere from 20-30 individual logs per second. Is anyone utilizing this type of configuration and experiencing any similar issues? Strangely enough a search in the forum for "kinesis" was returned with 0 results. Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_json Module xm_exec Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); #Get logs from Windows EventLog API Module im_msvistalog #Send logs to customized file #define OUTDIR C:\Program Files (x86)\logs\data_nxlogs define OUTDIR D:\nxlog-logs Exec create_var('offset'); Module om_file File '%OUTDIR%\output.json' When @hourly if not dir_exists('%OUTDIR%\Older_Logs') dir_make('%OUTDIR%\Older_Logs'); rotate_to('%OUTDIR%\Older_Logs\' +strftime(now(), '%m_%d_%Y-%H_%M.log')); $EventTime = strftime($EventTime, '%Y-%m-%d %H:%M:%S %z'); to_json(); #Check back the previous logs and delete all logs older than 24hrs When @hourly #log_info(now()); #log_info(now() - 86400); set_var('offset', now() - 86400); if file_exists('D:\nxlog-logs\Older_Logs\' +strftime(get_var('offset'), '%m_%d_%Y-%H_%M.log')) file_remove('D:\nxlog-logs\Older_Logs\' +strftime(get_var('offset'), '%m_%d_%Y-%H_%M.log')); Path eventlog => out

I can't get NXlog to read hidden files in Windows server 2012 R2. The source files are in this hidden folder: C:\ProgramData. There are no errors in the log file even if i put the debug mode on. I've double checked the path in input module and it is correct. The NXLog version is 4.4.4347. I also tried to look at Windows logs but no sign of problems there either. Any ideas on what to look next?

Scenario: Two inputs are configured. One is incorrect and results in a 400 response from the endpoint defined in om_http. The other input can send events without generating an error.

Observed behavior: NXLog will continue to resend an event that failed which causes a loop condition that prevents any other events from being sent.

Is there a way to change this behavior?

Thanks!

Hi All,

Is there a better solution to capturing IIS logs across multiple sites on a single server than adding an input per site in nxlog.conf?

Thanks,

Matt.

currently running 2.10.2150 on windows with a config that reads a debug DNS log (on c:), parses the logs, drops 99% of the logs, and writes the remainder out with file_write()

we are seeing this memory pool allocation error and looking for info about if it's a known issue, something that we should be changing the config to deal with, or what's happening.

our current config is something very close to (filenames may vary):

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert #define CONFDIR %ROOT%\conf define CONFDIR "C:\Program Files (x86)\nxlog\conf define LOGDIR %ROOT%\data #define LOGFILE %LOGDIR%\nxlog.log define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% #LogLevel DEBUG define OUTFILE H:\dnsadvlogs\dns-filtered.log

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _json> Module xm_json </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the log file size every hour and rotate if larger than 5 MB
<Schedule>
    Every 1 hour
    <Exec>
        if (file_exists('%LOGFILE%') and file_size('%LOGFILE%') >= 5M)
            file_cycle('%LOGFILE%', 8);
    </Exec>
</Schedule>

# Rotate log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

<Extension _fileop2> Module xm_fileop

# Rotate log file every week on Sunday at midnight
<Schedule>
    Every 5 min
    Exec    if file_exists('%OUTFILE') file_cycle('%OUTFILE%', 8);
</Schedule>

</Extension>

<Extension win_dns_parser> Module xm_multiline # look for a date at the start of the line HeaderLine /^\d+/\d+/\d+/ #filter blank and header lines from the input Exec if $raw_event =~ /^\s*$/ drop(); Exec if $raw_event =~ /^DNS Server log file creation/ drop(); Exec if $raw_event =~ /^Log file wrap at / drop(); Exec if $raw_event =~ /^Message logging key/ drop(); Exec if $raw_event =~ /^\sField # Information Values/ drop(); Exec if $raw_event =~ /^\s------- ----------- ------/ drop(); Exec if $raw_event =~ /^\s1 Date/ drop(); Exec if $raw_event =~ /^\s2 Time/ drop(); Exec if $raw_event =~ /^\s3 Thread ID$/ drop(); Exec if $raw_event =~ /^\s4 Context$/ drop(); Exec if $raw_event =~ /^\s5 Internal packet identifier$/ drop(); Exec if $raw_event =~ /^\s6 UDP/TCP indicator$/ drop(); Exec if $raw_event =~ /^\s7 Send/Receive indicator$/ drop(); Exec if $raw_event =~ /^\s8 Remote IP$/ drop(); Exec if $raw_event =~ /^\s9 Xid (hex)$/ drop(); Exec if $raw_event =~ /^\s10 Query/Response R = Response$/ drop(); Exec if $raw_event =~ /^\sblank = Query$/ drop(); Exec if $raw_event =~ /^\s11 Opcode Q = Standard Query$/ drop(); Exec if $raw_event =~ /^\sN = Notify$/ drop(); Exec if $raw_event =~ /^\sU = Update$/ drop(); Exec if $raw_event =~ /^\s*? = Unknown$/ drop(); Exec if $raw_event =~ /^\s12 [ Flags (hex)$/ drop(); Exec if $raw_event =~ /^\s13 Flags (char codes) A = Authoritative Answer$/ drop(); Exec if $raw_event =~ /^\sT = Truncated Response$/ drop(); Exec if $raw_event =~ /^\sD = Recursion Desired$/ drop(); Exec if $raw_event =~ /^\sR = Recursion Available$/ drop(); Exec if $raw_event =~ /^\s14 ResponseCode ]$/ drop(); Exec if $raw_event =~ /^\s15 Question Type$/ drop(); Exec if $raw_event =~ /^\s16 Question Name/ drop(); </Extension>

<Input win_dns> Module im_file #File "H:\dnsadvlogs\dns.log" File "H:\dnsadvlogs\dns.log" # for testing we want to re-read from the start of the file each time SavePos False ReadFromLast True InputType win_dns_parser <Exec> $Message = $raw_event; $Message = replace($Message, "\r", ""); </Exec> </Input>

<Output win_dns_trimmed> Module om_file

File "H:\dnsadvlogs\dns-trimmed.log"

File      "H:\dnsadvlogs\dns-trimmed.log"
<Exec>
    # manipulate the log entry here, not in input so that we can do other things with the raw input as well
    $Message =~ /^(?<timestamp>\d+\/\d+\/\d+ \d+\:\d+\:\d+\s+\S+)\s+(?<pid>\d+)\s+(?<win_dns_type>[^ ]*)\s+(?<win_dns_packetID>[^ ]*)\s+(?<win_dns_protocol>[^ ]*)\s+(?<win_dns_direction>[^ ]*)\s+(?<win_dns_IP>[^ ]*)\s+(?<win_dns_hexID>[^ ]*) (?<win_dns_qr>.) (?P<win_dns_opcode>.) \[(?<win_dns_flags_hex>\S+) (?<win_dns_flags>.*) (?<win_dns_resultcode>\S+)\]\s+(?<win_dns_recordType>\S+)\s+(?<win_dns_query>\S*).*ANSWER SECTION:.(?<answer>.+)\s+AUTHORITY SECTION:/s; 
    $timestamp=$1;
    $pid=$2;
    $win_dns_type=$3;
    $win_dns_packetID=$4;
    $win_dns_protocol=$5;
    $win_dns_direction=$6;
    $win_dns_IP=$7;
    $win_dns_hexID=$8;
    $win_dns_qr=$9;
    $win_dns_opcode=$10;
    $win_dns_flags_hex=$11;
    $win_dns_flags=$12;
    $win_dns_resultcode=$13;
    $win_dns_recordType=$14;
    $win_dns_query=$15;
    $answer=$16;
    # drop messages if they are not replies (since the replies contain the query info)
    if $win_dns_qr == " " drop();
    # drop logs that have no answer info
    #if $answer =~ /^\s+empty\s+$/ drop();
    # drop logs that don't parse (if we don't have a requestion IP address, the log is worthless to UBA)
    if not defined $win_dns_IP drop();
    # drop logs from dnsmasq caching servers.
    #if $win_dns_IP IN ("10.16.169.32","10.49.58.4","10.49.58.3","10.16.6.22","199.47.139.239","199.47.139.238","199.47.139.182") drop();
    # for the first pass, just filter the logs, don't change the format
    # this greatly simplifies the Splunk changes needed as the log parsing doesn't need to change
    $orig = replace($raw_event, "\r", "\r\n") + "\r\n";
    file_write("%OUTFILE%",$orig); drop();
    delete($orig); delete($Message);
    delete($EventReceivedTime); delete($SourceModuleName); delete($SourceModuleType); delete($pid);
    delete($win_dns_type); delete($win_dns_packetID); delete($win_dns_protocol); delete($win_dns_direction);
    delete($win_dns_hexID); delete($win_dns_qr); delete($win_dns_opcode); delete($win_dns_flags_hex);
    delete($win_dns_flags);
    $answer = replace($answer,"(3)",".");
    $answer = replace($answer,"(6)",".");
    $answer = replace($answer,"(9)","");
    $answer = replace($answer,"(0)","");
    $answer =~ s/\[\S\S\S\S\]//g;
    $win_dns_query = replace($win_dns_query,"(3)",".");
    $win_dns_query = replace($win_dns_query,"(6)",".");
    $win_dns_query = replace($win_dns_query,"(9)","");
    $win_dns_query =~ s/\(0\)UDP//;
    $win_dns_query =~ s/\(0\)TCP//;
    $win_dns_query =~ s/\[\S\S\S\S\]//g;
    rename_field("win_dns_query","q");
    rename_field("win_dns_resultcode","rc");
    rename_field("win_dns_IP","ip");
    rename_field("win_dns_recordType","type");
    if $rc == "NXDOMAIN" delete($answer);
    to_json();
</Exec>

</Output> <Route win_dns_route> Path win_dns => win_dns_trimmed </Route>

Hello,

I've read a few posts about ingesting multiple .evt files using the im_msvista module. Is there a way to do this?

I can point to one, but I need to look at several.

thank you! Franz

i am getting a cab error when loading on server 2016

Hello, I want to know if it's possible to have a dynamic variable like that : if .... {
$1 = $2 ; } Thanks !

Hello!

How can i get logs from shared folder by domain user? Should i start nxlog service under this user or there is another way?