Is there a way to state a headerline to be blank while using multiline module? My data shows as:

data
data
(blank)
data
data
(blank)

Thanks

I've been using NXLOG for quite some time now all of a sudden on server 2016 it's not working. Please provide the necessary config for server 2016 as im_msvistalog is returning errors:

<11>Mar 23 10:33:28 onsitephysio-welsh MSWinEventLog 4 N/A N/A Fri Mar 23 10:33:28 2018 N/A nxlog-ce N/A N/A N/A onsitephysio-welsh N/A EvtNext failed with error 13: The data is invalid. N/A <11>Mar 23 10:33:28 onsitephysio-welsh MSWinEventLog 4 N/A N/A Fri Mar 23 10:33:28 2018 N/A nxlog-ce N/A N/A N/A onsitephysio-welsh N/A ### ASSERTION FAILED at line 319 in im_msvistalog.c/im_msvistalog_event_to_logdata(): "event != NULL" ### N/A

Hello,

I'm new to nxlog and tries to integrate it with GrayLog. It's working fine but to filter data from certain Windows Event Logs into a separate stream in GrayLog I need to insert the Channel value (available in im_msvistalog) into the message text. How can this be done?

Mats-Ove

Does anyone know how to do bit-wise arithmetic in NxLog? I am attempt to decode New and Old UAC Values from the Windows message id 4720. For example, the Old UAC Value is 0x1 and the New UAC Value is 0x21.

I have nxlog configured to capture syslog messages and write them to a file and all is fine but now I would like the ability to parse for a specific string in the syslog message being received and then write those syslog messages to a separate file (nxdomain.log). For example, when receiving syslog messages that contain the string "NXDOMAIN", I want to then write that specific syslog message to a separate file just for those cases so I can track those specific messages and not have to later parse the daily log (log.txt) file.

How would I modify this configuration file to do just that?

define ROOT /usr/bin define FILENAME /logs/log.txt

Moduledir /usr/libexec/nxlog/modules CacheDir %ROOT%/data Pidfile %ROOT%/data/nxlog.pid SpoolDir /var/spool/nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO

<Extension xm_exec> Module xm_exec </Extension>

<Extension xm_fileop> Module xm_fileop </Extension>

<Extension syslog> Module xm_syslog </Extension>

<Input in> Module im_tcp Host 0.0.0.0 Port 514 Exec parse_syslog_bsd() ; </Input>

<Output out> Module om_file File "%FILENAME%" <Schedule> When @daily Exec file_rename ("%FILENAME%","%FILENAME%"+'.'+strftime(now(),"%Y%m%d"));
out->reopen(); </Schedule> </Output>

<Route 1> Path in => out

I want to use source_code to product a msi package for windows. I run ./pkgmsi.sh , It's error ./pkgmsi.sh: line 1: c:\Program Files (x86)\Windows Installer XML v3.5\bin\candle.exe: command not found How can I package msi correctly??? if you help me, I will be grateful for you.

Hello

I'm using "NXLog CE" for send a file by syslog. So far I got this:

<Input filezilla>
	Module		im_file
	File		"C:\\Program Files (x86)\\FileZilla Server\\Logs\\fzs-*.log"
	SavePos 	TRUE
	Exec		$Message = '%FileZilla: ' + $raw_event;
</Input>
<Output outfiles>
    Module      om_tcp
    Host        10.226.6.210
    Port        514
    Exec		to_syslog_bsd();
</Output>

The file has lines like "(125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT" I want to get (on my syslog sever) "%FileZilla: (125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT" Instead, I got: "Mar 13 12:33:00 DSCRESJ %FileZilla: (125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT"

I think the "Mar 13 12:33:00 DSCRESJ" part is the header from to_syslog_bsd(). Anyway, I wonder If there a way to remove that header.

Regards.

I am trying to evaluate om_kafka module on RHEL system.

It starts ok and sends some messages from files to kafka but after a while these messages appear:

2018-03-13 14:52:52 ERROR Unable to produce message 2018-03-13 14:52:55 ERROR last message repeated 183044 times

Sometimes it can be fixed with NXLog service restart, while others not.

nxlog-3.99.3332_RHEL7_x86_64_trial Any suggestion?

Hello there,

I'm using NXLOG Community Edition, and I want to inquire about how can I just forward the events log from Windows OS without modified the original log's contents. Because, the NXLOG extracted the fields from any message. So, is there a way to avoid the extraction?

Thanks and Regard.

Hello all,

I'm trying out NXLog to do some basic log file rotation. I'm just looking for it to rotate a specifed log file when it gets past 100K. For a PoC I've set up my nxlog.conf as follows, but no logs are ever rotated nor do any of the log_info calls ever get into the nxlog.log. Can someone help me with what I'm trying to accomplish? Thanks.

define TESTLOG 'C:\Logs\testlog.log' <Extension fileop> Module xm_fileop </Extension>

<Input logrotatein> Module im_file File "%TESTLOG%" </Input>

<Output logrotateout> Module om_file File %TESTLOG%

<Schedule>
    Every 30 seconds
	log_info('I am doing something');
    Exec if (file_size('%TESTLOG%') >= 100K) 
	{
		log_info('I am rotating');
		file_cycle('%TESTLOG%',500);
		logrotate->reopen();
	}
</Schedule>

</Output>

<Route 1> Path logrotatein => logrotateout </Route>

Attached log:

tail -f nxlog
INFO input file '/kafka/logs/s3/s3sinkfirewall.log' was truncated,
restarting from the beginning
INFO input file '/kafka/logs/s3/s3sinkfirewall.log' was truncated,
restarting from the beginning
INFO input file '/kafka/logs/s3/s3sinkfirewall.log' was truncated,
restarting from the beginning
Attached my config file:

## This is a sample configuration file. See the nxlog reference manual about
the
## configuration options. It should be installed locally under
## /usr/share/doc/nxlog-ce/ and is also available online at
## http://nxlog.org/docs

########################################
# Global directives #
########################################
User nxlog
Group nxlog

LogFile /var/log/nxlog/nxlog.log
LogLevel INFO

# Machine Specific Variables
define CurrentHost kafkavm5
define HeadNxPort 8084
# Common Functions/Variables
include
/kafka/connectors/consumers/MonitorScripts/nxLogConf/nxlogCommon.conf

<Route S3consumer>
Path s3consumerlogs => nxlog_dailys3consumerlogs
</Route>
Also adding nxLogCommon.conf:

<Input s3consumerlogs>
Module im_file
File "%Consumers3Logs%/s3sinkfirewall.log"
SavePos TRUE
ReadFromLast TRUE
<Schedule>
# Check processes every 5 min
Every 30 sec
Exec exec_async("%ScriptPath%/CheckProcesses.py","%Email%","%ConsumerSplunkLogs%");
</Schedule>
<Schedule>
# Remove logs daily
When @daily
<Exec>
file_remove("%DailyLogPrefix%*");
</Exec>
</Schedule>
<Exec>

#$Message = $raw_event;

$Message = substr($raw_event, 0, 1000);
$SourceFile = file_name();
$SourceHost = "%CurrentHost%";
to_json();
</Exec>
I can add if anything else is required.

Hello,

I am working on shipping snort logs to an Elastic stack environment. I have found this https://nxlog.co/documentation/nxlog-user-guide.pdf but unfortunately it doesn't seem to help me. It all looks promising on the pdf, but the output does not look like what it shows at the end there. Instead, all of the logs are processed and shipped one line at a time, and as mentioned in that pdf, that is not helpful in this case.

I am using the linux version of the Community Edition and here are sample files:

/var/log/snort/alert:

[] [1:1000001:1] I saw mommy kissing Santa Clause [] [Classification: Generic ICMP event] [Priority: 3] 03/09-15:47:56.187476 src -> dest ICMP TTL:124 TOS:0x0 ID:16888 IpLen:20 DgmLen:60 Type:8 Code:0 ID:1 Seq:60 ECHO

[] [1:1000001:1] I saw mommy kissing Santa Clause [] [Classification: Generic ICMP event] [Priority: 3] 03/09-15:47:56.187583 src -> dst ICMP TTL:64 TOS:0x0 ID:62815 IpLen:20 DgmLen:60 Type:0 Code:0 ID:1 Seq:60 ECHO REPLY

/etc/nxlog.conf:

<Extension snort> Module xm_multiline HeaderLine /^[**] [\S+] (.*) [**]/ Exec if $raw_event =~ /^\s+$/ drop(); </Extension>

<Extension _json> Module xm_json </Extension>

<Input in> Module im_file File "/var/log/snort/alert" InputType snort SavePos FALSE ReadFromLast FALSE <Exec> if $raw_event =~ /(?x)^[**]\ [\S+]\ (.*)\ [**]\s+ (?:[Classification:\ ([^]]+)]\ )? [Priority:\ (\d+)]\s+ (\d\d).(\d\d)-(\d\d:\d\d:\d\d.\d+) \ (\d+.\d+.\d+.\d+):?(\d+)?\ -> \ (\d+.\d+.\d+.\d+):?(\d+)?\s+\ /

    {
            $EventName = $1;
            $Classification = $2;
            $Priority = $3;
            $EventTime = parsedate(year(now()) + "-" + $4 + "-" + $5 + " " + $6);
            $SourceIPAddress = $7;
            $SourcePort = $8;
            $DestinationIPAddress = $9;
            $DestinationPort = $10;
    }

</Exec>

</Input>

<Output out> Module om_file File "/root/nxlog/snort"

Exec to_json();

This had to be commented out or all the log entries looked like this...{"EventReceivedTime":"2018-03-09 16:14:21","SourceModuleName":"in","SourceModuleType":"im_file"}

</Output>

<Route> Path in => out </Route>

but the output in /root/nxlog/snort looks just like it did when it went in. there is no separation of any of the data and everything is on the sames lines as it went in. Nothing is in quotes like in the example.

Any help would be great. Thanks!

i configured my nxlog that forwards sharepoint uls log to log server. my nxlog.conf is like below ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define SHAREPOINT_LOGS D:\SHAREPOINTLOGS Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_syslog Module xm_csv Fields Timestamp, Process, TID, Area, Category, EventID, Level, Message, \ Correlation Delimiter \t Module im_file # Use a file mask to read from ULS trace log files only File '%SHAREPOINT_LOGS%\*-????????-????.log' # Drop header lines and empty lines if $raw_event =~ /^(\xEF\xBB\xBF|Timestamp)/ drop(); else { # Remove extra spaces $raw_event =~ s/ +(?=\t)//g; # Parse with uls_parser instance defined above uls_parser->parse_csv(); # Set $EventTime field (second precision only) $EventTime = strptime($Timestamp, "%m/%d/%Y %H:%M:%S"); # Add $Hostname field $Hostname = hostname_fqdn(); } # # Module im_msvistalog # For windows 2003 and earlier use the following: # Module im_mseventlog # Module om_udp Host 192.168.99.52 Port 514 Exec to_syslog_bsd(); Module om_udp Host 192.168.99.52 Port 514 Exec to_syslog_bsd(); Path trace_file => out nxlog sends message to log server to the one moment when he stops with the work.why?

on my mssql server i installed nxlog enterprise version and i want that audit sharepoint content database i configured nxlog.conf with next code Module im_odbc ConnectionString Driver={ODBC Driver 13 for SQL Server}; \ Server=dri-spsqlph; Database=WSS_Content_lok; \ Trusted_Connection=yes SQL SELECT Occurred as id, * FROM dbo.AuditData WHERE Occurred > ? IdIsTimestamp TRUE # Fix time zone and set $EventTime $EventTime = parsedate(strftime($id, '%Y-%m-%d %H:%M:%SZ')); delete($id); Module om_udp Host 192.168.99.52 Port 514 Exec to_syslog_bsd(); Path audit_db => out but when i restart nxlog service in nxlog i receive error **ERROR apr_sockaddr_info failed for ENTER_MANAGEMENT_ADDRESS_HERE:4041; No such host is known.** why?

<Input Result_Log> Module im_file File "C:\xxx\Result*.log" SavePos TRUE </Input>

above is my nxlog input config.
Result-3156.log is my log file name and 3156 is pid. The log file will rotate when it reaches the max size and generate Result-3156.log.1 file. I find some logs loss at the bottom of Result-123.log.1 And nxlog's log shows "2018-03-01 20:48:02 INFO inode changed for 'C:\xxx\Result-3156.log' (25746->25799): reopening possibly rotated file" at the same time point.

I suspect this is caused by the monitor interval 1 second.

Is it a bug ? or nxlog can not guarantee this scenario ? or something can do to avoid it ?

i have a log with format time, date, description. how i can exclude time and date from $raw_event. i just need field description?

Hi, I am using nxlog first time and I need to send out application logs from windows OS. But there is new log every day with date time .... . Will be enought use this: Module im_file File "D:\Logs\AppLogs\*.log" SavePos TRUE ReadFromLast TRUE PollInterval 1 Exec $Message = $raw_event; $SyslogFacilityValue = 22; thank you

hi, i newbie on nxlog. i need help how convert common log format (apache) to syslog and send to log server (KiwiSyslog server) windows

Hello,

I can send the event log from Server 2012 with the same configuration, but it is not running on Server 2016. The event log does not go to Graylog. Does NXLog not work on Server 2016? If so, what is the appropriate nxlog.conf?

Thanks.

So...

I have about 3 days of experience with NXLog and what I am attempting to do is pull Cisco Umbrella Logs via an s3fs mount, unzip them and then read the logs using nxlog.

I am still working out the specifics of how to get all the logs unzipped and into one working file but my test file is failing to be read and sent to the syslog server.

I have taken a new nxlog.conf and set it up with the following:


########################################
# Global directives #
########################################
User nxlog
Group nxlog

LogFile /var/log/nxlog/nxlog.log
LogLevel INFO

########################################
# Modules #
########################################
<Extension csv>
Module xm_csv
</Extension>

<Input in>
Module im_file
File "/tmp/incoming.csv"
</Input>

<Output out>
Module om_udp
Host 10.x.x.x
Port 10514
Exec to_syslog_bsd();
</Output>

########################################
# Routes #
########################################
<Route 1>
Path in => out
</Route>


My nxlog.log file shows no issues with the connection or service but I am not getting any data from my test file.

Has anyone been able to do this or can you provide guidance on why I am not getting any data to transfer using my current setup?