Hi I'm using nxlog v2.9.1716.

I've created the following input:

<Input in> Module im_file File "C:\Program Files\LogFiles\*.log" SavePos TRUE Recursive TRUE

Exec if $raw_event =~ /^#/ drop();
Exec if $raw_event =~ ^([^;]+);([^;]+);([^;]+)(?>;([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);([^;]+);(.+)$)?/gx; \
	{ \
		$date = $1; \
		$time = $2; \
		$site-instance = $3; \
		$event = $4; \
		$client-ip = $5; \
		$via-header = $6; \
		$http-x-forwarded-for = $7; \
		$host-header = $8; \
		$additional-info-1 = $9; \
		$additional-info-2 = $10; \
		$additional-info-3 = $11; \
		$additional-info-4 = $12; \
		$additional-info = $13; \
		$EventTime = parsedate($date + " " + $time); \
		$SourceName = "WAF"; \
	}								

</Input>

The regex being used has been successfully tested with https://regex101.com/

Sample data below:

2018-06-28 ; 10:23:52 ; W3SVC2 ; OnPreprocHeaders ; 10.10.10.10 ; ; 8.8.8.8 ; my.domain.com ; GET ; /account/login ; ALERT: '/account/' not allowed in URL ; HTTP/1.0 ; 0 ; ; Actional Intermediary

When I start the nxlog service, I get the following error:

2018-06-28 16:44:51 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:89; couldn't parse statement at line 89, character 24 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; syntax error 2018-06-28 16:44:51 ERROR module 'in' has configuration errors, not adding to route '2' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:116 2018-06-28 16:44:51 ERROR route 2 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:116 2018-06-28 16:44:51 WARNING not starting unused module in 2018-06-28 16:44:51 INFO nxlog-ce-2.9.1716 started 2018-06-28 16:44:51 INFO reconnecting in 1 seconds

I also tried the following:

<Input in> Module im_file File "C:\Program Files\AQTRONIX Webknight\LogFiles\*.log" SavePos TRUE Recursive TRUE <Exec> if $Message =~ /^#/ drop(); $Message =~ ^(?<date>[^;]+);(?<time>[^;]+);(?<site_instance>[^;]+)(?>;(?<event>[^;]+);(?<client_ip>[^;]+);(?<via_header>[^;]+);(?<http_x_forwarded_for>[^;]+);(?<host_header>[^;]+);(?<additional_info_1>[^;]+);(?<additional_info_2>[^;]+);(?<additional_info_3>[^;]+);(?<additional_info_4>[^;]+);(?<additional_info>.+)$)? /gx; </Exec> </Input>

But I receive the following error on starting nxlog:

2018-06-28 17:15:54 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:70; couldn't parse statement at line 72, character 15 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; syntax error 2018-06-28 17:15:54 ERROR module 'in' has configuration errors, not adding to route '2' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:100 2018-06-28 17:15:54 ERROR route 2 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:100 2018-06-28 17:15:54 WARNING not starting unused module in 2018-06-28 17:15:54 INFO nxlog-ce-2.9.1716 started

I tried various syntax changes, but just cannot see the issue.

This is the first time I've tried using a regex with nxlog.

Any help or guidance much appreciated.

I have been trying to get NXLog to send Syslog entries from the Teamviewer "Connections_incoming.txt" log file. It is a tab delimited file. I found this site which supplied the basic code for the task: https://gist.github.com/idefux/949e84c8ec8d4db1775c which i couldn't get working as expected. To cut a long story short, i have discovered that the $raw_event is often blank, so the Syslog entries do not contain the necessary information. I have tried to google this issue, but i have not been able to find the information to figure this one out. There has been times when $raw_event did contain the information required from the original log file, but it is not reliable. Can someone please give me some suggestions on how to get information into the $raw_event function?

On the positive side, i do get a reliable Syslog entry whenever someone accesses the computer through Teamviewer. Using the code below, i get the message "Teamviewer Login Event"

Below is my current configuration. The if-else statement is designed to be as simple as possible to try fault-find the issue

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension _syslog> Module xm_syslog </Extension>

<Input in_teamviewer> Module im_file File 'c:\Program Files (x86)\TeamViewer\Connections_incoming.txt' SavePos TRUE ReadFromLast TRUE PollInterval 10 <Exec> $Hostname = hostname(); $SeverityValue = 5; $MessageSourceAddress = hostname(); $SyslogFacilityValue = 4; $SourceName = 'TeamViewer'; $EventTime = parsedate($5 + '-' + $4 + '-' + $3 + ' ' + $6); $user = $2; if ($raw_event == '') $Message = $raw_event + "Teamviewer Login Event"; else $Message = $raw_event + '['+ file_name() + ']'; to_syslog_bsd(); </Exec> </Input>

<Output out_syslog> Module om_udp Host localhost Port 514 </Output>

<Route 1> Path in_teamviewer => out_syslog </Route>

Hi, I'm using nxlog ce latest version with centos 7 with latest patches to read files from a directory that is a cifs mount to a windows server. Logs are in w3c format. However from times to times, I'm getting the information in nxlog log file "INFO input file '' was truncated, restarting from the beginning", but the file isn't truncated. Using strace with options -tfp (since debug option of nxlog doesn't give me much information) i get this: pid 7675] 12:52:58 close(16 [pid 7676] 12:52:58 {st_mode=S_IFREG|0400, st_size=144080774, ...}) = 0 [pid 7676] 12:52:58 stat("", [pid 7675] 12:52:58 ) = 0 [pid 7676] 12:52:58 {st_mode=S_IFREG|0400, st_size=54218426, ...}) = 0 [pid 7675] 12:52:58 stat("", [pid 7676] 12:52:58 stat("", [pid 7675] 12:52:58 {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7675] 12:52:58 fstat(26, {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7676] 12:52:58 {st_mode=S_IFREG|0400, st_size=141529135, ...}) = 0 [pid 7676] 12:52:58 stat("", {st_mode=S_IFREG|0400, st_size=148346198, ...}) = 0 My nxlog configuration is this: User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO define DIR Module xm_syslog Module im_file File '%DIR%/_*_.w3c' SavePos TRUE ReadFromLast TRUE Recursive TRUE RenameCheck TRUE CloseWhenIdle TRUE PollInterval 5 DirCheckInterval 10 ActiveFiles 4 to_syslog_bsd(); Module om_tcp Host Port 514 Path in1 => out_syslog-ng Can you understand why nxlog says that file is truncated when it's not?

Unable to consume AWS s3 log file using NXlog

Hi,

I want to send syslog from Windows Server 2012 R2 (using NxLog) from my SIEM (FORTISIEM)

I will install nxlog client with many computers ，but it means I will configure the same nxlog.conf one by one , it seems too trouble，please give me some good advise, thinks!

Good afternoon. I am trying to see if NXlog can solve a use case I have with IIS.

Input file file contains:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken 2017-11-22 11:00:17 192.168.30.60 HEAD /Autodiscover - 443 - 192.168.30.58 HttpProxy.ClientAccessServer2010Ping 401 2 5 325 166 0

What I was hoping I could do with NXlog was to read the log file then send it in syslog in key value pairs using the fields I defined (which would match what IIS gives me

I need Nxlog to send in syslog ->

<12> date="2017-11-22",time="11:00:17",s-ip="192.168.30.60",cs-method="HEAD".....

Thanks

Dave

Is there a way to put a wildcard in a regular expression? Here is an example of my Headerline that I need to set:

Jun 11 16:10:05 tst-tst01-rp adevents: ---Begin event transaction---

So I have this for my headerline, but it doesn't seem to work:

Headerline /^\w\w\w \d\d \d\d:\d\d:\d\d \w\w\w-\w\w\w\d\d-\w\w adevents: ---Begin event transaction---/

I was hoping to be able to use a wildcard like this, but it doesn't work:

Headerline /^*---Begin event transaction---/

Thanks for your time.

Hi,

New to nxlog i don't even know where to search for that kind of needs...

SFB is a Microsoft UC client with several hundred Mo logs per day. I need to extract only a few information between strings for each PC. How can i achieve that ??

For Exp i need to extract this spécific info between those delimiters : <v3:Density>0.004195841</v3:Density>

Thanks for any help.

I am having trouble getting multiline to work with ClamSentinel DriveAdd logs.

below is the raw log as ClamSentinel outputs the information.

#####################################################################################################################

##### Thursday, June 07, 2018 2:05:09 PM (jereme.powers@POS-LOGFIL1)

-------------------------------------------------------------------------------


----------- SCAN SUMMARY -----------
Known viruses: 6539461
Engine version: 0.99.4
Scanned directories: 9
Scanned files: 13
Infected files: 0
Data scanned: 21.52 MB
Data read: 169.16 MB (ratio 0.13:1)
Time: 17.069 sec (0 m 17 s)

##### Thursday, June 07, 2018 2:10:31 PM (jereme.powers@POS-LOGFIL1)

-------------------------------------------------------------------------------


----------- SCAN SUMMARY -----------
Known viruses: 6539461
Engine version: 0.99.4
Scanned directories: 9
Scanned files: 13
Infected files: 0
Data scanned: 21.52 MB
Data read: 169.16 MB (ratio 0.13:1)
Time: 16.608 sec (0 m 16 s)

##### Thursday, June 07, 2018 2:12:13 PM (jereme.powers@POS-LOGFIL1)

-------------------------------------------------------------------------------


----------- SCAN SUMMARY -----------
Known viruses: 6539461
Engine version: 0.99.4
Scanned directories: 9
Scanned files: 13
Infected files: 0
Data scanned: 21.52 MB
Data read: 169.16 MB (ratio 0.13:1)
Time: 16.234 sec (0 m 16 s)

#####################################################################################################################

I have tried many different regex combinations for the headerline but so far I have not been able to parse everything together into one line.

#####################################################################################################################

Here is my nxlog config...


define ROOT C:\Program Files (x86)\nxlog
define LOGFILE %ROOT%\data\nxlog.log

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

##################################################################################

##################
# Extensions #
##################

<Extension json>
Module xm_json
</Extension>

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension charconv>
Module xm_charconv
</Extension>

<Extension Eventlog_CSV>
Module xm_csv
Fields $EventTime, $EventType, $Severity, $Channel, $Hostname, $EventID, $SourceName, $AccountName, $AccountType, $Domain, $Message, $Task, $Category, $Keywords, $UserID, $SeverityValue, $ProviderGuid, $Version, $OpcodeValue, $Opcode, $ActivityID, $RelatedActivityID, $ProcessID, $ThreadID, $RecordNumber
FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string
Delimiter ;
</Extension>

<Extension multiline_CLAMWIN-ScanLog>
Module xm_multiline
Exec if $raw_event =~ /^\s*$/ drop();\
else\
{\
$raw_event = replace($raw_event, "\r", " ");\
}
HeaderLine /^\w{4}\s\w{7}\s\w{0,5}\s\w{0,4}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4}/
</Extension>

<Extension multiline_CLAMWIN-UpdateLog>
Module xm_multiline
Exec if $raw_event =~ /^\s*$/ drop();\
else\
{\
$raw_event = replace($raw_event, "\r", " ");\
}
HeaderLine /^--------------------------------------/
</Extension>

<Extension multiline_CLAMSentinel-DriveAddLog>
Module xm_multiline
Exec if $raw_event =~ /^\s*$/ drop();\
else\
{\
$raw_event = replace($raw_event, "\r", " ");\
}
HeaderLine /^[#]{5}\s\w{6,9},\s\w{3,9}\s\d{2},\s\d{4}\s\d+:\d+:\d+\s\w{2}\s[(].*[)]$/
</Extension>

##################################################################################

#####################
# WINDOWS Events #
#####################

<Processor Filter_XML>
Module pm_pattern
PatternFile %ROOT%\conf\patterndb.xml
</Processor>

<Input IN_Eventlog>
Module im_msvistalog
Query <QueryList> \
<Query Id="0"> \
<Select Path="Security">*</Select> \
<Select Path="Application">*</Select> \
<Select Path="System">*</Select> \
<Select Path="Setup">*</Select> \
</Query> \
</QueryList>

#########################################
# Windows Events - Filter Application #
#########################################

#Exec if ($Application =~ /appdata\\roaming\\dropbox\\bin\\dropbox.exe/) drop();
#Exec if ($Application =~ /nxlog\\nxlog.exe/) drop();
#Exec if ($Application =~ /windows\\system32\\spoolsv.exe/) drop();

#########################################
# Filter by Source and Destination IP #
#########################################

#Exec if ($SourceAddress =~ /8.8.8.8/) drop();
#Exec if ($DestAddress =~ /8.8.8.8/) drop();

#########################
# Filter by EventID #
#########################

Exec if ($EventID == 4656 or $EventID == 4663) drop();
</Input>

<Output OUT_Eventlog>
Module om_file
File "C:\\ProgramData\\.clamwin\\log\\WINEVENTLOGS.log"
Exec if not defined $PatternID or not defined $Message { drop(); }
Exec $Message = replace($Message, "\t", " "); $Message = replace($Message, "\n", " "); $Message = replace($Message, "\r", " ");
Exec if not defined $AccountName { $AccountName = "-"; }
Exec if not defined $AccountType { $AccountType = "-"; }
Exec if not defined $Domain { $Domain = "-"; }
Exec Eventlog_CSV->to_csv(); $raw_event = $Hostname + ' WIN-NXLOG ' + $raw_event + ' Task: ' + $Task + ' Category: '+ $Category + ' Keywords: ' + $Keywords + ' UserID: ' + $UserID + ' Severity: ' + $SeverityValue + ' ProviderGuid: ' + $ProviderGuid + ' Version: ' + $Version + ' OpcodeValue: ' + $OpcodeValue + ' Opcode: ' + $Opcode + ' ActivityID: ' + $ActivityID + ' RelatedActivityID: ' + $RelatedActivityID + ' ProcessID: ' + $ProcessID + ' ThreadID: ' + $ThreadID + ' RecordNumber: ' + $RecordNumber;
</Output>

<Route route_winevents_logs>
Path IN_Eventlog => Filter_XML => OUT_Eventlog
</Route>

##################################################################################

###############
# CLAMWIN #
###############

##############
# ScanLog #
##############

<Input IN_CLAMWIN-ScanLog>
Module im_file
File "C:\\ProgramData\\.clamwin\\log\\ClamScanLog.txt"
InputType multiline_CLAMWIN-ScanLog
SavePos FALSE
</Input>

<Output OUT_CLAMWIN-ScanLog>
Module om_file
File "C:\\ProgramData\\.clamwin\\log\\CLAMWIN-ScanLog.log"
Exec $Hostname = hostname_fqdn();
Exec $raw_event = $Hostname + " " + $raw_event + " " + 'CLAM-NXLOG';
</Output>

<Route route_CLAMWIN-ScanLog>
Path IN_CLAMWIN-ScanLog => OUT_CLAMWIN-ScanLog
</Route>

#################
# UpdateLog #
#################

<Input IN_CLAMWIN-UpdateLog>
Module im_file
File "C:\\ProgramData\\.clamwin\\log\\ClamUpdateLog.txt"
InputType multiline_CLAMWIN-UpdateLog
SavePos FALSE
</Input>

<Output OUT_CLAMWIN-UpdateLog>
Module om_file
File "C:\\ProgramData\\.clamwin\\log\\CLAMWIN-UpdateLog.log"
Exec $Hostname = hostname_fqdn();
Exec $raw_event = $Hostname + " " + $raw_event + " " + 'CLAM-NXLOG';
</Output>

<Route route_CLAMWIN-UpdateLog>
Path IN_CLAMWIN-UpdateLog => OUT_CLAMWIN-UpdateLog
</Route>

##################################################################################

#####################
# CLAMSentinel #
#####################

###################
# DriveAddLog #
###################

<Input IN_CLAMSentinel-DriveAddLog>
Module im_file
File "C:\\ProgramData\\.clamwin\\log\\ClamSentinel_DriveAddLog.txt"
InputType multiline_CLAMSentinel-DriveAddLog
SavePos FALSE
</Input>

<Output OUT_CLAMSentinel-DriveAddLog>
Module om_file
File "C:\\ProgramData\\.clamwin\\log\\CLAMSentinel-DriveAddLog.log"
Exec $Hostname = hostname_fqdn();
Exec $raw_event = $Hostname + " " + $raw_event + " " + 'CLAM-NXLOG';
</Output>

<Route route_CLAMSentinel-DriveAddLog>
Path IN_CLAMSentinel-DriveAddLog => OUT_CLAMSentinel-DriveAddLog
</Route>

#####################################################################################################################

Everything is working but the DriveAddLog section. Please help!

We recently enabled logging on CIFS share hosted on our Netapp. The audit logs that are generated are stored on a network share currently in EVTX format (XML logs are also an option). I have a windows server that has NXLog installed and can mount the network share where the EVTX files are located. What is the best module to use get these EVTX or XML files into our Graylog server on a regular basis?

We currently have an issue where we pass log4net data into logstash it stops writing events after a period of time. We deployed Nxlogs to pull IIS logs into logstash where logstash stopped processing new events after 20 minutes using the Udp appender.

Unfortunately using tcp isn't an option due to the protential risk of it slowing our production environment down. Has anyone got any suggestions on how we can use NXlogs and stop logstash from stopping (we use the community edition) or is the only way to parse it into a messaging queue like Redis first then logstash?

Trying to upgrade nxlog which requires the un-install of the old version. The program is not listed under windows programs and I cannot locate an unistall.exe. How do you uninstall?

Hi I need to replace any accented vowel (á,é,í,ó,ú) on raw event. I tried (for just 'a' vowel):

$raw_event = replace($raw_event, "á", 'a');

But it doesn't work

I am trying out nxlog community edition in centos i am getting an error

Error : om_udp apr_socket_send failed;Connection refused

What could be the solution

i'm unable to connect centralized log server from client machine

nxlog.config:

User nxlog
Group nxlog

LogFile /var/log/nxlog/nxlog.log
LogLevel INFO


########################################
# Modules #
########################################


<Input in>
Module im_file
File "/var/log/messages"

</Input>

<Output out>
Module om_udp
Host 192.168.58.175
Port 514

########################################
# Routes #
########################################
<Route>
Path in => out
</Route>

Hello there,

I am working with a multiline module. This particular file has 38 lines, but I'd like to only ship the lines that contain a colon. Is there a way to write an exec that if the line does not contain a colon then drop the line?

Here is my config so far:

<Extension log>

Module	xm_multiline
#FixedLineCount
HeaderLine /^---Begin event transaction---/
#EndLine /^---Event Reporting Complete---/

</Extension>

<Extension json>

Module xm_json

</Extension>

<Input in>

Module im_file

	File   "C:\\Users\\Administrator\\Desktop\\SRR_Error.txt"
	InputType		log
	SavePos			FALSE
	ReadFromLast	FALSE
	Exec			$message = $raw_event; to_json();

</Input>

Thanks,

Currently we are sending our Exchange 2013 logs to our Graylog server using the CSV converter built into Graylog. We parse based on the following fields in 2013.

Fields $date-time,$client-ip,$client-hostname,$server-ip,$server-hostname,$source-context,$connector-id,$source,$event-id,$internal-message-id,$message-id,$network-message-id,$recipient-address,$recipient-status,$total-bytes,$recipient-count,$related-recipient-address,$reference,$message-subject,$sender-address,$return-path,$message-info,$directionality,$tenant-id,$original-client-ip,$original-server-ip,$custom-data

FieldTypes Datetime,ip4addr,String,ip4addr,String,String,String,String,String,String,String,String,String,String,Integer,String,String,String,String,String,String,String,String,String,ip4addr,ip4addr,String`

When I use these strings to parse Exchange 2016 logs I am unable to breakup the data within the CSV being sent. It appears that some fields changed but I don't know what the fields should be labeled or what type of field they are.

Does anyone have a working Exchange 2016 Message Tracking exporter for NXLog-CE?

When using NXlogs does any data get collected by Nxlogs.co and what data is passed?

Hi,

We have a compliance requirement to keep our software up-to-date and wonder if it'd be possible to subscribe to something where you notify us of updates and include your change log from the installation files \usr\share\doc\nxlog-ce\changelog.txt

Thanks James