Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Question: How to filter-out syslog messages that are not received by a specific NIC?
giorgiobuffa created
Hello!
I'd like to log syslog messages received via broadcast on UPD:514. In the logger PC there are multiple NICs, but I'm interested only to syslog messages received from these local interfaces: 10.200.255.254 and 127.0.0.1. I've tried with the configuration posted below, but it doesn't always work (especially at system boot) because of the following error by the NXLog Windows Service.
ERROR failed to start im_udp; couldn't bind udp socket to 10.200.255.254:514; The requested address is not valid in its context.
This is my actual configuration
...
define LOGS_FILE ...
...
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Input udp_lan>
Module im_udp
Host 10.200.255.254
Port 514
Exec parse_syslog();
</Input>
<Input udp_host>
Module im_udp
Host 127.0.0.1
Port 514
Exec parse_syslog();
</Input>
<Output file_syslog>
Module om_file
CreateDir TRUE
Sync TRUE
File '%LOGS_FILE%'
</Output>
<Route udp_to_file>
Path udp_lan, udp_host => file_syslog
</Route>
The NIC has a static IP address, but I'm not sure it is already up when the nxlog service is executed at system boot by Windows.
How can I perform this sort of filtering without using the firewall? I know I can listen to anything and drop() received messages with an Exec statement, but I don't know which variable to use (e.g. like $MessageSourceAddress) to specify the interface the message was received by. I was thinking to something like that:
<Input udp_lan_host>
Module im_udp
Host 0.0.0.0 # <--- accept all
Port 514
<Exec>
if $??? != "10.200.255.254" and $??? != "127.0.0.1" drop(); # <--- which variable here?
parse_syslog();
</Exec>
</Input>
Thanks,
Giorgio
giorgiobuffa created
vCenter Remote Logging Issue
Tenways created
Setup according to documentation here:
vCenter CLI installed
The issue I'm having isn't so much with NXLog, but rather the .pl script in the documentation. When attempting to run the script:
c:\ perl vcenter.pl -s=x.x.x.x -u=ServiceAccount -p=password
I receive the following:
`{"EventTime":"2021-04-15T19:40:39","Message":"Server version unavailable at 'https://x.x.x.x:443/sdk/vimService.wsdl' at C:/Program Files (x86)/VMware/VMware vSphere CLI/Perl/lib/VMware/VICommon.pm line 551.
","UserName":"ServiceAccount"}
{"EventTime":"2021-04-15T19:40:39","Message":"Vim::get_service_content called, but no global session is defined at vcenter.pl line 87
","UserName":"ServiceAccount"}`
I can confirm these credentials were working before, as I used to have an Arcsight Smart Connector pulling these logs. Any insight that can be provided would be greatly appreciated. Thanks in advance!
Tenways created
Log Forwarding to Azure Sentinel
test4711 created
Hi
I'm trying to get nxlog ee running with Microsoft Sentinel still get the error
2021-04-14 19:02:04 INFO [om_http|AzureHTTP] connecting to xxxxxxxx.ods.opinsights.azure.com(xx.xx.xx.xx):443
2021-04-14 19:02:04 ERROR [om_http|AzureHTTP] SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\test.crt', reason: no certificate or crl found
2021-04-14 19:02:04 INFO [om_http|AzureHTTP] reconnecting in 4 sec
Forwarding to Sentinel Logspace is not working.
What's wrong ? Johannes
Panic Soft
define INSTALLDIR C:\Program Files\nxlog
#ModuleDir %INSTALLDIR%\modules
#CacheDir %INSTALLDIR%\data
#SpoolDir %INSTALLDIR%\data
define CERTDIR %INSTALLDIR%\cert
define CONFDIR %INSTALLDIR%\conf\nxlog.d
define WORKSPACE xxxxxxxxxxxxxxxxxxxx
define SHAREDKEY xxxxxxxxxxxxxxxxxxxx
define SUBDOMAIN ods.opinsights.azure.com
define RESOURCE api/logs
define APIVER api-version=2016-04-01
define SIZELIMIT 65000
Note that these two lines define constants only; the log file location
is ultimately set by the LogFile directive (see below). The
MYLOGFILE define is also used to rotate the log file automatically
(see the _fileop block).
define LOGDIR %INSTALLDIR%\data
define MYLOGFILE %LOGDIR%\nxlog.log
If you are not using NXLog Manager, disable the include line
and enable LogLevel and LogFile.
include %CONFDIR%*.conf
#LogLevel INFO
#LogFile %MYLOGFILE%
<Extension _json>
Module xm_json
</Extension>
<Input VAT_Eventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="ForwardedEvents">
*[System[Level=0 and (EventID=4624 or EventID=4647)]]
</Select>
</Query>
</QueryList>
</QueryXML>
RemoteServer xxx
RemoteUser xxx
RemoteDomain hq
RemotePassword xxx
</Input>
<Extension plxm>
Module xm_perl
PerlCode %INSTALLDIR%\modules\extension\perl\sentinelauth.pl
</Extension>
<Output AzureHTTP>
Module om_http
URL https://%WORKSPACE%.%SUBDOMAIN%/%RESOURCE%?%APIVER%
ContentType application/json
HTTPSAllowUntrusted TRUE
HTTPSCAFile %INSTALLDIR%\cert\test.crt
<Exec>
create_stat('ec', 'COUNT');
create_stat('bc', 'COUNT');
create_var('batch');
create_var('nextbatch');
add_stat('ec',1);
#---BEGIN--- the enrichment of this event with any new fields:
$BatchNumber = get_stat('bc');
$EventNumber = get_stat('ec');
to_json();
#---END--- the enrichment of this event
if (size(get_var('batch')) + size($raw_event) + 3) > %SIZELIMIT%
# Flush this batch of events
{
set_var('nextbatch', $raw_event);
$raw_event = '[' + get_var('batch') + ']';
add_stat('bc',1);
set_var('batch',get_var('nextbatch'));
$Workspace = "%WORKSPACE%";
$SharedKey = "%SHAREDKEY%";
$ContentLength = string(size($raw_event));
$dts = strftime(now(),'YYYY-MM-DDThh:mm:ssUTC');
$dts_no_tz = replace($dts,'Z','');
$parsedate_utc_false = parsedate($dts_no_tz,FALSE);
$x_ms_date = strftime($parsedate_utc_false, '%a, %d %b %Y %T GMT');
plxm->call("genauth");
add_http_header('Authorization',$authorization);
add_http_header('Log-Type',$SourceModuleName);
add_http_header('x-ms-date',$x_ms_date);
}
else
{
$delimiter = get_stat('ec') == 1 ? '' : ",\n";
set_var('batch', get_var('batch') + $delimiter + $raw_event);
drop();
}
</Exec>
</Output>
#<Output TempFile>
Module om_file
File 'C:\Program Files\nxlog\data\dnsetw.json'
#</Output>
<Route DnsRoute1>
Path VAT_Eventlog => AzureHTTP
</Route>
test4711 created
NXlog error with "Extended configuration example of security-focused event IDs to monitor" Nxlog configuration for Windows events
sejoneshull created
Hi,
I am trying to test/deploy the "Extended configuration example of security-focused event IDs to monitor" NXlog configuration for Windows events, as per the article/NXlog conf file example here: https://nxlog.co/documentation/nxlog-user-guide/eventlog-eventids.html (Example 543). I am using NXLog CE (latest version) on Windows 2012R2.
Here's the NXlog.conf I have, using the above conf example and added to the usual NXLog conf needs:
# NXLog.conf
define ROOT C:\Program Files (x86)\nxlog
define OUTPUT_DESTINATION_ADDRESS <REDACT IP>
define OUTPUT_DESTINATION_PORT 514
# define Account Usage Events
define AccountUsage 4740, 4648, 4781, 4733, 1518, 4776, 5376, 5377, \
4625, 300, 4634, 4672, 4720, 4722, 4782, 4793, \
4731, 4735, 4766, 4765, 4624, 1511, 4726, 4725, \
4767, 4728, 4732, 4756, 4704
# define Application Crash Events
define AppCrashes 1000, 1002, 1001
# define Application Whitelisting Events
define AppWhitelisting 8023, 8020, 8002, 8003, 8004, 8006, 8007, 4688, \
4689, 8005, 865, 866, 867, 868, 882
# define Boot Events
define BootEvents 13, 12
# define Certificate Services Events
define CertServices 95, 4886, 4890, 4874, 4873, 4870, 4887, 4885, \
4899, 4896, 1006, 1004, 1007, 1003, 1001, 1002
# define Clearing Event Logs Events
define ClearingLogs 1100, 104, 1102
# define DNS and Directory Services Events
define DNSDirectoryServ 5137, 5141, 5136, 5139, 5138, 3008, 3020
# define External Media Detection events
define ExtMedia 400, 410
# define Group Policy Error Events
define GroupPolicyError 112, 1001, 1125, 1126, 1127, 1129
# define Kernel Driver Signing Events
define KernelDriver 3001, 3002, 3003, 3004, 3010, 3023, 5038, \
6281, 219
# define Microsoft Cryptography API Events
define MSFTCryptoAPI 11, 70, 90
# define Mobile Device Activities
define MobileDeviceEvents 10000, 10001
# define Network Host Activities
define NetworkHost 4714, 4713, 4769, 6273, 6275, 6274, 6272, \
6278, 6277, 6279, 6276, 6280, 5140, 5145, \
5142, 5144, 4706, 1024, 4897, 4719, 4716, \
4779, 4778, 5632
# define PassTheHash Detection Events
define PassTheHash 4624, 4625
# define PowerShell Activities
define PowerShell 800, 169, 4103, 4104, 4105, 4106
# define Printing Services Events
define PrintingServices 307
# define Logon Events
define LogonEvents 4624, 4634
# define Software Service Installation Events
define Installation 903, 904, 6, 1022, 1033, 7045, 907, 908, 7000, \
800, 2, 905, 906, 19
# define System Integrity Events
define SystemIntegrity 4657, 1, 4616
# define System or Service Failure Events
define SystemServiceFail 7022, 7023, 7024, 7026, 7031, 7032, 7034
# define Task Scheduler Activities
define TaskScheduler 106, 141, 142, 200
# define Windows Defender Activities
define WinDefender 1008, 1006, 1116, 1010, 2003, 2001, 1009, 1118, \
1119, 1007, 1117, 3002, 2004, 1005, 5008
# define Windows Firewall Events
define WinFirewall 2009, 2004, 2005, 2006, 2033
# define Windows Update Error Events
define WinUpdateError 1009, 20, 24, 25, 31, 34, 35
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input internal>
Module im_internal
</Input>
<Input extendedeventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Application">*</Select>
<Select Path="Security">*</Select>
<Select Path="System">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Assistant">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Troubleshooter">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Inventory">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Program-Telemetry">*</Select>
<Select Path="Microsoft-Windows-Application-Experience/Steps-Recorder">*</Select>
<Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select>
<Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select>
<Select Path="Microsoft-Windows-AppLocker/Packaged app-Deployment">*</Select>
<Select Path="Microsoft-Windows-AppLocker/Packaged app-Execution">*</Select>
<Select Path="Microsoft-Windows-CAPI2/Operational">*</Select>
<Select Path="Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational">*</Select>
<Select Path="Microsoft-Windows-DNS-Client/Operational">*</Select>
<Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select>
<Select Path="Microsoft-Windows-Kernel-PnP/Configuration">*</Select>
<Select Path="Microsoft-Windows-NetworkProfile/Operational">*</Select>
<Select Path="Microsoft-Windows-NTLM/Operational">*</Select>
<Select Path="Microsoft-Windows-PowerShell/Admin">*</Select>
<Select Path="Microsoft-Windows-PowerShell/Operational">*</Select>
<Select Path="Microsoft-Windows-PrintService/Admin">*</Select>
<Select Path="Microsoft-Windows-PrintService/Operational">*</Select>
<Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
<Select Path="Microsoft-Windows-TerminalServices-RDPClient/Operational">*</Select>
<Select Path="Microsoft-Windows-User Profile Service/Operational">*</Select>
<Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select>
<Select Path="Microsoft-Windows-Windows Defender/WHC">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/Firewall">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallDiagnostics">*</Select>
<Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose">*</Select>
<Select Path="Network Isolation Operational">*</Select>
<Select Path="Microsoft-Windows-WindowsUpdateClient/Operational">*</Select>
<Select Path="Windows PowerShell">*</Select>
<Select Path="Microsoft-Windows-CodeIntegrity/Operational">*[System[Provider[@Name='Microsoft-Windows-CodeIntegrity']]]</Select>
<Select Path="Microsoft-Windows-LSA/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if ($EventID NOT IN (%AccountUsage%)) and
($EventID NOT IN (%AppCrashes%)) and
($EventID NOT IN (%AppWhitelisting%)) and
($EventID NOT IN (%BootEvents%)) and
($EventID NOT IN (%CertServices%)) and
($EventID NOT IN (%ClearingLogs%)) and
($EventID NOT IN (%DNSDirectoryServ%)) and
($EventID NOT IN (%ExtMedia%)) and
($EventID NOT IN (%GroupPolicyError%)) and
($EventID NOT IN (%KernelDriver%)) and
($EventID NOT IN (%MSFTCryptoAPI%)) and
($EventID NOT IN (%MobileDeviceEvents%)) and
($EventID NOT IN (%NetworkHost%)) and
($EventID NOT IN (%PassTheHash%)) and
($EventID NOT IN (%PowerShell%)) and
($EventID NOT IN (%PrintingServices%)) and
($EventID NOT IN (%LogonEvents%)) and
($EventID NOT IN (%Installation%)) and
($EventID NOT IN (%SystemIntegrity%)) and
($EventID NOT IN (%SystemServiceFail%)) and
($EventID NOT IN (%TaskScheduler%)) and
($EventID NOT IN (%WinDefender%)) and
($EventID NOT IN (%WinFirewall%)) and
($EventID NOT IN (%WinUpdateError%)) drop();
</Exec>
</Input>
# Output MS Event Log
<Output out_mseventlog_nxlog>
Module om_udp
Host %OUTPUT_DESTINATION_ADDRESS%
Port %OUTPUT_DESTINATION_PORT%
Exec $EventTime = integer($EventTime) / 1000000;
Exec $Message = to_json(); to_syslog_bsd();
</Output>
# Route for MS eventlog logs:
<Route route_msevent_nxlog>
Path extendedeventlog => out_mseventlog_nxlog
</Route>
But this results in the following errors in the log:
2021-04-12 16:26:55 ERROR invalid keyword: TolerateQueryErrors at C:\Program Files (x86)\nxlog\conf\nxlog.conf:116
2021-04-12 16:26:55 ERROR module 'extendedeventlog' has configuration errors, not adding to route 'route_msevent_nxlog' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:201
2021-04-12 16:26:55 ERROR route route_msevent_nxlog is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:201
2021-04-12 16:26:55 WARNING no routes defined!
2021-04-12 16:26:55 WARNING not starting unused module internal
2021-04-12 16:26:55 WARNING not starting unused module extendedeventlog
2021-04-12 16:26:55 WARNING not starting unused module out_mseventlog_nxlog
If I remove 'TolerateQueryErrors', I get:
2021-04-12 17:42:04 INFO nxlog-ce-2.10.2150 started
2021-04-12 17:42:04 ERROR failed to subscribe to msvistalog events,the channel was not found [error code: 15007]; The specified channel could not be found. Check channel configuration.
Any obvious things for me to start checking/debugging this?
Thanks in advance!
sejoneshull created
New to NXLog
gglover created
I'm new to NXLog. I have inherited this from previous engineers no longer with my company. I have to update the server OS and wanted to know when updating the server, do I have to immediately update the NXLog software or will the previous software work? If I update the NXLog software, do I have to immediately deploy the newest version of the endpoint application in my environment?
I am also asking if there are any alerts I can configure on the server. We recently learned a connection was lost and want to ensure we are notified when this happens.
Thanks all!
gglover created
Need to read only last line from my log file and post it to centralized log
Vijay created
We are looking the log file for n number of Application and pushing into one centralized log. We are reading the log file for every 15 min.Now I want read only last line of all logfiles and need to push that into centralized log. Any cone could you please help on this.
Vijay created
How can I connect from NXLog Manager to NXLog Service to query
Mayuran created
I'm seeing we can add an agent https://nxlog.co/documentation/nxlog-user-guide/nxlog_manager_installation.html#nxlog_manager_agentinstall
But I'm not seeing it in the downloads https://nxlog.co/products/nxlog-manager/download so, I'm not able to download the agent. Could someone help me to connect to NXLog Service from NXLog Manager to query or search.
Mayuran created
NXlog config error for DC events
tyros77 created
Hello,
I am having issues setting up the config file for NXlog to pull security events from DC and send to Graylog for my school Capstone project. In the log I keep getting an error "Couldnt parse Exec block at ... couldnt parse statement at line 58, character 218 in .... sytnax error, unexpected )". I utilized a config setup from NXlog documentation and adjusted for my needs. I am not sure what is wrong as the exec block looks proper to me. Here is what I have in my config file:
define SecuritySrc Microsoft-Windows-Security-Auditing
<Input DC_events>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Security">*[System[Provider[
@Name='%SecuritySrc%']]]
</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if not (defined($SourceName) and
($EventID IN (%SecurityIDs%) and $SourceName == "%SecuritySrc%")) <------ line 58
drop();
</Exec>
</Input>
I appreciate any help on this!
tyros77 created
Upgrade queries
Anjan_nxlog created
Hi Team,
We are planning to upgrade nxlog enterprise edition from 3.x to 5.x version.
While planning we were looking at the compatibility matrix of nxlog collector , agent , elastic search and kibana.
to be precise what version of Elastic search will be supoourted with Nxlong v5.x.
Regards
Anjan Kumar Tripathy
Anjan_nxlog created
Unnecessary logs when forwarding Windows DNS logs as syslog
johnrclark54 created
Hello everyone,
I have been noticing some seemingly unnecessary logs being sent to our SIEM when using nxlog for Windows DNS syslog forwarding.
Here is my current configuration:
nxlog.conf
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_file
File 'C:\Windows\Sysnative\dns\dns.log'
SavePos TRUE
ReadFromLast TRUE
PollInterval 1
Exec $Message = $raw_event; $SyslogFacilityValue = 22;
</Input>
<Output out1>
Module om_tcp
Host 10.5.1.3
Port 1470
Exec to_syslog_bsd();
</Output>
<Route 1>
Path in => out1
</Route>
<Extension _fileop>
Module xm_fileop
</Extension>
Windows DNS Debug Logging Configuration:
Log packets for debugging: check
Packet direction:
Outgoing: check
Incoming: check
Transport protocol:
UDP: check
TCP: check
Packet contents:
Queries/Transfers: check
Updates: check
Notifications: not checked
Packet Type:
Request: check
Response: check
Other options:
Log unmatched incoming response packets: not checked
Details: not checked
Filter packets by IP address: not checked
File path and name: c:\windows\system32\dns\dns.log
Maximum size (bytes): 500000000
The result is that all of the necessary logs are being sent to our SIEM, but there are a bunch of unnormalized logs being sent that I do not quite understand.
Here are examples:
<181>(date/time) (System Name)
As an example, for a server named DC1 on March 19th, 14:39.
<181>Mar 19 14:39:17 DC1
A ton of these are being sent, close to 1 for each normal DNS log. As you could imagine this is using a lot of our event log storage. Does anyone know what is causing this? Or what this <181> is referring to?
Thank you!
johnrclark54 created
Empty values in xm_kvp
pihentagy created
xm_kvp seems to not tolerate empty fields.
Used in the following scenario:
<Extension kvp>
Module xm_kvp
KVPDelimiter |
KVDelimiter :
</Extension>
The test values are:
dummy:|hello:world
How can I allow empty values for values?
pihentagy created
Fortigate reliable syslog does not work with NXLog
h.petroll created
Hi,
we have a test setup with one Fortigate (v6.4.4) and we wanted to use tcp for log collection. We can see the Forti sending the packets (tcpdump) to our NXLog-Server and we can see them arriving (tcpdump) but the packets are not being processed by the NXLog. Using udp evertyhing works fine.
The config on the Forti is standard:
config log syslogd setting
set status enable
set server "10.0.172.41"
set mode reliable
set port 2570
end
If we switch to mode legacy-reliable we can see log entries but the look rubbish.
On the NXLog we use im_tcp as input and we route it with om_file into a text file.
Pretty straight forward but it does not work.
Has anyone ever used Fortinet tcp syslog with NXLog?
Regards Hardy
h.petroll created
NXLog Multline being split
AaronChapman created
Hi All,
I've setup NXLog to read an application log file and forward the raw_event to Graylog manager. It seems initially when I start NXLog everything works correctly and the Headerline correctly breaks the log sections but after a while it seems to ignore the headerline and individual lines start coming through to graylog. I've loaded the log into regex testers and confirmed that the headerline is correctly recognised, is there anything else I can try? Can I confirm this is an NXLog issue instead of Graylog in some way?
<Extension proforma_email_parse>
Module xm_multiline
HeaderLine /\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d.\d\d\d .... ........ ----------------------------------------------------/
</Extension>
<Input proforma_email>
Module im_file
File "D:\Program Files\ProformaEmailService\Logs\ProformaEmail_*.txt"
SavePos true
ReadFromLast true
Exec $Message = $raw_event;
InputType proforma_email_parse
</Input>
Thanks
AaronChapman created
Setting $director from Regex of file location
rbpalmer2401 created
Hi All,
I need to change this format 'C:\logfile\directory1\directory2'
into '\logfile\directory1\directory2\'
using regex and the command
exec $directory =
this is to be able to set the destination folder the same as the location folder recursively
rbpalmer2401 created
NXLog CE - Exchange log
gtsintaris created
Hi all,
I'm trying to integrate the Exchange Logs to NXLog CE using the example config from https://nxlog.co/documentation/nxlog-user-guide/exchange.html which does'nt work. After reading through the forums I have come up with the following nxlog.conf which still doesn't work producing the errors at the end of this post. Is there any comprehensive guide to setup it up?
---------------- nxlog.conf ----------------------
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
define BASEDIR E:\Exchange Server
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Extension _fileop>
Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
<Extension csv_parser>
Module xm_csv
Fields date-time, client-ip, client-hostname, server-ip, server-hostname,
source-context, connector-id, source, event-id,
internal-message-id, message-id, network-message-id,
recipient-address, recipient-status, total-bytes, recipient-count,
related-recipient-address, reference, message-subject,
sender-address, return-path, message-info, directionality,
tenant-id, original-client-ip, original-server-ip, custom-data,
transport-traffic-type, log-id, schema-version,
session, source, Destination, direction, description,
session-id, sequence-number,
local-endpoint, remote-endpoint, event, data, context
Exec rename_field('sender-address', 'sender_address'); $sender_address = lc($sender_address);
Exec rename_field('date-time', 'date_time'); $date_time = lc($date_time);
</Extension>
<Input messagetracking>
Module im_file
File '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
<Exec>
if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
else
{
csv_parser->parse_csv();
$EventTime = parsedate($date_time);
}
</Exec>
</Input>
<Input connectivity>
Module im_file
File '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG'
<Exec>
if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
else
{
csv_parser->parse_csv();
$EventTime = parsedate($date_time);
}
</Exec>
</Input>
<Input smtp_receive>
Module im_file
File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG'
<Exec>
if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
else
{
csv_parser->parse_csv();
$EventTime = parsedate($date_time);
}
</Exec>
</Input>
<Input smtp_send>
Module im_file
File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG'
<Exec>
if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
else
{
csv_parser->parse_csv();
$EventTime = parsedate($date_time);
}
</Exec>
</Input>
<Extension _json>
Module xm_json
</Extension>
<Output out>
Module om_http
URL http://kibanaip:9200
ContentType application/json
<Exec>
set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" +
$SourceModuleName));
rename_field("timestamp", "@timestamp");
to_json();
</Exec>
</Output>
---------------- nxlog.conf ----------------------
Errors:
2021-03-05 11:46:03 WARNING no routes defined!
2021-03-05 11:46:03 WARNING not starting unused module messagetracking
2021-03-05 11:46:03 WARNING not starting unused module connectivity
2021-03-05 11:46:03 WARNING not starting unused module smtp_receive
2021-03-05 11:46:03 WARNING not starting unused module smtp_send
2021-03-05 11:46:03 WARNING not starting unused module out
2021-03-05 11:46:03 INFO nxlog-ce-2.10.2150 started
Thank You in advance!
Gregory
gtsintaris created
Forwarding WEF / collector events onto another source.
DW_268040 created
Hi, we have windows event forwarding configured with a lot of subscripts and filtering already configured across a deployemnt of Windows servers.
I would like to use NXLOG EE to install on our two Windows event collectors to forward these events off to a SIEM. Looking at the documentation seems I need to setup a Linux server to act as the WEC for this purpose?
DW_268040 created
Firewall Event Aggregation
Tenways created
I've got a simple config listening on 514 UDP/TCP and forwarding everything received out to another server for ingest. One of the things I've been having trouble figuring out is how to do simple event aggregation for firewall logs. Ideally it would aggregate over a time window and append the message with a new field containing the count of messages.
I know something like this used to be done via module "pm_norepeat", but I think this is being deprecated, and I'm not aware that it is capable of appending message count to the original message. It seems this should somehow be done using variables going forward.
To add to the complexity, we have two separate firewall types within our environment, (Cisco ASA's and Palo's). Greatly appreciate if anyone can point me in the right direction.
Tenways created
NXlog to read the contents of several files in a directory and send them to a Syslog Watcher server
ChristopheC13 created
Hello NXLog user,
I'm new to Nxlog and I'm looking to send the contents of files (*.log) located in a single directory to a Syslog Watcher server (snmpsoft).
A trace can be on one or multiple lines but each start of a new trace starts with the time.
Here is an example of a trace on one line (columns => time type severity thread file function content) :
17:13:00.000 APP__ WARNING 1aa20b54100 <capp.cpp:56> [[CApp::ExistInDirectory]] File 'tool.exe' not found in C:/prog
Here is an example of multiple traces and one on multiple line (columns => time type severity thread file function content) :
10:57:25.924 SQL__ ERROR__ 20d962b1270 <csqlquery.cpp:292> [[CSqlQuery::ErrQuery]] [cnx2238] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Violation de la contrainte PRIMARY KEY « PK__T__3214EC27053CB3F2 ». Impossible d'insérer une clé en double dans l'objet « B.T ». Valeur de clé dupliquée : (44422). [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]L'instruction a été arrêtée. QODBC3: Impossible d'exécuter l'instruction
Source query *
..
Query with values *
...
10:57:25.926 APP__ ERROR__ 20d962b1270 <chistosystemautomate.cpp:67> [[CHistoSystemAutomate::MsgReceivedSystem]] {
"code": 500004,
"details": "",
"message": "Impossible d'exécuter la requête SQL."
}
Nxlog.conf that I tried to do
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %ROOT%\data\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Input messages>
Module im_file
File "C:\Users\toto\Documents\nxlog*.log"
Exec $Message = $raw_event;
</Input>
<Output tcp>
Module om_tcp
Host 0.0.0.0
Port 514
Exec to_syslog_bsd();
</Output>
<Output udp>
Module om_udp
Host 0.0.0.0
Port 514
Exec to_syslog_bsd();
</Output>
<Route messages_to_tcp>
Path messages => tcp
</Route>
<Route messages_to_udp>
Path messages => udp
</Route>
Can you help me to write the conf file.
A big thank you in advance
Christophe
ChristopheC13 created
NXlogv5 om_http connection bug
sk_adm created
We probably found a bug in nxlog-5.2.6388. We use om_http to transfer logs to the server. If there are some new logs in our log file, the nxlog on start connects to server, makes ssl handshake and sucessfully sends logs. BUT if there is no new log entry during the nxlog startup, the om_http connects to the server while NOT doing SSL handshake (only tcp handshake), after some time the web server closes the connection (sends FIN+ACK because it expects SSL handshake to occur during its timeout), after that the nxlog sends ACK and client hello. This seems to be very buggy behaviour as nxlog does not perform ssl handshake until the server tries to close the connection BUT ALWAYS tries to after the server initiates TCP closing. After adding some new line to the log file, the nxlog succefully connects to web server, makes ssl handshake and keeps the connection.
Could you confirm the bug?
sk_adm created
random extra comma showing up in Splunk
boostcreep created
So I'm working on moving our logging away from SolarWinds and into nxlog. We like to punt off our logs to Splunk so that our security department can create dashboards and such for the events they care about. I've just about got the output perfect, besides ONE random extra comma and I can't figure out where it's coming from.
Here is my config:
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LOGFILE %ROOT%\data\nxlog.log
LogLevel INFO
<Extension syslog>
Module xm_syslog
</Extension>
define MonitoredEventIDs 1100, 1102, 1104, 4608, 4609, 4624, 4625, 4634, 4647, 4648,
4656, 4658, 4659, 4660, 4661, 4662, 4663, 4664, 4670, 4672,
4673, 4724, 4719, 4720, 4722, 4723, 4724, 4725, 4726, 4727,
4728, 4729, 4731, 4732, 4733, 4734, 4735, 4737, 4738, 4739,
4740, 4766, 4767, 4768, 4776, 4781, 4801, 4825, 4907, 5136,
5137, 5139, 5141, 5145, 6416, 13002, 13003, 18500, 18502, 307
<Input eventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Security"></Select>
<Select Path="Microsoft-Windows-PrintService/Operational"></Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
if $EventID NOT IN (%MonitoredEventIDs%) drop();
</Exec>
</Input>
<Input GFI>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="GFI EndPointSecurity">*</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
<Output out>
Module om_udp
Host 10.1.0.1
Port 514
Exec to_syslog_snare();$raw_event = replace($raw_event, "\t", ",");
</Output>
<Route>
Path eventlog, GFI => out
</Route>
That creates this beautiful output in Splunk, except for the random extra comma before 'An account was logged off.' This single extra comma is throwing off the parsing of the logs that our customer receives and I need to get rid of it... Any advice?
Mar 1 21:58:26 mycomputer.com MSWinEventLog,1,Security,79,Mon Mar 01 14:58:26 2021,4634,Microsoft-Windows-Security-Auditing,N/A,N/A,Success Audit,mycomputer.com,Logoff,,An account was logged off. Subject: Security ID: S-1-5-21-2294171146-2094350030-1588952898-500 Account Name: pcgroup Account Domain: mycomputer Logon ID: 0x2D069A2F Logon Type: 2 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.,1023042
boostcreep created