Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
No logs are collected from Fortinet units, but tcpdump on NXlog collector shows ingoing traffic coming from them
DS_534595 created
I have a setup using NXlog instances as collectors in a large number of security zones.
<Input in0>
Module im_tcp
Host XXX.XXX.XXX.XXX
</Input>
but for some reason this does not capture logs coming in on port 514 from Fortinet units; all other logs (from Windows and Linux servers) are received and processed just fine.
tcpdump -nvvA host [Fortinet unit IP]
shows log traffic coming in on the NIC from the given IP address.
What am I missing?
DS_534595 created
NXLog EE Trial Limitations
Moulos created
Hello.
We are testing the NXLog EE Trial version on Windows and want to know what are its limitations?
Will be expire after some time?
Are some modules not working?
How many days can we try it?
Is there a FAQ to explain further the Trial limitations? because we did not find any.
Thank you.
Moulos created
NX .conf - Drop Windows events based on hostname
Dingofest2 created
Hello everyone
I have the following EXEC IF statement in my configuration file to drop events if username fields are equal to the computer account name. As you know Windows computer account names always end in $.
host1$
host2$
etc.
if $EventID == 4624 AND ($TargetUserName == 'DESKTOP-XY43$' OR $SubjectUserName == 'DESKTOP-XY43$') drop();
Above IF statement works perfectly, however, I have several other IF statements for various event ID's AND several hundred Windows hosts. You can imagine the time and effort required to customize several hundred .conf files for each Windows desktop.
Does anyone know if I could use REGEX or some other technique like wildcard to simply say if TargetUserName or SubjectUserName string ends in $, then drop the event.
IF $EventID == 4624 AND ($TargetUserName == *$ OR $SubjectUserName == *$) drop();
Thank you
Dingofest2 created
'parse_csv' failed - statement execution has been aborted; Invalid CSV input:
Raji created
Hello Team,
I have configured nxlog.conf for .csv file in my production server.
.csv file content :
(11/6/2021 4:18:18 PM) | 06/11/2021 15:42:26 | "SM/D Mior Ahmad Suhaimi bin Mohd Alwi" <miorsuhaimi@rmp.gov.my> | PUCHONG REPOT 8259/20 | Wrong or No request form
(11/6/2021 4:28:06 PM) | 06/11/2021 15:19:56 | "SI Mohamad Azizi B Yunus" <aziziyunus@rmp.gov.my> | JAYA GADING REPOT 2297/2021, SEK 395/397 KK. | Success
(11/6/2021 4:39:33 PM) | 06/11/2021 15:03:32 | "ASP Muhd Shamsuddin Wong bin Abdullah" <muhdshamsuddin@rmp.gov.my> | SRI PETALING 2159/21 | Success
(11/6/2021 5:14:42 PM) | 06/11/2021 16:06:01 | "Analisis Risikan Maklumat Telefon JSJN Bukit Aman" <telconaco@rmp.gov.my> | LMAF/JSJN/83/2021 | Success
(11/6/2021 6:00:43 PM) | 06/11/2021 17:10:16 | "Insp Ishak B Abu Bakar" <ishak_ab@rmp.gov.my> | PERMOHONAN BUTIR-BUTIR PEMILIK NOMBOR TELEFON | Wrong or No request form
(10/5/2021 9:00:45 AM) | 05/10/2021 08:04:08 | lflee@moh.gov.my | FW: JBS/021886/19 | Wrong or No request form
nxlog Conf file content for csv part :
<Extension csv>
Module xm_csv
Fields date1, date2, mailid, name, result
Delimiter |
#EscapeControl TRUE
</Extension>
<Input Security_processedlog>
Module im_file
Exec $type = 'Security_processedlog';
File 'D:\RPAMain\Logs\General\Security\ProcessedLog*.csv'
InputType LineBased
PollInterval 1
SavePos TRUE
Exec csv->parse_csv();
Exec $Message = $raw_event;
</Input>
I can see only the output for this entry
(10/5/2021 9:00:45 AM) | 05/10/2021 08:04:08 | lflee@moh.gov.my | FW: JBS/021886/19 | Wrong or No request form
Nxlog log says error for rest entries
(11/6/2021 4:18:18 PM) | 06/11/2021 15:42:26 | "SM/D Mior Ahmad Suhaimi bin Mohd Alwi" <miorsuhaimi@rmp.gov.my> | PUCHONG REPOT 8259/20 | Wrong or No request form
(11/6/2021 4:28:06 PM) | 06/11/2021 15:19:56 | "SI Mohamad Azizi B Yunus" <aziziyunus@rmp.gov.my> | JAYA GADING REPOT 2297/2021, SEK 395/397 KK. | Success
(11/6/2021 4:39:33 PM) | 06/11/2021 15:03:32 | "ASP Muhd Shamsuddin Wong bin Abdullah" <muhdshamsuddin@rmp.gov.my> | SRI PETALING 2159/21 | Success
(11/6/2021 5:14:42 PM) | 06/11/2021 16:06:01 | "Analisis Risikan Maklumat Telefon JSJN Bukit Aman" <telconaco@rmp.gov.my> | LMAF/JSJN/83/2021 | Success
(11/6/2021 6:00:43 PM) | 06/11/2021 17:10:16 | "Insp Ishak B Abu Bakar" <ishak_ab@rmp.gov.my> | PERMOHONAN BUTIR-BUTIR PEMILIK NOMBOR TELEFON | Wrong or No request form
(10/5/2021 9:00:45 AM) | 05/10/2021 08:04:08 | lflee@moh.gov.my | FW: JBS/021886/19 | Wrong or No request form
2021-06-11 19:44:39 ERROR procedure 'parse_csv' failed at line 376, character 22 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Invalid CSV input: '(20/5/2021 5:20:19 PM) | 05/20/2021 16:21:12 | "Analisis Risikan Maklumat Telefon JSJN Bukit Aman" <telconaco@rmp.gov.my> | LMAF/JSJN/56/2021 | Success'
2021-06-11 19:44:39 ERROR procedure 'parse_csv' failed at line 376, character 22 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Invalid CSV input: '(20/5/2021 6:07:09 PM) | 05/20/2021 17:21:40 | "SI Mohamad Azizi B Yunus" <aziziyunus@rmp.gov.my> | KUANTAN REPOT 7090/21, SEK 302 KK | Success'
Please suggest
Thanks
Raji created
WARNING Module ##### has no input files to read
Raji created
Hello Team,
I have added nxlog.conf for our windows application server.
See the nxlog reference manual at
http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
Include fileop while debugging, also enable in the output module below
#<Extension fileop>
Module xm_fileop
#</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Extension csv>
Module xm_csv
Fields date1, date2, mailid, name, result
Delimiter |
#EscapeControl TRUE
</Extension>
<Input internal>
Module im_internal
</Input>
Watch your own files
<Input file1>
Module im_file
File '%ROOT%\data\nxlog.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input COPNewMNPAR_general>
Module im_file
Exec $type = 'COPNewMNPAR_general';
File 'D:\RPAMain\Logs\General\COPNewMNPAR\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input COPNewMNPAR_approcessed>
Module im_file
Exec $type = 'COPNewMNPAR_approcessed';
File 'D:\RPAMain\Logs\General\COPNewMNPAR\AppsProcessed*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input COPNewMNPAR_processing>
Module im_file
Exec $type = 'COPNewMNPAR_processing';
File 'D:\RPAMain\Logs\General\COPNewMNPAR\AppsProcessing*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input COPNewMNPAR_exception>
Module im_file
Exec $type = 'COPNewMNPAR_exception';
File 'D:\RPAMain\Logs\Exception\COPNewMNPAR\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input igniteserver>
Module im_file
Exec $type = 'igniteserver';
File 'C:\ProgramData\AutomationAnywhere\Logs\IgniteServer*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input HR Process1_security>
Module im_file
Exec $type = 'HR Process1_security';
File 'D:\RPAMain\Logs\General\HR Process1\Security\Security*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input HR Process1_general>
Module im_file
Exec $type = 'HR Process1_general';
File 'D:\RPAMain\Logs\General\HR Process1\Genaral*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input HR Process1_exception>
Module im_file
Exec $type = 'HR Process1_exception';
File 'D:\RPAMain\Logs\Exception\HR Process1\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input HR Process2_general>
Module im_file
Exec $type = 'HR Process2_general';
File 'D:\RPAMain\Logs\General\HR Process2\Genaral*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input HR Process2_exception>
Module im_file
Exec $type = 'HR Process2_exception';
File 'D:\RPAMain\Logs\Exception\HR Process2\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot1_general>
Module im_file
Exec $type = 'Bot1_general';
File 'D:\RPAMain\Logs\General\Bot1\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot1_exception>
Module im_file
Exec $type = 'Bot1_exception';
File 'D:\RPAMain\Logs\Exception\Bot1\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot1_approcessed>
Module im_file
Exec $type = 'Bot1_approcessed';
File 'D:\RPAMain\Logs\App Process\Bot1\AppProcessed*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot1_processing>
Module im_file
Exec $type = 'Bot1_processing';
File 'D:\RPAMain\Logs\AppProcess\Bot1\AppProcessing*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot1_security>
Module im_file
Exec $type = 'Bot1_security';
File 'D:\RPAMain\Logs\Security\Bot1\Security*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot3_general>
Module im_file
Exec $type = 'Bot3_general';
File 'D:\RPAMain\Logs\General\Bot3\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot3_exception>
Module im_file
Exec $type = 'Bot3_exception';
File 'D:\RPAMain\Logs\Exception\Bot3\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot3_approcessed>
Module im_file
Exec $type = 'Bot3_approcessed';
File 'D:\RPAMain\Logs\App Process\Bot3\AppProcessed*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot3_processing>
Module im_file
Exec $type = 'Bot3_processing';
File 'D:\RPAMain\Logs\AppProcess\Bot3\AppProcessing*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot3_security>
Module im_file
Exec $type = 'Bot3_security';
File 'D:\RPAMain\Logs\Security\Bot3\Security*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input MNPSAC_general>
Module im_file
Exec $type = 'MNPSAC_general';
File 'D:\RPAMain\Logs\General\MNPSAC\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input MNPSAC_exception>
Module im_file
Exec $type = 'MNPSAC_exception';
File 'D:\RPAMain\Logs\Exception\MNPSAC\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPARefund_general>
Module im_file
Exec $type = 'CPARefund_general';
File 'D:\RPAMain\Logs\General\CPARefund\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPARefund_exception>
Module im_file
Exec $type = 'CPARefund_exception';
File 'D:\RPAMain\Logs\Exception\CPARefund\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPARefund_processed>
Module im_file
Exec $type = 'CPARefund_processed';
File 'D:\RPAMain\Logs\General\CPARefund\processed*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPARefund_processing>
Module im_file
Exec $type = 'CPARefund_processing';
File 'D:\RPAMain\Logs\General\CPARefund\processing*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPA_general>
Module im_file
Exec $type = 'CPA_general';
File 'D:\RPAMain\Logs\General\CPA\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPA_exception>
Module im_file
Exec $type = 'CPA_exception';
File 'D:\RPAMain\Logs\Exception\CPA\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input CPA_processed>
Module im_file
Exec $type = 'CPA_processed';
File 'D:\RPAMain\Logs\RefundMasterList\ProcessedLog*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot2_general>
Module im_file
Exec $type = 'Bot2_general';
File 'D:\RPAMain\Logs\General\Bot2\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot2_exception>
Module im_file
Exec $type = 'Bot2_exception';
File 'D:\RPAMain\Logs\Exception\Bot2\Exception*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot2_approcessed>
Module im_file
Exec $type = 'Bot2_approcessed';
File 'D:\RPAMain\Logs\App Process\Bot2\AppProcessed*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot2_processing>
Module im_file
Exec $type = 'Bot2_processing';
File 'D:\RPAMain\Logs\AppProcess\Bot2\AppProcessing*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Bot2_security>
Module im_file
Exec $type = 'Bot1_security';
File 'D:\RPAMain\Logs\Security\Bot2\Security*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Security_general>
Module im_file
Exec $type = 'Security_general';
File 'D:\RPAMain\Logs\General\Security\General*.txt'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Input Security_processedlog>
Module im_file
Exec $type = 'Security_processedlog';
File 'D:\RPAMain\Logs\General\Security\ProcessedLog*.csv'
InputType LineBased
PollInterval 1
SavePos TRUE
Exec csv->parse_csv();
Exec $Message = $raw_event;
</Input>
Windows Event Log
<Input eventlog>
Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog
Uncomment im_mseventlog for Windows XP/2000/2003
Module im_mseventlog
</Input>
<Output out>
Module om_tcp
Host 172.31.244.219
Port 3515
Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();
# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
<Route 1>
Path internal, file1, eventlog, COPNewMNPAR_general, OPNewMNPAR_approcessed, COPNewMNPAR_processing, COPNewMNPAR_exception, igniteserver, HR Process1_security, HR Process1_general, HR Process1_exception, HR Process2_general, HR Process2_exception, Bot1_general, Bot1_exception, Bot1_approcessed, Bot1_processing, Bot1_security, Bot3_general, Bot3_exception, Bot3_approcessed, Bot3_processing, Bot3_security, MNPSAC_general, MNPSAC_exception, CPARefund_general, CPARefund_exception, CPARefund_processed, CPARefund_processing, CPA_general, CPA_exception, CPA_processed, Bot2_general, Bot2_exception, Bot2_approcessed, Bot2_processing, Bot2_security, Security_general, Security_processedlog => out
</Route>
I see below warning for all application log files , nxlog is not able to read the logs from application log files. Please suggest.
021-06-11 13:22:59 WARNING Module Bot1_general has no input files to read
Module Bot1_exception has no input files to read
Module Bot1_security has no input files to read
I also see this log , nxlog is making an attempt but not able to read. These application log files are written every minute.
An attempt was made to access an object.
Subject:
Security ID: S-1-5-18
Account Name: SHTVRPACTRLP01$
Account Domain: DIGICR
Logon ID: 0x3E7
Object:
Object Server: Security
Object Type: File
Object Name: D:\RPAMain\Logs\Exception\Bot3
Handle ID: 0xcd4
Resource Attributes:
Process Information:
Process ID: 0x66ec
Process Name: C:\Program Files (x86)\nxlog\nxlog.exe
Access Request Information:
Accesses: ReadData (or ListDirectory)
Access Mask: 0x1
Thanks
Raji created
Linux to Alien Vault formatting
rwebster created
I'm relatively new to NXLog and to Alien Vault Log ingestion. I have followed their set up guide here, https://cybersecurity.att.com/documentation/usm-anywhere/deployment-guide/setup/linux-logs-nxlog.htm?Highlight=linux%20logs%20nxlog.
We are using NxLog EE and the nxlog manager to push configs.
I have two modules I'm looking to collect logs with and I feel that they are pretty straight forward. I'm trying to capture /var/log/messages and the audit.log. Also wanting to capture the FIM.
Using the im_file module. I'm able to capture these logs and they get to AlienVault, they are even parsed in Json which looks nice. My issue is that you can really report on anything as they don't generate any useful flags in which Alien Vault can use to trigger alarms and such. A lot of the logs get flagged as Alien Vault Generic Results, which means that the format isn't triggering their Alien Vault Data Source plugins. Some logs are getting recognized by Alien Vault and triggering the appropriate data source, but they are also not getting any useful information to report on. When I compare these logs to the Windows logs that we are capturing, the Linux logs have significantly less metadata within them. Something as simple as eventoutcome would be nice, like if I wanted to see failed attempts to elevate to sudo and see that event fail and generate and alert.
I'm wondering if anyone would mind sharing their module configs for Linux if you use Alien Vault (ATT USM) as your SIEM.
Here is my current config I am using.
LogLevel INFO
Logfile %LOGDIR%/nxlog.log
<Extension agent_managment>
Module xm_soapadmin
Connect 1.1.1.1
Port 4041
SocketType SSL
CAFile %CERTDIR%/agent-ca.pem
AllowUntrusted FALSE
RequireCert TRUE
<ACL conf>
Directory %CONFDIR%
AllowRead TRUE
AllowWrite TRUE
</ACL>
<ACL cert>
Directory %CERTDIR%
AllowRead TRUE
AllowWrite TRUE
</ACL>
</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input var_messages_in>
Module im_file
File '/var/log/messages'
InputType LineBased
SavePos TRUE
Exec parse_syslog();
</Input>
<Input fim_linux>
Module im_fim
File '/bin/'
File '/etc/'
File '/lib/'
File '/opt/nxlog/bin/'
File '/opt/nxlog/lib/'
File '/sbin/'
File '/usr/bin/'
File '/usr/sbin/'
Exclude '/etc/hots.deny'
Exclude '/etc/mtab'
</Input>
<Input var_audit_in>
Module im_file
File '/var/log/audit/audit.log'
InputType LineBased
SavePos FALSE
Exec parse_syslog();
$Hostname = hostname();
$FQDN = hostname_fqdn();
$Tag = "audit";
$SourceName = "selinux";
$Message = $Raw_Event;
</Input>
<Output out_syslog_ssl_br>
Module om_ssl
Host 1.1.1.1
Port 6514
OutputType LineBased
CAFile %CERTDIR%/agent-ca.pem
CertFile %CERTDIR%/agent-cert.pem
CertKeyFile %CERTDIR%/agent-key.pem
Exec $EventTime = $EventReceivedTime;
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;
Exec $Message = to_json(); to_syslog_bsd();
</Output>
<Output out_fim_linux_ssl_BR>
Module om_ssl
Host 1.1.1.1
Port 6514
OutputType Binary
CAFile %CERTDIR%/agent-ca.pem
CertFile %CERTDIR%/agent-cert.pem
CertKeyFile %CERTDIR%/agent-key.pem
Exec $SourceName = "FIM-LINUX-NXLOG-EE";
Exec $EventTime = strftime($EventTime, '%Y-%m-%d %H:%M:%S');
Exec $EventReceivedTime = strftime($EventReceivedTime, '%Y-%m-%d %H:%M:%S');
Exec to_json(); to_syslog_bsd();
</Output>
<Route route_syslog>
Priority 1
Path var_audit_in, var_messages_in => out_syslog_ssl_br
</Route>
<Route FIM>
Priority 2
Path fim_linux => out_fim_linux_ssl_BR
</Route>
rwebster created
Log forwarding to Cloud VM
hkatlia created
Hello Everyone,
I am trying to forward Windows Server logs to Azure VM. We have a config for on premises forwarding. I have tried to make changes in config according to what i found in documentation.
New Config: (WIth Changes)
When @weekly
Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
</Schedule>
</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension resolver>
Module xm_resolver
</Extension>
########################INPUTS##########################
<Input eventlog>
Module im_msvistalog
# ReadFromLast and SavePos control when we read data.
# False is good for troubleshooting, but not likely what you want in Prod
#ReadFromLast False
#SavePos False
TolerateQueryErrors True
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Microsoft-Windows-Sysmon/Operational"></Select>
<Select Path="Security"></Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
# This if is just so I only see 4662 events
#if $EventID != 4662 drop();
# These fields are set, but you will need to add the values to your output at some point.
# This is done automatically in Syslog IETF and JSON, but not Snare
$TypeR = ad_guid_to_name($ObjectType);
$NameR = ad_guid_to_name($ObjectName);
# These lines just log the values to the internal log file. Not needed for production, only troubleshooting.
log_info($raw_event);
log_info("TypeR: " + $TypeR);
log_info("NameR: " + $NameR);
</Exec>
</Input>
########################OUTPUTS##########################
<Output out>
Module om_udp
Host 52.165.172.76
Port 518
Exec to_syslog_bsd();
########################PUTS EVENT IN IETF FORMAT########
Exec to_syslog_snare();
</Output>
<Route 1>
Path eventlog => syslogout
</Route>
On-premises config
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _resolver>
Module xm_resolver
</Extension>
# This block rotates %MYLOGFILE% on a schedule. Note that if LogFile
# is changed in log4ensics.conf via NXLog Manager, rotation of the new
# file should also be configured there.
<Extension _fileop>
Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
<Exec>
if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') >= 5M) )
{
file_cycle('%MYLOGFILE%', 8);
}
</Exec>
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
</Schedule>
</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension resolver>
Module xm_resolver
</Extension>
########################INPUTS##########################
<Input eventlog>
Module im_msvistalog
# ReadFromLast and SavePos control when we read data.
# False is good for troubleshooting, but not likely what you want in Prod
#ReadFromLast False
#SavePos False
TolerateQueryErrors True
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Microsoft-Windows-Sysmon/Operational"></Select>
<Select Path="Security"></Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
# This if is just so I only see 4662 events
#if $EventID != 4662 drop();
# These fields are set, but you will need to add the values to your output at some point.
# This is done automatically in Syslog IETF and JSON, but not Snare
$TypeR = ad_guid_to_name($ObjectType);
$NameR = ad_guid_to_name($ObjectName);
# These lines just log the values to the internal log file. Not needed for production, only troubleshooting.
log_info($raw_event);
log_info("TypeR: " + $TypeR);
log_info("NameR: " + $NameR);
</Exec>
</Input>
########################OUTPUTS##########################
<Output syslogout>
Module om_udp
#Module om_tcp
Host 10.0.0.129
Port 515
########################PUTS EVENT IN IETF FORMAT########
Exec to_syslog_snare();
</Output>
<Route 1>
Path eventlog => syslogout
</Route>
</Output>
If anyone can provide help it will be helpfull
Thank you.
hkatlia created
Can NXLOG module im_dbi use AD Service account to authenticate to DB read?
bane created
I am trying to read from a MSSQL DB with an AD service account. I know the im_dbi module supports local DB account for auth but not sure about how to leverage AD service account credentials.
bane created
is it possible to negate a match using pm_match? (nxlog ce version for now)
wallet created
I am trying to us pm_pattern to filter messages that are approved for a 'limited' feed, and still have the full feed go to the admin feed.
I have set up the config to send the input to two feeds, that works fine.
I've then tried to use pm_pattern to match certain strings and DROP them from the restricted feed. So far no luck.
I'm sure I'm missing something really simple here, and would really appreciate if anyone had the time to check the configs for me...
in nxlog.conf (edited for brevity)...
Module pm_pattern
PatternFile "/data/conf/nxlog-patternmatch.yaml"
Module om_http
URL https:// destination string
#Batchmode none
HTTPSAllowUntrusted TRUE
HTTPSCADir /etc/ssl/certs/
HTTPSCertFile /etc/ssl/certs/ca-certificates.crt
Exec if defined $PatternID drop();
Path client2001 => client_filter => out2001,client-test
and this is the patternmatch yaml (some matchfields removed)...
2021-06-03 01:02:03
1
<group>
<name>Client</name>
<pattern>
<id>1</id>
<name>client input</name>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>rsyslogd</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>NetworkManager</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>Systemd</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>dnf</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>dbus</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>chrony</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>CRON</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>motd</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>snapd</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>promtail</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>nxlog</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>kernel</value>
</matchfield>
<matchfield>
<Name>raw_event</Name>
<type>regexp</type>
<value>loki</value>
</matchfield>
</pattern>
</group>
wallet created
.xlsx extension file NXLog configuration
Raji created
Hello Team,
I have a .xlsx file , I need to add NXLog configuration to send this .xlsx file contents to my Nagios Log server.
can we send .xlsx file ? please help with this extension module and input configuration.
Thanks
Raji created
Using to_syslog_snare() but with ISO8601 date format (need timezone attached to time)
heikis created
Hello. I would like to use the "to_syslog_snare()" procedure but with the use of ISO8601 timeformat so that the date and time would be formatted as "2021-05-28T07:35:49+00:00" instead of "May 28 07:35:49". How would I achieve this? Thank you!
heikis created
100% CPU Usage in Windows Server 2016
robvas created
Any suggestions on where to look for more information? Server Manager shows performance alerts and has nxlog using 100% CPU. This happens several times per day. Running nxlog-ce-2.10.2150 on Windows Server 2016 Standard.
robvas created
A certain windows event log has not been sent
AyakoFukumoto created
Hi, everyone.
I haven’t overcome the problem above.
Could anyone please share idea of ;
-The possible methods of determining the root cause of the problem
-The possible methods of overcoming this problem
As soon as you can, please!
Thank you.
<The problem>
When sending event logs from NXlog, a certain event log has never been sent to the windows log collection server.
E.g.
Event ID: 4624(Successful Logon)->Has not been sent
Event ID: 4634(Logout)->have been sent
<The methods already tested >
1.Debugging;
The following debug log was configured in order to test that target event log (ID: 4624) was recognized by NXlog.
Exec if ($EventID == 4624) log_info("EventID = 4624 | " + $EventID + " | " + $EventTime);
The test shows that the event log was recognized by NXlog, as it was output to NXlog as follows.
2021-05-14 19:22:17 INFO EventID = 4624 | 4624 | 2021-05-14 19:22:17
Explicit output of the target event log (ID4624);
The test shows that the expected event log has not been sent, though the following event logs were sent after specified event logs explicitly.
#In Windows Event Log (Event ID:4624 or 4625)
<Input In_eventlog_logon>
Module im_msvistalog
Exec if ($EventID == 5156) drop();
Exec if ($EventID == 4624) log_info("EventID = 4624 | " + $EventID + " | " + $EventTime + " | " + $Hostname);
<QueryXML>
<QueryList>
<Query Id='0'>
<Select Path="Security">*[System[(EventID=4624 or EventID=4625)]]</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
AyakoFukumoto created
Windows event log service and nxlog service dependancies
unda.verse created
Hi everyone,
I would like to know if some standard installation of nxlog on a AD could lead to the nxlog service becoming dependant of the windows event log service ?
It not supposed to be, but for some reason (that I didn't understand yet) it happens in my case.
Thanks.
Regards
unda.verse created
Formatting codes in multi line windows event ?
farridem created
Hello,
I am attempting to use Nxlog on windows to forward windows event logs as syslog. I am finding that the windows event 4672 (and only this event oddly enough) keeps getting broken into multiple lines and showing the character strings
#011 and #015
May 18 10:29:20 desktop-XXXX #011#011#011SeLoadDriverPrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeBackupPrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeRestorePrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeDebugPrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeAuditPrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeSystemEnvironmentPrivilege
#015
May 18 10:29:20 desktop-XXXX #011#011#011SeImpersonatePrivilege#015
May 18 10:29:20 desktop-XXXX #011#011#011SeDelegateSessionUserImperso
natePrivilege" EventReceivedTime="2021-05-18 10:29:20" SourceModuleName="eventlog" SourceMod
uleType="im_msvistalog"] {"EventTime":"2021-05-18 10:29:19","Hostname": <snip>
This event also shows up with the FQDN instead of the hostname that the other events are sent with. The logs are being formatted to JSON prior to sending
I reviewed the documentation and I can't determine if there is a way to effect the parsing of this message.
Thanks for any input !
farridem created
Nxlog maxing out CPU on server
SocAnalyst created
Hello,
We have installed NXlog on a server (remote desktop VM) but it is maxing out the CPU usage along with Events service control manager.
Before turning on Nxlog service, CPU is at 25%, after it is turned on CPU spikes up to 98% with just those two.
We suspected a loop due to auditevents and the only two that are activated are Registery and handle audits. So we went ahead and disabled those but it just went from 98% to around 90% so we just turned them back on.
Patterndb is not changing anything to it either.
We have other VMs with the same set up and with Nxlog that are running smoothly.
Anyone has a possible solution for this issue?
Thank you
SocAnalyst created
IIS Compressed Logs
Pizza1 created
I currently have the nxlog EE pulling IIS logs to a McAfee SIEM. The IIS logs are arriving fine from some devices, but others not. noticed during an incident that the IIS logs are in blue, which turns out that they are compressed. The other modules are working fine, the IIS module loads, there are no errors nor warnings given in the nxlog agent log, but no data gets collected.
Is there a different module to use, or a verbatim command to add to grab these compressed files?
Pizza1 created
include_stdout not working correctly (enterprise ediction)
Pizza1 created
I'm trying to use the stdout to use a powershell script for IIS to check all location for any w3svc* folders and collect the logs inside them. Currently getting the error:
"Couldn't process 'include' directly at <nxlog default location>; Invalid 'include_stdout' directive at <nxlog default location>; im_exec process %ROOT%\get_iis_paths.cmd exited normally with exitval: 1; The specified child process is done executing"
The verbatim config uses:
include_stdout %ROOT%\get_ftp_log_paths.cmd
InputType IIS_W3C
What am I missing here?
Pizza1 created
im_file - parse if file is newer than X days (or fixed date)
kumdabur created
Hi,
I'm testing nxlog with IIS servers. It works, however I have following issue:
If server has IIS since years and lets assume that logs were stored for 1 year, I have bunch of iis logfiles which all together combined results with around 5GB of Data.
While using nxlog as-it-is, it consumes CPU and also SIEM itself is not entirely happy about receiving sudden "bombarding" of new logs within few minutes timeframe. Unfortunately, I have around 50 servers like that and I really do not need to inject past logs.
I wonder if there is an option to either throttle nxlog a bit or in best case I'd like to send ONLY new iis logs, even starting from exact today. Below you may find part of config
<Input IIS_Logs_1>
Module im_file
File 'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
ReadFromLast FALSE
Recursive TRUE
PollInterval 1
Exec $FileName = file_name();
Exec if $raw_event =~ /^#/ drop();\
else\
{\
w3c_1->parse_csv();\
$SourceName = "IIS";\
}
</Input>
I'd appreciate any hints.
kumdabur created
Ability to read / parse newest log record from source log files
rnandikotkur created
I am using NxLog to read and output logs from various source files (im_file module). I configured NxLog with "ReadFromLast" hint so it is capturing new logs since last "SavePos".
Now I want to change this little bit, I want to read just the newest entry in my log files since "SavePos". How can I achieve this? Is there a hint / filter that can help?
rnandikotkur created