Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Windows 2016 to RHEL 7 Syslog collector
syslog_guru_192 created
Hi,
I am trying to forward Windows 2016 event logs to a RHEL 7 syslog collector, on the community edition. I am not receiving any logs at the collector. I know that this is not a network issue as syslog generator tools (such as kiwi) are working. Current config is below:
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_msvistalog
Query <QueryList>
<Query Id="0">
<Select Path="Security">*[System[(EventID=4624 or EventID=4634 or EventID=4656 or EventID=4659 or EventID=4662 or EventID=4663 or EventID=4672 or EventID=4676 or EventID=6272 or EventID=6278)]]</Select>
</Query>
</QueryList>
</Input>
<Output out>
Module om_udp
Host <myhostip>
Port 514
Exec to_syslog_snare();
</Output>
<Route 1>
Path in => out
</Route>
Any pointers as to where I'm going wrong? Ideally I would like to send all Windows Event logs, not just the IDs listed above.
syslog_guru_192 created
High CPU/service stopping
EmilyH created
Hi all.
We've got nxlog installed on multiple servers, and at least once a day we get an incident where the CPU usage massively spikes - 90%+ being used by nxlog. Then the service stops (presumably killed because of the high CPU).
Has anyone experienced this before?
EmilyH created
gzipped content to a socket
davidatpinger created
Hey!
I can think of several ways to solve this, but I'm looking to see if anyone has a better idea. I'm getting some logs that are uploaded as a gzipped file. I need to uncompress the file and then send the log content to several destinations, but for this purpose, let's assume that I'm using om_tcp as the destination. I'm considering a couple of options, but has anyone already done something like this or have a brilliant idea for handling it?
Thanks!
davidatpinger created
NXLOG works over udp, not in tcp
sbergami created
Hi,
I'm new in NXLOG and Graylog. I'm using nxlog to send Windows server log to Graylog,
My final goal is to transfer and log in ssl, but proceed by step. All my machines are on my company lan
The configuration of NXLOG with OM_UDP
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_gelf
</Extension>
<Input in>
Module im_msvistalog
Query <QueryList>
<Query Id="0">
<Select Path="Security">*[System[(EventID=4624 or EventID=4634 or EventID=4656 or EventID=4659 or EventID=4662 or EventID=4663 or EventID=4672 or EventID=4676 or EventID=6272 or EventID=6278)]]</Select>
</Query>
</QueryList>
</Input>
<Output out>
Module om_udp
Host 10.0.0.22
Port 12201
#Exec to_syslog_snare();
Exec if ($ProcessName =~ /TbService.exe/ or $ProcessName =~ /w3wp.exe/) drop();
OutputType GELF
</Output>
<Route 1>
Path in => out
</Route>
works correctly, while the one, identical, that I did with OM_TCP
<Output out>
Module om_tcp
Host 10.0.0.22
Port 12201
#Exec to_syslog_snare();
Exec if ($ProcessName =~ /TbService.exe/ or $ProcessName =~ /w3wp.exe/) drop();
OutputType GELF
</Output>
create this log
018-04-11 19:48:06 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-11 19:48:06 INFO reconnecting in 2 seconds
2018-04-11 19:48:07 INFO connecting to 10.0.0.22:12201
2018-04-11 19:48:07 INFO reconnecting in 1 seconds
2018-04-11 19:48:07 ERROR om_tcp send failed; An established connection was aborted by the software in your host machine.
2018-04-11 19:48:08 INFO connecting to 10.0.0.22:12201
2018-04-11 19:52:59 ERROR couldn't connect to tcp socket on 10.0.0.22:12201; No connection could be made because the target machine actively refused it.
2018-04-11 19:53:01 INFO connecting to 10.0.0.22:12201
2018-04-11 19:53:02 INFO reconnecting in 4 seconds
2018-04-12 10:02:09 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-12 10:02:10 INFO connecting to 10.0.0.22:12201
2018-04-12 10:02:10 INFO reconnecting in 1 seconds
2018-04-12 10:02:10 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-12 10:02:11 INFO connecting to 10.0.0.22:12201
2018-04-12 10:02:11 INFO reconnecting in 1 seconds
2018-04-12 10:02:11 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-12 10:02:11 INFO reconnecting in 2 seconds
2018-04-12 10:02:12 INFO connecting to 10.0.0.22:12201
2018-04-12 10:02:12 INFO reconnecting in 1 seconds
2018-04-12 10:02:12 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-12 10:02:13 INFO connecting to 10.0.0.22:12201
2018-04-12 10:02:13 INFO reconnecting in 1 seconds
2018-04-12 10:02:13 ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
2018-04-12 10:02:13 INFO connecting to 10.0.0.22:12201
does not work correctly
In graylog i recive 1 message / minute Instead of 15/20 which I receive with the udp onfiguration,even if the Graylog input message counter reports about 20 / minute. It seems that most messages are discarded as incorrect
On Graylog server firewall port 12201 tcp/udp is open, and on windows server 12201 port is open too.
I have no idea what's going on. thank you
sbergami created
Nxlog Exchange
james811 created
Hello,
I would like to use nxlog-xchg utility as described in the documentation.
Where can I find this utility ? I use the community edition.
Thanks.
james811 created
om_http bulk / batch output to Splunk
woodsb02 created
Hi everyone,
The user guide section for the om_http module explains:
Each event is transferred in a single POST request.
https://nxlog.co/documentation/nxlog-user-guide#om_http
When https is used, is the TLS handshake also performed for each event? This seems like it would be quite inefficient.
Are there any plans to support bulk / batch exporting of many events in a single http post, specifically for exporting data to the Splunk HEC?
rsyslog and syslog-ng support batch exporting events to Splunk using the external python script omsplunkhec.py, as explained here:
https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splunk.html
NXLog supports batch exporting events to Elasticsearch using their bulk API:
https://nxlog.co/documentation/nxlog-user-guide#om_elasticsearch
woodsb02 created
nxlog-ce om_http missing add_http_header() procedure
woodsb02 created
With reference to the following guide for sending NXLog output to Splunk using HEC:
https://nxlog.co/documentation/nxlog-user-guide#splunk-hec-collection
The NXLog community edition om_http is missing the add_http_header() procedure:
https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#om_http
This means it is not possible to output nxlog-ce data to http where a header needs to be added to the http post, such as the Splunk HEC:
https://docs.splunk.com/Documentation/Splunk/7.0.3/Data/HECWalkthrough
Is this functionality deliberately missing from the community edition, or can it please be added?
woodsb02 created
nxlog source IP addresses
rbeeler created
This is a two part problem relating to source IP addresses used by nxlog:
We have several systems that use 192.168.x.x addresses as their source - not sure what this is about, but I'm thinking it may be related to the nxlog service starting before the Windows network services are completely up?
We have several systems that have multiple IP addresses (interfaces) and we would like to force nxlog to report from one IP address. Is there a way to do this?
Thanks for your help.
rbeeler created
nxlog use high cpu on AIX system
wenxiaoming created
nxlog use high cpu on AIX system,bug low cpu use percent on Linux system. So why?
wenxiaoming created
nxlog error on windows
aries135 created
On first load nxlog seems to run just fine. After a new reload I get this error in the logs.
"nxlog failed to start: Failed to load module from C:/msys/1.0\extension\xm_gelf.dll, The specified module could not be found.
The specified module could not be found."
Any idea how to fix this? Never ran into this before and used nxlog in many other windows machines without a problem. Reinstalling hasn't fixed it.
aries135 created
Why is Nxlog not reading the full line from Cerberus FTP log?
matsovef created
Hello
I've defined this as an input file in nxlog.conf
<Input cerberus_log>
Module im_file
File "C:\ProgramData\Cerberus LLC\Cerberus FTP Server\log\server.1.log"
InputType LineBased
PollInterval 5
<Exec>
log_info("Msg <" + $raw_event + ">");
$date = substr($raw_event,1,10);
$time = substr($raw_event,12,8);
$cmd = substr($raw_event,23,7);
$pid = substr($raw_event,32,6);
$action = substr($raw_event,42);
$Hostname = hostname_fqdn();
$SourceName = "Cerberus FTP Server";
$ProcessID = $pid;
$EventTime = parsedate($date + " " + $time);
$Message = $cmd + ": " + $action;
$SyslogSeverityValue = 6;
$SyslogFacilityValue = 11;
</Exec>
</Input>
When this input is read from the log file
[2018-03-28 09:21:48]: REPLY [ 5445] - 234 Authentication method accepted
[2018-03-28 09:21:48]:CONNECT [ 5445] - SSL connection using TLSv1.2 (ECDHE-RSA-AES256-GCM-SHA384), 256 bit encryption
[2018-03-28 09:21:48]:CONNECT [ 5445] - SSL connection established
[2018-03-28 09:21:48]:COMMAND [ 5445] - USER PandoraManuellt
[2018-03-28 09:21:48]: REPLY [ 5445] - 331 User PandoraManuellt, password please
[2018-03-28 09:21:48]:COMMAND [ 5445] - PASS ***********
the following is logged in nxlog.log
2018-03-28 09:21:51 INFO Msg <[>
2018-03-28 09:21:51 INFO Msg <>
2018-03-28 09:21:51 INFO Msg <[>
2018-03-28 09:21:51 INFO last message repeated 3 times
2018-03-28 09:21:51 INFO Msg <>
2018-03-28 09:21:51 INFO Msg <[>
Empty lines are empty, but when a line that starts with a '[' then $raw_event only contains that character and nothing else. Why?
Mats-Ove
matsovef created
NxLog multiline use a blank line as a headerline
Deleted user created
Is there a way to state a headerline to be blank while using multiline module?
My data shows as:
data
data
(blank)
data
data
(blank)
Thanks
Deleted user created
Server 2016 configuration
dbreise created
I've been using NXLOG for quite some time now all of a sudden on server 2016 it's not working. Please provide the necessary config for server 2016 as im_msvistalog is returning errors:
<11>Mar 23 10:33:28 onsitephysio-welsh MSWinEventLog 4 N/A N/A Fri Mar 23 10:33:28 2018 N/A nxlog-ce N/A N/A N/A onsitephysio-welsh N/A EvtNext failed with error 13: The data is invalid. N/A
<11>Mar 23 10:33:28 onsitephysio-welsh MSWinEventLog 4 N/A N/A Fri Mar 23 10:33:28 2018 N/A nxlog-ce N/A N/A N/A onsitephysio-welsh N/A ### ASSERTION FAILED at line 319 in im_msvistalog.c/im_msvistalog_event_to_logdata(): "event != NULL" ### N/A
dbreise created
Adding additional information into message for im_msvistalog
matsovef created
Hello,
I'm new to nxlog and tries to integrate it with GrayLog. It's working fine but to filter data from certain Windows Event Logs into a separate stream in GrayLog I need to insert the Channel value (available in im_msvistalog) into the message text. How can this be done?
Mats-Ove
matsovef created
Bit-wise arithmetic
michaelkendall created
Does anyone know how to do bit-wise arithmetic in NxLog? I am attempt to decode New and Old UAC Values from the Windows message id 4720. For example, the Old UAC Value is 0x1 and the New UAC Value is 0x21.
michaelkendall created
How to parse incoming syslog messages for specific string and write to separate file
bluelotus created
I have nxlog configured to capture syslog messages and write them to a file and all is fine but now I would like the ability to parse for a specific string in the syslog message being received and then write those syslog messages to a separate file (nxdomain.log). For example, when receiving syslog messages that contain the string "NXDOMAIN", I want to then write that specific syslog message to a separate file just for those cases so I can track those specific messages and not have to later parse the daily log (log.txt) file.
How would I modify this configuration file to do just that?
define ROOT /usr/bin
define FILENAME /logs/log.txt
Moduledir /usr/libexec/nxlog/modules
CacheDir %ROOT%/data
Pidfile %ROOT%/data/nxlog.pid
SpoolDir /var/spool/nxlog
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
<Extension xm_exec>
Module xm_exec
</Extension>
<Extension xm_fileop>
Module xm_fileop
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_tcp
Host 0.0.0.0
Port 514
Exec parse_syslog_bsd() ;
</Input>
<Output out>
Module om_file
File "%FILENAME%"
<Schedule>
When @daily
Exec file_rename ("%FILENAME%","%FILENAME%"+'.'+strftime(now(),"%Y%m%d"));
out->reopen();
</Schedule>
</Output>
<Route 1>
Path in => out
bluelotus created
nxlog windows packageing
caibaoying created
I want to use source_code to product a msi package for windows. I run ./pkgmsi.sh , It's error
./pkgmsi.sh: line 1: c:\Program Files (x86)\Windows Installer XML v3.5\bin\candle.exe: command not found
How can I package msi correctly???
if you help me, I will be grateful for you.
caibaoying created
Send File by Syslog w/o header
_omar_ created
Hello
I'm using "NXLog CE" for send a file by syslog. So far I got this:
<Input filezilla>
Module im_file
File "C:\\Program Files (x86)\\FileZilla Server\\Logs\\fzs-*.log"
SavePos TRUE
Exec $Message = '%FileZilla: ' + $raw_event;
</Input>
<Output outfiles>
Module om_tcp
Host 10.226.6.210
Port 514
Exec to_syslog_bsd();
</Output>
The file has lines like "(125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT"
I want to get (on my syslog sever) "%FileZilla: (125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT"
Instead, I got: "Mar 13 12:33:00 DSCRESJ %FileZilla: (125419) 13/03/2018 12:32:59 - prtg (10.4.171.245)> QUIT"
I think the "Mar 13 12:33:00 DSCRESJ" part is the header from to_syslog_bsd(). Anyway, I wonder If there a way to remove that header.
Regards.
_omar_ created
om_kafka - ERROR Unable to produce message
bourazaniss created
I am trying to evaluate om_kafka module on RHEL system.
It starts ok and sends some messages from files to kafka but after a while these messages appear:
2018-03-13 14:52:52 ERROR Unable to produce message
2018-03-13 14:52:55 ERROR last message repeated 183044 times
Sometimes it can be fixed with NXLog service restart, while others not.
nxlog-3.99.3332_RHEL7_x86_64_trial
Any suggestion?
bourazaniss created
NXLOG extracted the fields from any message
mammari created
Hello there,
I'm using NXLOG Community Edition, and I want to inquire about how can I just forward the events log from Windows OS without modified the original log's contents. Because, the NXLOG extracted the fields from any message. So, is there a way to avoid the extraction?
Thanks and Regard.
mammari created