Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Duplicates 4624 or other Event Viewer Log
IB_179669
Hi, I'm using NXLOG Community to transfer logs in and out, from Windows clients to a Linux server with an NXLOG agent for log collection.
The logs arrive correctly, the only thing is that for the event viewer (example) 4624, I see logs with the same time, even in milliseconds, but the message varies only for a few lines of the "message" field.
I wanted to avoid using the repeat module because I would create the same a log recording increasing the database.
This is client configuration file :
define ROOT C:\Program Files\nxlog
define ROOT_STRING C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
define MonitoredEventIds 4624, 4647
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id='0'>
<Select Path="Security">*</Select>
</Query>
</QueryList>
</QueryXML>
Exec if $EventID NOT IN (%MonitoredEventIds%) drop();
Exec if $TargetUserName == "SYSTEM" drop();
Exec if $TargetUserName =~ /$/ drop();
Exec if $TargetUserName =~ /UMFD/ drop();
Exec if $TargetUserName =~ /DWM/ drop();
Exec if $LogonType == "5" drop();
</Input>
<Output out>
Module om_tcp
Host (10.*****)
Port 1514
Exec to_json();
</Output>
<Route eventlog_to_out>
this is server nxlog.conf
User nxlog
Group nxlog
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
########################################
# Modules #
########################################
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in1>
Module im_tcp
Host 0.0.0.0
Port 1514
<Exec>
parse_json();
</Exec>
</Input>
<Output dbi>
Module om_dbi
SQL INSERT INTO SystemEvents (ReceivedAt, DeviceReportedTime, EventID, EventUser, EventSource, EventLogType, FromHost, NTSeverity, Priority, Message)
VALUES (NOW(), NOW(), $EventID, $TargetUserName, $SourceName, $EventType, $Hostname, $Severity, $SeverityValue, $Message )
Driver mysql
Option host 127.0.0.1
Option username ****
Option password *******
Option dbname Syslog
</Output>
########################################
# Routes #
########################################
<Route 1>
Path in1 => dbi
</Route>
it's possible to not register duplicates at the origin?
it's possible to delete duplicates also in mysql database?
thanks you
IB_179669
sending txt logs to a remote server using snare do not show logs content
eebs
#Hello, tanks in advance .
#I am sending multiple logs from windows server to a linux collector
#I have no issues with windows system logs ,
#Seems i can not send via snare windows system logs, and test plain text logs.
#Is there any way to do that?
But when i switch to snare i can see no description about the warning
2022-10-21T09:21:21+00:00 Winserver MSWinEventLog#0111#011N/A#0111#011Fri Oct 21 09:21:21 2022#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011#011N/A#011N/A#015
The same line with snare commented:
2022-10-21T09:18:23.208210+00:00 Winserver WARNING: Can't open file \?\C:...\UPPS\UPPS.BIN: Permission denied#015
#My config:
Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\App\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Input internal>
Module im_internal
Exec $Hostname = hostname_fqdn();
</Input>
<Input vistalog>
Module im_msvistalog
</Input>
<Input eventlog>
Module im_mseventlog
</Input>
<Input testFile>
Module im_file
SavePos True
RenameCheck True
Recursive True
PollInterval 0.5 #near real time
File "C:\test\myfile.txt"
ReadFromLast True
</Input>
<Output out>
Module om_tcp
Host linux
Port 514
#Exec to_syslog_snare();
</Output>
<Route r>
Path internal, eventlog, vistalog, testFile => out
Path testFile => out
</Route>
eebs
replace using regex
Sajeshvv23
Hi,
i have below replace function for replacing "|0" to "Zero".
Exec $Message = replace($Message, "|0 ", "Zero");
now, i want to replace "|0" to "|15" with Zero.
do i need to add 15 more Exec replace function, or is there is any way to replace using single exec using regex?
Sajeshvv23
jQuery vulnerability in nxlog-manager-5.5.5398
sgiliyal
Greetings!
I have a vulnerability in jQuery to be addressed in the nxlog manager v5 (5.5.5398).
This is related to XSS vulnerabilities from the version of jQuery installed (1.8.3, 1.9.2 ui) .
Will upgrading Manager to 5.6.5633 resolve the issue and update jQuery to 3.5.0+ ?
If not, please can you provide steps to update jQuery manually?
Thanks!
Shyam (on behalf of Shashidhar Ghiliyal)
sgiliyal
NXlog EE 5.6.7727 im_odcb module buggy when FreeTDS used on Debian11
ppum
Hi all,
I think there might be a bug in the im_odbc module in the Linux (Debian11) im_odbc module for NXLog 5.6.7727 EE when used with the FreeTDS driver (via unixODBC) - so for example if you try to connect to MS SQL or Sybase databases.
Although all the drivers etc. were configured correctly (it was basically a copy from a system where it used to work) and connecting to the databases via ISQL worked fine, the Agent wasn't able to fetch logs and produced following errors in the nxlog.log:
INFO [im_odbc|inputxy] im_odbc successfully connected to the database
WARNING [im_odbc|inputxy] im_odbc detected a disconnection, attempting to reconnect in 1200 seconds
ERROR [im_odbc|inputxy] SQLDescribeParam failed, IM001:2:0:[unixODBC][Driver Manager]Driver does not support this function (odbc error code: -1)
I've tested multiple FreeTDS driver versions with multiple compile options - but none of them worked.
Finally I figured to try a different NXLog version, so I downloaded the NXLog Agent EE 4.10.5000 for debian10 and it worked out of the box with the standard FreeTDS driver from the Debian11 repo (v1.2.3) and unixODBC (2.3.6).
So my conclusion is that there is a bug in the 5.6 EE NXLog Agent Version regarding the im_odbc module when used with FreeTDS (at least for Debian 11).
So while using 4.10 as a fallback is nice, it would be great to use the newer 5.x versions in the future.
BR Reinhard
ppum
nxlog-ce-3.0.2284 crashing randomly after upgrading from 2.10.2150
rdefulio
Hello,
I am having stability issues with the newest nxlog-ce release 3.0.2284. I have been using nxlog-ce-2.10.2150 for several years and it has been very stable in our environment with few issues. I use papertrail for log collection, and I have a highly customized configuration file. I did a test roll-out of 3.0.2284 to a few servers and did not notice any issues at first. However after rolling out the update to approximately 40 servers it started crashing randomly in ntdll.dll causing the nxlog service to stop and re-start itself. There was no rhyme or reason to it. It would work fine for 15 minutes and then suddenly I would start getting multiple random crashes and service restarts which would then crush papertrail with the previous 30 days of event log history (per crashed server) and my papertrail storage utilization doubled my average daily usage in a few hours. I had to roll-back all the servers back to 2.10.2150 to stop the bleeding. The servers used in the test were a variety of Windows 2012r2, 2016, 2019, and 2022 servers. By far the majority of the servers are running server 2016. Some are hyper-v hosts running on bare metal, others are virtual machines that run on those hyper-v hosts.
My papertrail logs are full of these errors, but here is a small sampling. These are server 2019 and 2022 respectively.
Oct 10 21:31:21 hv-host19-f4 Application-Error
{
"Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:libssl-1_1-x64.dll, version:1.1.1.13, time stamp:0x00000000|Exception code:0xc0000005|Fault offset:0x0000000000021b97|Faulting process id:0xf5d0|Faulting application start time:0x01d8dd0fc8bd493e|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:c:\\apps\\nxlog\\libssl-1_1-x64.dll|Report Id:08b0fd02-0e57-44b0-81fc-b1e7fb47f472|Faulting package full name:|Faulting package-relative application ID:",
"Hostname": "hv-host19-f4",
"EventType": "ERROR",
"SeverityValue": 4,
"Severity": "ERROR",
"EventID": 1000,
"SourceName": "Application-Error",
"Task": 100,
"RecordNumber": 12236,
"ProcessID": 0,
"ThreadID": 0,
"Channel": "Application",
"EventTime": "2022-10-10 21:23:29",
"Category": "Application Crashing Events",
"Opcode": "Info"
}
Oct 10 21:52:21 ws-ops22-2 nxlog-ce nxlog-ce-3.0.2284 startup profile 2022.01.25 (DEFAULT)
Oct 10 21:52:22 ws-ops22-2 nxlog-ce connecting to logs99.papertrailapp.com:12345
Oct 10 21:52:22 ws-ops22-2 nxlog-ce successfully connected to logs99.papertrailapp.com:12345
Oct 10 21:52:23 ws-ops22-2 Application-Error
{
"Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:ntdll.dll, version:10.0.20348.803, time stamp:0xbee6f04c|Exception code:0xc0000374|Fault offset:0x00000000001044a9|Faulting process id:0x26f4|Faulting application start time:0x01d8ca290a171970|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:C:\\WINDOWS\\SYSTEM32\\ntdll.dll|Report Id:5e5549b8-3c8b-405a-a78f-fd4c1f296a40|Faulting package full name:|Faulting package-relative application ID:",
"Hostname": "ws-ops22-2",
"EventType": "ERROR",
"SeverityValue": 4,
"Severity": "ERROR",
"EventID": 1000,
"SourceName": "Application-Error",
"Version": 0,
"Task": 100,
"OpcodeValue": 0,
"RecordNumber": 1100,
"ProcessID": 0,
"ThreadID": 0,
"Channel": "Application",
"EventTime": "2022-09-16 21:30:36",
"Category": "Application Crashing Events",
"Opcode": "Info"
}
Oct 10 21:54:35 ws-ops22-2 Application-Error
{
"Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:libcrypto-1_1-x64.dll, version:1.1.1.13, time stamp:0x00000000|Exception code:0xc0000005|Fault offset:0x00000000001ba014|Faulting process id:0x30e8|Faulting application start time:0x01d8caef9eba1346|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:c:\\apps\\nxlog\\libcrypto-1_1-x64.dll|Report Id:12281218-b154-47ae-a426-1495de2adf0d|Faulting package full name:|Faulting package-relative application ID:",
"Hostname": "ws-ops22-2",
"EventType": "ERROR",
"SeverityValue": 4,
"Severity": "ERROR",
"EventID": 1000,
"SourceName": "Application-Error",
"Version": 0,
"Task": 100,
"OpcodeValue": 0,
"RecordNumber": 1304,
"ProcessID": 0,
"ThreadID": 0,
"Channel": "Application",
"EventTime": "2022-09-17 20:00:44",
"Category": "Application Crashing Events",
"Opcode": "Info"
}
The crashing seems to indicate an issue with TLS or crypto but my existing papertrail configuration has been working fine for literally years.
Another issue I ran into, while removing nxlog-ce-3.0.2284 is that issuing a stop-service to command to the service returns "The pipe has been ended" error instead of a normal service shutdown gracefully message. This happened every time I tried to stop the service. The service did stop, but given the error I don't know if it was a graceful stop or if it was a hard stop that ends up causing the eventlog to be re-uploaded in its entirety when the service started again. I had a lot of that going on so I can't say for sure if it happened or not.
[SC] ControlService FAILED 109:
The pipe has been ended.
Finally here is a snippet of the bottom of my nxlog.conf file where I set up the connection to papertrail. I've changed the host parameters slightly for security.
<Route nxlog>
Path from_nxlog => to_papertrail
</Route>
<Route eventlogs>
Path from_eventlog => noisefilter => cleanup => reorder => jsonify => to_papertrail
</Route>
<Route c_logs>
Path from_c_logs => to_papertrail
</Route>
<Output to_papertrail>
Module om_ssl
Host logs99.papertrailapp.com
Port 12345
CAFile %ROOT%/cert/papertrail-bundle.pem
AllowUntrusted FALSE
# Convert to syslog format
Exec to_syslog_bsd();
</Output>
I'm considering pushing the logs to a local linux server with om_udp and let that server relay the logs to papertrail over TLS to workaround the issue but that adds extra complexity to the environment that I would rather not have to support.
Thanks
Ron
rdefulio
Split json record in multiple records?
joost.bijl
Hi,
I'm experimenting with reading from an Azure eventhub with im_kafka. The eventhub receives security data from various security related azure components.
The im_kafka module works great after i found out that the username should be $connectionstring ;).
The output of the eventhub is a json dict with an array, like this: { records: [ {id: 1, msg: "xyz", etc},{id: 2, msg: "abc", etc}]}.
I tried to use extract_json("$.records") but that does not iterate over the array.
I also made a python script that writes the logs to a file, one line at a time:
from confluent_kafka import Consumer
import json
c = Consumer({....})
while True:
msg = c.poll(1.0)
eventhub_records = json.loads(msg.value())
for record in eventhub_records['records']:
print(json.dumps(record))
This works great, but i'd like to have something like this in nxlog. Can this be done, or does nxlog not support to split a single record into multiple records?
Thanks!
joost.bijl
Add IP Address (Community Edition)
JDiaz
Hi!
Newbie on NXlog here.
I'm trying to add the IP Address to the logs that I'm sending but I couldn't found any funcionality to do this (at least on the Community Edition).
Is there any way to do that with the CE? Something like $ip = whats_my_ip?() ?
Thanks a lot!
JDiaz
ONEOF block
Petex
hi I have an application that writes many log files to folder on my windows server. Each log only contain one or two lines.
Previously the files were deleted by another system but that is now decommissioned.
I wanted to use NXLOG to delete the files once it has treated its contents(nothing else will be added after)
looking in the community edition docs I can use an ONEOF block with a remove.
The config is parsed correctly and the logs are sent to my graylog server, however the logs are never removed.
I saw some posts that suggested this block only works with the paid version but its strange that it is documented in the community docs?
Can any one tell me where i am going wrong ou another way of doing this
This is an extract from my config
version nxlog-ce-3.0.2284
<Extension _fileop>
Module xm_fileop
</Extension>
<Extension csv1>
Module xm_csv
Fields $Method,$Host,$Application,$EventID,$Severity,$Date,$Time,$Description,$Value
Delimiter |
</Extension>
<Input hyp-in>
Module im_file
SavePos TRUE
ReadFromLast FALSE
ActiveFiles 20
CloseWhenIdle TRUE
File "C:\HYPERVISION\logFiles*.*"
<Exec>
csv1->parse_csv();
$InputFileName1 = file_name();
$Message = $raw_event;
</Exec>
<OnEOF>
Exec file_remove(file_name(), now() - 6000);
</OnEOF>
</Input>
Petex
pm_buffer not filling on network error
dmuensterer
Hi,
I have the requirement to buffer logs that were unable to be sent during network failures.
I have the following config which makes NXLog CE create a buffer file but it stays at 0KB size with a missing network.
My logs show the expected network failure:
2022-10-04 10:01:52 INFO nxlog-ce-3.0.2284 started
2022-10-04 10:01:52 INFO reconnecting in 1 seconds
2022-10-04 10:01:52 ERROR apr_sockaddr_info failed for myserver.com:1514; No such host is known.
2022-10-04 10:01:53 INFO reconnecting in 2 seconds
2022-10-04 10:01:53 ERROR apr_sockaddr_info failed for myserver.com:1514; No such host is known.
2022-10-04 10:01:55 INFO reconnecting in 4 seconds
nxlog.conf
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input inWindowsAudit>
Module im_msvistalog
ReadFromLast True
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
</Query>
</QueryList>
</QueryXML>
Exec parse_syslog(); to_json();
</Input>
<Processor buffer>
Module pm_buffer
Type Disk
# 40 MiB buffer
MaxSize 40960
# Generate warning message at 20 MiB
WarnLimit 20480
</Processor>
<Output ssl>
Module om_ssl
Host myserver.com
Port 1514
CAFile %CERTDIR%\ca-root.cer
CertFile %CERTDIR%\client.cer
CertKeyFile %CERTDIR%\client_private.key
KeyPass secret
AllowUntrusted FALSE
</Output>
<Route main>
Path inWindowsAudit => buffer => ssl
</Route>
Any ideas? Thanks.
dmuensterer
Splunk HF and NXlog via SSL
nktj
Did anyone manage to send logs from Splunk HF to NXlog server with SSL mutual authentication enabled? Able to share your configuration? Received an error on SSL version. Not sure it is due to Splunk HF conf or NXlog. Thanks.
nktj
NXLog fail to start on macOS
ygini
Hello
I'm using NXLog 5.6.7727 version for Apple Silicon
When I start the service as root manually with the CLI /opt/nxlog/bin/nxlog -f the service works and my logs are sent.
The same settings started with launchd works but don't send anything.
Any ideas why?
Here is my current configuration:
User nxlog
Group nxlog
Panic Soft
# default values:
define INSTALLDIR /opt/nxlog
PidFile %INSTALLDIR%/var/run/nxlog/nxlog.pid
CacheDir %INSTALLDIR%/var/spool/nxlog
ModuleDir %INSTALLDIR%/libexec/nxlog/modules
SpoolDir %INSTALLDIR%/var/spool/nxlog
define CERTDIR %INSTALLDIR%/var/lib/nxlog/cert
define CONFDIR %INSTALLDIR%/etc/nxlog.d
# Note that these two lines define constants only; the log file location
# is ultimately set by the `LogFile` directive (see below). The
# `MYLOGFILE` define is also used to rotate the log file automatically
# (see the `_fileop` block).
define LOGDIR %INSTALLDIR%/var/log/nxlog
define MYLOGFILE %LOGDIR%/nxlog.log
<Extension _json>
Module xm_json
</Extension>
<Extension gelf>
Module xm_gelf
</Extension>
<Input macos_es>
Module im_maces
NotifyEvents All
<Exec>
$Hostname = hostname();
to_json();
</Exec>
</Input>
<Input macos_uls>
Module im_maculs
UUIDTextPath "/var/db/uuidtext"
<Exec>
if $subsystem == 'com.apple.SkyLight'
{
drop();
}
$Hostname = hostname();
to_json();
</Exec>
</Input>
<Output graylog>
Module om_tcp
Host 10.0.0.10:12201
OutputType GELF_TCP
</Output>
<Route Mac2Graylog>
Path macos_uls, macos_es => graylog
</Route>
ygini
Adding field as source IP gateway address
JustasS
Hello,
I need to add extra field to send External/gateway IP address to Graylog.
Using output UDP:
<Output out>
Module om_udp
Host ***
Port 4514
OutputType GELF
Exec $hostname = 'test_SRV';
</Output>
Its even posible?
JustasS
xml log parsing on windows with community edition won't get extract_xml recognized.
har3005
I'm trying to ws_ftp logs that are xml formatted as such :
<?xml version="1.0" encoding="utf-8" ?>
<log>
<entry>
<log_time> xxxx </log_time>
<description>data <![CDATA[Connection established]]></description>
<service>data </service>
<sessionid> xxxx </sessionid>
<type>0</type> <severity>0</severity>
<lstnconnaddr> ip </lstnconnaddr>
<cliconnaddr>ip </cliconnaddr>
<sguid> xxx </sguid>
</entry>
...
with this configuration :
define ROOT C:\Program Files\nxlog
<Extension gelf>
Module xm_gelf
</Extension>
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension multiline>
Module xm_multiline
HeaderLine /^<event>/
EndLine /^</event>/
</Extension>
<Extension xmlparser>
Module xm_xml
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in>
Module im_file
File "PATH TO .XML"
SavePos FALSE
PollInterval 1
ReadFromLast FALSE
InputType multiline
<Exec>
$EventData = extract_xml("/log/entry");
if $EventData == ""
{
delete($EventData);
}
to_json();
</Exec>
</Input>
<Output out>
Module om_file
File "C:\\temp\\output_test.txt"
</Output>
<Route 1>
Path in => out
</Route>
i can't get past this error :
ERROR Couldn't parse Exec block at C:\Program Files\nxlog\conf\nxlog.conf:34; couldn't parse statement at line 35, character 40 in C:\Program Files\nxlog\conf\nxlog.conf; function 'extract_xml()' does not exist or takes different arguments
ERROR module 'in' has configuration errors, not adding to route '1' at C:\Program Files\nxlog\conf\nxlog.conf:51
how can i get further?
har3005
Link to download older version of trial installer
bryan.tabb.secops
Hi
Ive got a customer with windows server core (no gui) and they are wanting to uninstall the trial version and have asked if we can give them a copy of the original msi.
They are after version 5.4.7313
I've had a look at the download section and it isn't there.
Any suggestions please?
Thanks
Bryan
bryan.tabb.secops
Is it possible to use a variable in a regex?
opoplawski
Is it possible to use a variable in a regex? I'm trying to do something like the following:
Exec if ($EventID == 4104) {
if defined(get_var('scriptblockid')) {
$id = get_var('scriptblockid');
if ($Message =~ /ScriptBlock ID: $id/) drop();
}
if ($Message =~ /ClassName = 'Root\/Microsoft\/Windows/) {
if ($Message =~/ScriptBlock ID: (\S+)/) {
set_var('scriptblockid', $1);
}
drop();
}
}
opoplawski
Agent not showing as online in manager
nervevector
Hello,
We've deployed NXLog EE agents on numerous Windows servers, and we're facing a strange issue with just one agent.
It does not, and we're unable to get it to, show as online within the NXLog Manager. The agent's log states that it was able to successfully connect to the manager, but it's still showing as offline in the UI.
I have tried the following:
Upgraded agent to the latest version
Verified that the agent's logs said it was able to connect to the manager (10.40.1.90)
Removed the agent from the manager interface
Removed the agent off of the server
Reinstalled the agent on the server, and reconfigured the certificate and managed.conf file
Verified that it connected to the manager in the agent's logs
And even after all of that, the agent is still showing as offline on the manager's UI.
2022-09-23 16:22:40 WARNING [CORE|main] no functional input modules!
2022-09-23 16:22:40 INFO [CORE|main] nxlog-5.6.7727 (50b2a4353@REL_v5.6) started on Windows
2022-09-23 16:22:40 INFO [xm_admin|agent_management] connecting to 192.168.1.1:4041
2022-09-23 16:23:01 ERROR [xm_admin|agent_management] couldn't connect to 192.168.1.1:4041; A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
2022-09-23 16:23:01 INFO [xm_admin|agent_management] reconnecting to 192.168.1.1:4041 in 1 sec
2022-09-23 16:23:02 INFO [xm_admin|agent_management] connecting to 192.168.1.1:4041
2022-09-23 16:23:21 WARNING [CORE|main] stopping nxlog service
2022-09-23 16:23:21 WARNING [CORE|main] nxlog received a termination request signal, exiting...
2022-09-23 16:23:23 ERROR [xm_admin|agent_management] couldn't connect to 192.168.1.1:4041; A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
2022-09-23 16:23:23 INFO [xm_admin|agent_management] reconnecting to 192.168.1.1:4041 in 2 sec
2022-09-23 16:23:24 WARNING [CORE|main] no functional input modules!
2022-09-23 16:23:24 INFO [CORE|main] nxlog-5.6.7727 (50b2a4353@REL_v5.6) started on Windows
2022-09-23 16:23:24 INFO [xm_admin|agent_management] connecting to 10.40.1.90:4041 <==== Here I set the manager's correct IP in managed.conf
2022-09-23 16:23:24 INFO [xm_admin|agent_management] tcp connection established with 10.40.1.90:4041
2022-09-23 16:28:24 WARNING [xm_admin|agent_management] did not receive requests from agent manager in the past 300 seconds, disconnecting
2022-09-23 16:28:24 INFO [xm_admin|agent_management] reconnecting to 10.40.1.90:4041 (last connection attempt was 300 sec ago)
2022-09-23 16:28:24 INFO [xm_admin|agent_management] connecting to 10.40.1.90:4041
2022-09-23 16:28:24 INFO [xm_admin|agent_management] tcp connection established with 10.40.1.90:4041 <==== Here it says it was able to connect to the manager just fine.
Does anyone have an idea as to what's going on here?
Any help would be greatly appreciated!
Thanks!
nervevector
im_vistalog parser failure. Introducing \n characters in syslog output that split a single record in two records...
Ak0
Hello,
I am experiencing truncated syslog messages, and found the root cause. Event are not correctly parsed such as this example:
Here is the original message read from eventviewer:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-a5ba-3e3b0328c30d}" />
<EventID>4663</EventID>
<Version>1</Version>
<Level>0</Level>
<Task>12812</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2022-09-22T13:19:53.982486800Z" />
<EventRecordID>5610</EventRecordID>
<Correlation />
<Execution ProcessID="4" ThreadID="2724" />
<Channel>Security</Channel>
<Computer>PC-1304.domain.priv</Computer>
<Security />
</System>
<EventData>
<Data Name="SubjectUserSid">S-1-5-21-894803723-839211917-112803419-1225</Data>
<Data Name="SubjectUserName">username</Data>
<Data Name="SubjectDomainName">Domain</Data>
<Data Name="SubjectLogonId">0x71979</Data>
<Data Name="ObjectServer">Security</Data>
<Data Name="ObjectType">File</Data>
<Data Name="ObjectName">\Device\HarddiskVolume5</Data>
<Data Name="HandleId">0x1054</Data>
<Data Name="AccessList">%%4416</Data>
<Data Name="AccessMask">0x1</Data>
<Data Name="ProcessId">0x484</Data>
<Data Name="ProcessName">C:\Windows\System32\mstsc.exe</Data>
<Data Name="ResourceAttributes" />
</EventData>
<RenderingInfo Culture="fr-FR">
<Message>Une tentative d’accès à un objet a été effectuée. Sujet : ID de sécurité : S-1-5-21-894803723-839211917-112803419-1225 Nom du compte : username Domaine du compte : Domain ID d’ouverture de session : 0x71979 Objet : Serveur de l’objet : Security Type d’objet : File Nom de l’objet : \Device\HarddiskVolume5\ ID du handle : 0x1054 Attributs de ressource : Informations sur le processus : ID du processus : 0x484 Nom du processus : C:\Windows\System32\mstsc.exe Informations sur la demande d’accès : Accès : Lecture données (ou liste de répertoire) Masque d’accès : 0x1</Message>
<Level>Information</Level>
<Task>Removable Storage</Task>
<Opcode>Informations</Opcode>
<Channel>Sécurité</Channel>
<Provider>Microsoft Windows security auditing.</Provider>
<Keywords>
<Keyword>Succès de l’audit</Keyword>
</Keywords>
</RenderingInfo>
</Event>
Here's the corresponding syslog line produced by om_syslog.
Please note that this line is spannig more than one line, this is the problem, read more please. (Forum post format may alter the rendering...)
<14>1 2022-09-22T15:19:53.982486+02:00 PC-1304.domain.priv Microsoft-Windows-Security-Auditing 4 - [NXLOG@14506 Keywords="-9214364837600034816" EventType="AUDIT_SUCCESS" EventID="4663" ProviderGuid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Version="1" Task="12812" OpcodeValue="0" RecordNumber="5610" ThreadID="2724" Channel="Security" Category="Removable Storage" Opcode="Informations" SubjectUserSid="S-1-5-21-894803723-839211917-112803419-1225" SubjectUserName="username" SubjectDomainName="domain" SubjectLogonId="0x71979" ObjectServer="Security" ObjectType="File" ObjectName="\Device\HarddiskVolume5\" HandleId="0x1054" AccessList="%%4416
" AccessMask="0x1" ProcessName="C:\Windows\System32\mstsc.exe" EventReceivedTime="2022-09-22 15:20:26" SourceModuleName="eventlog" SourceModuleType="im_msvistalog"] <Event><EventTime>2022-09-22 15:19:53</EventTime><Hostname>PC-1304.domain.priv</Hostname><Keywords>-9214364837600034816</Keywords><EventType>AUDIT_SUCCESS</EventType><SeverityValue>2</SeverityValue><Severity>INFO</Severity><EventID>4663</EventID><SourceName>Microsoft-Windows-Security-Auditing</SourceName><ProviderGuid>{54849625-5478-4994-A5BA-3E3B0328C30D}</ProviderGuid><Version>1</Version><Task>12812</Task><OpcodeValue>0</OpcodeValue><RecordNumber>5610</RecordNumber><ProcessID>4</ProcessID><ThreadID>2724</ThreadID><Channel>Security</Channel><Message>Une tentative d’accès à un objet a été effectuée.

Sujet :
 ID de sécurité : S-1-5-21-894803723-839211917-112803419-1225
 Nom du compte : username
 Domaine du compte : domain
 ID d’ouverture de session : 0x71979

Objet :
 Serveur de l’objet : Security
 Type d’objet : File
 Nom de l’objet : \Device\HarddiskVolume5&#xD;
 ID du handle : 0x1054
 Attributs de ressource : 

Informations sur le processus :
 ID du processus : 0x484
 Nom du processus : C:\Windows\System32\mstsc.exe

Informations sur la demande d’accès :
 Accès : Lecture données (ou liste de répertoire)
 
 Masque d’accès : 0x1</Message><Category>Removable Storage</Category><Opcode>Informations</Opcode><SubjectUserSid>S-1-5-21-894803723-839211917-112803419-1225</SubjectUserSid><SubjectUserName>username</SubjectUserName><SubjectDomainName>Domain</SubjectDomainName><SubjectLogonId>0x71979</SubjectLogonId><ObjectServer>Security</ObjectServer><ObjectType>File</ObjectType><ObjectName>\Device\HarddiskVolume5</ObjectName><HandleId>0x1054</HandleId><AccessList>%%4416
 </AccessList><AccessMask>0x1</AccessMask><ProcessName>C:\Windows\System32\mstsc.exe</ProcessName><EventReceivedTime>2022-09-22 15:20:26</EventReceivedTime><SourceModuleName>eventlog</SourceModuleName><SourceModuleType>im_msvistalog</SourceModuleType></Event>
Problem is the AccessList value (read from the syslog message):
HandleId="0x1054" AccessList="%%4416
" AccessMask="0x1" ProcessName="C:\Windows\System32\mstsc.exe"
If we comparing XML and syslog output:
XML output
<Data Name="AccessList">%%4416</Data>
syslog output
AccessList="%%4416
"
^^^^^^^^^^^^^^^^ Mind those characters !
AccessList="%%4416 <---- a new line character is invisible but here (this is THE bug, splitting the syslog message)
" <----- multiple tab characters are invisible but here (because of HTML)
End of line \x0a is record separator in syslog format so the line is split in two syslog records (of course the second record is invalid)
It seems that the content of AccessList Key Value is not extracted from XML but from the Message.
Looking at the <Message> we see AccessList is incorrect:
<AccessList>%%4416
 </AccessList>
where AccessList in XML is:
<Data Name="AccessList">%%4416</Data>
Conclusion there is a bug in the event parser that do not escape "#xD;
 " characters correctly, decoding the #xA; in \x0a cutting the syslog message...
You can easily reproduce the problem using this configuration:
<Input eventlog>
Module im_msvistalog
SavePos True
<QueryXML>
<QueryList>
<Query Id="10">
<Select Path="ForwardedEvents">*</Select>
</Query>
<Query Id="20">
<Select Path="Application">*</Select>
<Select Path="System">*</Select>
</Query>
</QueryList>
</QueryXML>
Exec $Message = to_xml();
</Input>
<Output debugsyslog>
Module om_file
File 'C:\Install\nxlog.debug.syslog.txt'
Exec to_syslog_ietf();
</Output>
<Route syslog_file>
Path eventlog => debugsyslog
</Route>
Ak0
TLS Syslog Cert Question - PEM vs CER
mwidesba
Hi All,
We are cutting our NXLog Community version over to a new solution and are currently using .pem certificate files to encrypt syslog in the om_ssl module. The new system is using a .cer file for the certificate and when cutting over an agent to reference the .cer rather than .pem, it doesn't seem like the logs are being decrypted. I didn't see anything in the documentation indicating NXLog doesn't support .cer or only supports .pem, but I wanted to query the forum to see if anyone else experienced issues with setting the TLS to utilize a .epm files for encrypted log forwrding.
Thanks.
mwidesba
Add on end of line
5ss0
Hi
I have this config:
<Input WarnLog>
Module im_file
File 'C:\warnlog.txt'
</Input>
<Input SpamLog>
Module im_file
File 'C:\spamlog.txt'
</Input>
<Output out>
Module om_udp
Host 192.168.1.2
Port 5555
</Output>
<Route 1>
Path WarnLog => out
</Route>
<Route 2>
Path SpamLog => out
</Route>
Warnlog look like this:
16.09.2022 11:54:54 Update Updater: Switch modules type retval = 0x00005007 [NOT NEED] SYSTEM
16.09.2022 11:54:54 Update Updater: retval = 0x5003, failures: 0, profile: aktualizacja, trigger: ConfigChange SYSTEM
16.09.2022 11:54:54 Update Mirror: retval = 0x5003 SYSTEM
Spamlog:
16.09.2022 12:07:24 some@address.com some@address.com RE: subject 16.09.2022 12:04:59 0 No rule classifies the email Retained H|RN=0;RNP=
16.09.2022 12:12:24 some@address.com some@address.com RE: subject 16.09.2022 12:09:51 0 No rule classifies the email Retained H|RN=0;RNP=
Everything works fine, i see messages on my syslog server, but i don't know what type of message it is. From spamlog or form warnlog.
How can I add something to the end of each line before sending to syslog. Or there is other way?
5ss0