Forgive my ignorance but I'm looking to use NX Log to capture all windows events under System, Application and Security whether they be Audit, Info, error or critical. Am I correct in my assumption that with no filter's it should collect everything? <Select Path='Application'></Select> <Select Path='Security'></Select> <Select Path='System'></Select> or do I need to specify on single lines each severity level? for example: <Select Path='Application'>[System/Level=4]</Select> <Select Path='Application'>[System/Level=3]</Select> <Select Path='Application'>[System/Level=2]</Select>
and so on?
scotty created
Hi guys,
We have NXLog CE 3.0.2272 installed on a server which was originally installed by another user manually. We have tried uninstalling it via the uninstall string MsiExec.exe /X {xxxxx} via ConnectWise which appeared to have uninstalled ok. Since then, we have installed a newer CE version 3.1.2319 however after installation the nxlog service is non-existent. We suspect the uninstallation via string may have broken this. Several attempts using the original installer to repair or uninstall/reboots does not fix this.
Is there a way we can start fresh to remove NXLog completely then install? Any help would be appreciated :)
TIA Jordan
j_shek created
Hellow everyone!
I have a scenario that uses Citrix MCS where I installed the agent on the master image that provides clone images that should go with the nxlog agent installed and running. But the agent goes up with some errors as below:
2022-09-23 13:51:38 ERROR couldn't connect to udp socket on <IP:XYZ:514>; The socket operation was attempted to an unreachable network. 2022-09-23 13:51:46 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-09-23 13:51:46 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: Setup WitnessClientAdmin 2022-09-23 13:52:14 WARNING received a system shutdown request 2022-09-23 13:52:14 WARNING stopping nxlog service 2022-09-23 13:52:14 WARNING nxlog-ce received a termination request signal, exiting... 2022-11-02 23:16:38 INFO nxlog-ce-2.11.2190 started 2022-11-02 23:16:44 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-11-02 23:16:44 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: Setup WitnessClientAdmin 2022-11-02 23:27:15 ERROR EvtNext failed with error 15007: The specified channel could not be found. Check channel configuration. 2022-11-02 23:27:16 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-11-02 23:27:16 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: WitnessClientAdmin 2022-11-02 23:27:16 ERROR Failed to retrieve eventlog fields; The handle is invalid.
Has anyone had a problem like this using Citrix MCS?
Thanks James \0/
gijosgun created
Hello! We have permanent memory leaks on Windows Event Collect server with any 5 version NXLog. If we install any 4 version - it work without memory leak, but very slowly - it's accumulating queue on single filter for windows events. How we can help to fix it in next release?
Roman_Andreev created
mcoussi created
Hello all,
I have an application that sends log files to a directory formatted at YYYY-MM-DD.log (year, month, day). I'm watching the directory with the following stanza in the configuration file, but it does not recognize when the date changes and a new file is created. A service restart gets it reading the new file.
The configuration is as follows: <Input cvdupdate> Module im_file File "/var/log/cvdupdate/*.log" </Input>
I'm running NXLog on a Ubuntu 18.04 system. The version is 3.0.2272.
abajosh created
The logs arrive correctly, the only thing is that for the event viewer (example) 4624, I see logs with the same time, even in milliseconds, but the message varies only for a few lines of the "message" field.
I wanted to avoid using the repeat module because I would create the same a log recording increasing the database.
This is client configuration file :
define ROOT C:\\Program Files\\nxlog
define ROOT_STRING C:\\Program Files\\nxlog
define CERTDIR %ROOT%\\cert
Moduledir %ROOT%\\modules
CacheDir %ROOT%\\data
Pidfile %ROOT%\\data\\nxlog.pid
SpoolDir %ROOT%\\data
LogFile %ROOT%\\data\\nxlog.log
define MonitoredEventIds 4624, 4647
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input eventlog>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id='0'>
<Select Path="Security">*</Select>
</Query>
</QueryList>
</QueryXML>
Exec if $EventID NOT IN (%MonitoredEventIds%) drop();
Exec if $TargetUserName == "SYSTEM" drop();
Exec if $TargetUserName =~ /\$/ drop();
Exec if $TargetUserName =~ /UMFD/ drop();
Exec if $TargetUserName =~ /DWM/ drop();
Exec if $LogonType == "5" drop();
</Input>
<Output out>
Module om_tcp
Host (10.*****)
Port 1514
Exec to_json();
</Output>
<Route eventlog_to_out>
this is server nxlog.conf
User nxlog
Group nxlog
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
########################################
# Modules #
########################################
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in1>
Module im_tcp
Host 0.0.0.0
Port 1514
<Exec>
parse_json();
</Exec>
</Input>
<Output dbi>
Module om_dbi
SQL INSERT INTO SystemEvents (ReceivedAt, DeviceReportedTime, EventID, EventUser, EventSource, EventLogType, FromHost, NTSeverity, Priority, Message) \
VALUES (NOW(), NOW(), $EventID, $TargetUserName, $SourceName, $EventType, $Hostname, $Severity, $SeverityValue, $Message )
Driver mysql
Option host 127.0.0.1
Option username ****
Option password *******
Option dbname Syslog
</Output>
########################################
# Routes #
########################################
<Route 1>
Path in1 => dbi
</Route>
it's possible to not register duplicates at the origin?
it's possible to delete duplicates also in mysql database?
thanks you
IB_179669 created
#Hello, tanks in advance . #I am sending multiple logs from windows server to a linux collector #I have no issues with windows system logs , #Seems i can not send via snare windows system logs, and test plain text logs. #Is there any way to do that?
But when i switch to snare i can see no description about the warning 2022-10-21T09:21:21+00:00 Winserver MSWinEventLog#0111#011N/A#0111#011Fri Oct 21 09:21:21 2022#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011#011N/A#011N/A#015
The same line with snare commented: 2022-10-21T09:18:23.208210+00:00 Winserver WARNING: Can't open file \?\C:...\UPPS\UPPS.BIN: Permission denied#015
#My config:
Panic Soft #NoFreeOnExit TRUE
define ROOT C:\App\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
<Extension _syslog> Module xm_syslog </Extension>
<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>
<Extension _exec> Module xm_exec </Extension>
<Input internal> Module im_internal Exec $Hostname = hostname_fqdn(); </Input>
<Input vistalog> Module im_msvistalog </Input>
<Input eventlog> Module im_mseventlog </Input>
<Input testFile> Module im_file SavePos True RenameCheck True Recursive True PollInterval 0.5 #near real time File "C:\test\myfile.txt" ReadFromLast True </Input>
<Output out> Module om_tcp Host linux Port 514 #Exec to_syslog_snare(); </Output>
<Route r>
Path internal, eventlog, vistalog, testFile => out
Path testFile => out </Route>
eebs created
Hi,
i have below replace function for replacing "|0" to "Zero".
Exec $Message = replace($Message, "|0 ", "Zero");
now, i want to replace "|0" to "|15" with Zero.
do i need to add 15 more Exec replace function, or is there is any way to replace using single exec using regex?
Sajeshvv23 created
Hi all,
I think there might be a bug in the im_odbc module in the Linux (Debian11) im_odbc module for NXLog 5.6.7727 EE when used with the FreeTDS driver (via unixODBC) - so for example if you try to connect to MS SQL or Sybase databases.
Although all the drivers etc. were configured correctly (it was basically a copy from a system where it used to work) and connecting to the databases via ISQL worked fine, the Agent wasn't able to fetch logs and produced following errors in the nxlog.log:
INFO [im_odbc|inputxy] im_odbc successfully connected to the database
WARNING [im_odbc|inputxy] im_odbc detected a disconnection, attempting to reconnect in 1200 seconds
ERROR [im_odbc|inputxy] SQLDescribeParam failed, IM001:2:0:[unixODBC][Driver Manager]Driver does not support this function (odbc error code: -1)
I've tested multiple FreeTDS driver versions with multiple compile options - but none of them worked.
Finally I figured to try a different NXLog version, so I downloaded the NXLog Agent EE 4.10.5000 for debian10 and it worked out of the box with the standard FreeTDS driver from the Debian11 repo (v1.2.3) and unixODBC (2.3.6). So my conclusion is that there is a bug in the 5.6 EE NXLog Agent Version regarding the im_odbc module when used with FreeTDS (at least for Debian 11).
So while using 4.10 as a fallback is nice, it would be great to use the newer 5.x versions in the future.
BR Reinhard
ppum created
rdefulio created
Hi,
I'm experimenting with reading from an Azure eventhub with im_kafka. The eventhub receives security data from various security related azure components.
The im_kafka module works great after i found out that the username should be $connectionstring ;).
The output of the eventhub is a json dict with an array, like this: { records: [ {id: 1, msg: "xyz", etc},{id: 2, msg: "abc", etc}]}.
I tried to use extract_json("$.records") but that does not iterate over the array.
I also made a python script that writes the logs to a file, one line at a time:
from confluent_kafka import Consumer
import json
c = Consumer({....})
while True:
msg = c.poll(1.0)
eventhub_records = json.loads(msg.value())
for record in eventhub_records['records']:
print(json.dumps(record))
This works great, but i'd like to have something like this in nxlog. Can this be done, or does nxlog not support to split a single record into multiple records?
Thanks!
joost.bijl created
Hi!
Newbie on NXlog here. I'm trying to add the IP Address to the logs that I'm sending but I couldn't found any funcionality to do this (at least on the Community Edition). Is there any way to do that with the CE? Something like $ip = whats_my_ip?() ?
Thanks a lot!
JDiaz created
hi I have an application that writes many log files to folder on my windows server. Each log only contain one or two lines.
Previously the files were deleted by another system but that is now decommissioned.
I wanted to use NXLOG to delete the files once it has treated its contents(nothing else will be added after)
looking in the community edition docs I can use an ONEOF block with a remove.
The config is parsed correctly and the logs are sent to my graylog server, however the logs are never removed.
I saw some posts that suggested this block only works with the paid version but its strange that it is documented in the community docs?
Can any one tell me where i am going wrong ou another way of doing this
This is an extract from my config
version nxlog-ce-3.0.2284
<Extension _fileop> Module xm_fileop </Extension>
<Extension csv1> Module xm_csv Fields $Method,$Host,$Application,$EventID,$Severity,$Date,$Time,$Description,$Value Delimiter | </Extension>
<Input hyp-in>
Module im_file
SavePos TRUE
ReadFromLast FALSE
ActiveFiles 20
CloseWhenIdle TRUE
File "C:\HYPERVISION\logFiles*.*"
<Exec>
csv1->parse_csv();
$InputFileName1 = file_name();
$Message = $raw_event;
</Exec>
<OnEOF>
Exec file_remove(file_name(), now() - 6000);
</OnEOF>
</Input>
Petex created
Hi,
I have the requirement to buffer logs that were unable to be sent during network failures.
I have the following config which makes NXLog CE create a buffer file but it stays at 0KB size with a missing network.
My logs show the expected network failure:
2022-10-04 10:01:52 INFO nxlog-ce-3.0.2284 started
2022-10-04 10:01:52 INFO reconnecting in 1 seconds
2022-10-04 10:01:52 ERROR apr_sockaddr_info failed for myserver.com:1514; No such host is known.
2022-10-04 10:01:53 INFO reconnecting in 2 seconds
2022-10-04 10:01:53 ERROR apr_sockaddr_info failed for myserver.com:1514; No such host is known.
2022-10-04 10:01:55 INFO reconnecting in 4 seconds
nxlog.conf
<Extension syslog> Module xm_syslog </Extension>
<Extension json> Module xm_json </Extension>
<Input inWindowsAudit> Module im_msvistalog ReadFromLast True
<QueryXML> <QueryList> <Query Id="0"> <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select> </Query> </QueryList> </QueryXML> Exec parse_syslog(); to_json(); </Input>
<Processor buffer> Module pm_buffer Type Disk # 40 MiB buffer MaxSize 40960 # Generate warning message at 20 MiB WarnLimit 20480 </Processor>
<Output ssl> Module om_ssl Host myserver.com Port 1514 CAFile %CERTDIR%\ca-root.cer CertFile %CERTDIR%\client.cer CertKeyFile %CERTDIR%\client_private.key KeyPass secret AllowUntrusted FALSE </Output>
<Route main> Path inWindowsAudit => buffer => ssl </Route>
Any ideas? Thanks.
dmuensterer created
Did anyone manage to send logs from Splunk HF to NXlog server with SSL mutual authentication enabled? Able to share your configuration? Received an error on SSL version. Not sure it is due to Splunk HF conf or NXlog. Thanks.
nktj created
ygini created
har3005 created
Hi
Ive got a customer with windows server core (no gui) and they are wanting to uninstall the trial version and have asked if we can give them a copy of the original msi.
They are after version 5.4.7313
I've had a look at the download section and it isn't there.
Any suggestions please?
Thanks
Bryan
bryan.tabb.secops created
Is it possible to use a variable in a regex? I'm trying to do something like the following:
Exec if ($EventID == 4104) {
if defined(get_var('scriptblockid')) {
$id = get_var('scriptblockid');
if ($Message =~ /ScriptBlock ID: $id/) drop();
}
if ($Message =~ /ClassName = 'Root\/Microsoft\/Windows/) {
if ($Message =~/ScriptBlock ID: (\S+)/) {
set_var('scriptblockid', $1);
}
drop();
}
}
opoplawski created
