Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Output udpfile rotate_to wrong filename
hate
Hi,I am using NXLog with the Example 108. “File Rotation Based on Size” from the NXLog Community Edition Reference Manual.In rare cases i have the problem that rotate_to uses the wrong filename and overwrite some other logfile. In the example below “logid.log” to “Mod-002”. See nxlog.logVersion: nxlog-ce-3.1.2319nxlog.logZeile 3644025: 2023-03-14 10:28:02 INFO om_file successfully rotated file 'C:\Program Files\nxlog\data\10.87.243.24\logid.log' to 'C:\Program Files\nxlog\data\10.87.243.24\Mod-002.20230314102802.log'nxlog.conf## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
define ROOT C:\Program Files\nxlog
#define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension exec>
Module xm_exec
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Extension fileop>
Module xm_fileop
</Extension>
<Input udp>
Module im_udp
Host 10.87.243.20
Port 514
Exec parse_syslog();
Exec dir_make('%LOGDIR%' + $Hostname);
</Input>
<Output udpfile>
Module om_file
CreateDir TRUE
File '%LOGDIR%' + $Hostname + '' + $SourceName + '.log'
Exec if udpfile->file_size() > 5M {
$newfile = '%LOGDIR%' + $Hostname + '' + $SourceName + '.' + strftime(now(), "%Y%m%d%H%M%S") + '.log';
udpfile->rotate_to($newfile);
exec_async('%CONFDIR%\bzip2.exe', $newfile);
}
</Output>
<Route udp>
Path udp => udpfile
</Route>Any ideas what's going wrong here?Thanks
hate
Community Edition - Failed to load module xm_python.dll
hukel
Running nxlog-ce-3.1.2319 on Windows.2023-01-19 08:12:46 ERROR Failed to load module from C:\xxxxx\nxlog\modules\extension\xm_python.dll, The specified module could not be found. ; The specified module could not be found.The NXLog Python DLL is on disk so I am wondering if this is complaining because I don't have the nxlog Python module (which I don't see in pip).I looked around for some setup instructions but I don't see any extra setup steps required for Python (aside from writing the script).Config:<Extension python> Module xm_python PythonCode "C:\xxx\NXLogDev\modules\convert_to_splunk_hec.py"</Extension> PythonCodeimport nxlogdef get_splunk_hec_format(event): nxlog.log_warning('in get_splunk_hec_format()') for field in event.field_names(): nxlog.log_debug('Received field:' + field)
hukel
WARNING nxlog-ce received a termination request signal, exiting...
vic chen
Why do I always receive the message "WARNING nxlog-ce received a termination request signal, exiting..." and I don't receive any message using GELF UDP in Graylog input, unless I use Raw/Plaintext UDP? Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf\nxlog.d
define LOGDIR %ROOT%\data
include %CONFDIR%\*.conf
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
define IISLOG "C:\inetpub\logs\LogFiles\W3SVC2\u_ex*.log"
#######################################################################
EXTENTIONS
#######################################################################
<Extension _gelf>
Module xm_gelf
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Extension fileop>
Module xm_fileop
</Extension>
#######################################################################
IIS NXLOG
#######################################################################
<Extension w3c>
Module xm_csv
Fields $date, $time, $s_ip
FieldTypes string, string, string
Delimiter ' '
QuoteChar '"'
EscapeControl FALSE
UndefValue -
</Extension>
<Input iis>
Module im_file
File "C:\inetpub\logs\LogFiles\W3SVC2\u_ex*.log"
SavePos TRUE
Exec if $raw_event =~/^#/ drop();\
else\
{\
w3c->parse_csv();\
$EventTime = parsedate($date + " " + $time);\
$EventTime = parsedate($date + " " + $time + "Z");\
$SourceName = "IIS";\
$raw_event = to_json();\
}
</Input>
<Output graylog>
Module om_udp
Host 192.168.3.250
Port 1322
OutputType GELF
Exec $Hostname = hostname_fqdn();
Exec $raw_event =$Hostname + ' IIS-NXLOG ' + $raw_event;
#Use the following line for debugging (uncomment the fileop extension above as well)
# exec file_write("C:\\Program Files\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>
<Route iis-to-graylog>
Path iis => graylog
</Route>
vic chen
WARNING not starting unused module dbi
yang server
nxlog-ce-3.1.2319 add custom conf to ` /etc/nxlog/nxlog.d`; systemctl restart nxlog;systemctl status nxlog;got message: how to enable the im_dbi module ?the config looks like:config file: `/etc/nxlog/nxlog.d/icslog.conf`:
<Input dbi>
Module im_dbi
Driver mysql
Option host 127.0.0.1
Option username root
Option password pp
Option dbname logs
SQL SELECT * FROM ics_alarm_log
</Input>
<Output file>
Module om_file
File '/tmp/ics_alarm_log.csv'
</Output>
yang server
Transformation of logs from json to kvp
giuseppe
Hello everyone,We should log on Windows server some IIS and SQL Server logs via agent in Community Edition. Through documentation I have examples that produce as results logs in csv and/or json format.Could you give me a hand in transforming the logs from json and/or csv format to key-value (kvp)Thank you very much for the support.
giuseppe
nxlog for Debian 7 download issue
rui.leitao@efacec.com
Hello community. I'm trying to download nxlog for Debian 7 (wheezy) and I'n getting error “Couldn't download something went wrong….”No problem with downloads for upper versions of Debian. Any help ?
rui.leitao@efacec.com
Nxlog Installation issue on Amazon Linux 2
Francis Ho
Hello,I created a new EC2 instance using Amazon Linxu AMI 2 and downloaded the Community edition of Nxlog Redhat 7 version and run into dependency error when I run yum install The version of python3 on the server is 3.7.16Anyone else run into the same issue found a way forward?yum install nxlog-ce-3.1.2319_rhel7.x86_64.rpmLoaded plugins: extras_suggestions, langpacks, priorities, update-motdExamining nxlog-ce-3.1.2319_rhel7.x86_64.rpm: nxlog-ce-3.1.2319-1.x86_64Marking nxlog-ce-3.1.2319_rhel7.x86_64.rpm to be installedResolving Dependencies--> Running transaction check---> Package nxlog-ce.x86_64 0:3.1.2319-1 will be installed--> Processing Dependency: apr >= 1.2 for package: nxlog-ce-3.1.2319-1.x86_64--> Processing Dependency: libdbi >= 0.8.1 for package: nxlog-ce-3.1.2319-1.x86_64--> Processing Dependency: libapr-1.so.0()(64bit) for package: nxlog-ce-3.1.2319-1.x86_64--> Processing Dependency: libdbi.so.0()(64bit) for package: nxlog-ce-3.1.2319-1.x86_64--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: nxlog-ce-3.1.2319-1.x86_64--> Running transaction check---> Package apr.x86_64 0:1.7.2-1.amzn2 will be installed---> Package libdbi.x86_64 0:0.8.4-6.amzn2.0.2 will be installed---> Package nxlog-ce.x86_64 0:3.1.2319-1 will be installed--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: nxlog-ce-3.1.2319-1.x86_64--> Finished Dependency ResolutionError: Package: nxlog-ce-3.1.2319-1.x86_64 (/nxlog-ce-3.1.2319_rhel7.x86_64) Requires: libpython3.6m.so.1.0()(64bit)You could try using --skip-broken to work around the problemYou could try running: rpm -Va --nofiles --nodigest
Francis Ho
Event levels
gavin.lacey@telegraph.co.uk
Example documentation reads <Select Path='Application'>*</Select> <Select Path='Security'>*[System/Level<4]</Select> <Select Path='System'>*</Select> I have <Select Path="ForwardedEvents">*</Select> can I specify the config as <Select Path="ForwardedEvents">*[System/Level<4]</Select>to pick up all log files as some are not forwarding
gavin.lacey@telegraph.co.uk
Windows Event Logs not forwarding
gavin.lacey@telegraph.co.uk
I have Windows server subscribing to a windows log event forwarder. I have noticed that some events that appear within the forwarded event log are not ingested by NX Log and forwarded to the SIEM platform. eg event id 1102 and 22both events are forwarded from the source servers to the windows forwarded where nx log is running so windows upload is fine, just nxlog sending on to SIEM
gavin.lacey@telegraph.co.uk
NXlog 2.10.2150 crashing on Windows 2022 after applied the latest windows Patch
Test Account
After applying the most recent Windows Patch, NXlog 2.10.2150 crashed on Windows 2022. Upgrades the nxlog agent to version 3.1.2319, however the problem continuesThe nxlog process became suspended in task manager, and the agent kept crashing.
Test Account
[HELP] ERROR Failed to load module out_file.so - cannot open shared object file - DSO load failed
pras92
Hi, I'm trying to send logs from a client (Ubuntu 20.04.4 LTS) to server (Ubuntu 22.04 LTS).nxlog -v from the server throws an error: ERROR Failed to load module from /usr/lib/nxlog/modules/output/out_file.so, /usr/lib/nxlog/modules/output/out_file.so: cannot open shared object file: No such file or directory;DSO load failed Can someone please help me identify the problem?Thanks. Below is the conf file for client, which has no errors:define ROOT /etc/nxlog
Group nxlog
Moduledir /usr/lib/nxlog/modules
CacheDir %ROOT%/data
SpoolDir %ROOT%/data
Pidfile /tmp/nxlog.pid
include /etc/nxlog/nxlog.d/*.conf
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in1>
Module im_file
File "/var/log/auth.log"
SavePos TRUE
ReadFromLast TRUE
</Input>
<Input in2>
Module im_file
File "/var/log/syslog"
SavePos TRUE
ReadFromLast TRUE
</Input>
<Output out1>
Module om_ssl
Host 10.XXX.XXX.XXX
Port 6514
</Output>
<Output out2>
Module om_ssl
Host 10.XXX.XXX.XXX
Port 6514
</Output>
<Route file_to_ssl>
Path in1 => out1
Path in2 => out2
</Route>Below is the conf file for server showing the above mentioned error: define ROOT /etc/nxlog
Group nxlog
Moduledir /usr/lib/nxlog/modules
CacheDir %ROOT%/data
SpoolDir %ROOT%/data
Pidfile /tmp/nxlog.pid
include /etc/nxlog/nxlog.d/*.conf
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in1>
Module im_ssl
Host 10.XXX.XXX.XXX
Port 6514
</Input>
<Input in2>
Module im_ssl
Host 10.XXX.XXX.XXX
Port 6514
</Input>
<Output out1>
Module out_file
File "/var/log/nxremotelogs/"$Hostname".log"
SavePos TRUE
ReadFromLast TRUE
</Output>
<Output out2>
Module out_file
File "/var/log/nxremotelogs/"$Hostname".log"
SavePos TRUE
ReadFromLast TRUE
</Output>
<Route 1>
Path in1 => out1
</Route>
<Route tcproute>
Path in2 => out2
</Route>
pras92
NXLog Community edition not performing any output
jhartman
SO I was in the process of creating a custom parser for NetMotion VPN logs but for some reason, no matter what I specify in the nxlog.conf I have no output.I originally had an older agent so I uninstalled and reinstalled with the latest download. - No changeI originally had an Exec stanza with some regex to capture some groups and assign them to some variables, I removed that whole section and am simply doing "parse_syslog(); - No change This was my original conf filepanic SOFT
define INSTALLDIR C:\Program Files\nxlog
define LOGDIR %INSTALLDIR%\data
define MYLOGFILE %LOGDIR%\nxlog.log
LogLevel DEBUG
LogFile %MYLOGFILE%
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input NetMotion>
Module im_file
File "C:\Testing-logs\NetMotion.txt"
<Exec>
if $raw_event =~ /m_user="([^"]+).+?pop_ip_srv="([^"]+).+?ses_start="([^"]+).+?ses_state="([^"]+).+?vip="([^"]+)/
{
if $4 == 'Connected' $event_type = 'VPN_SESSION_IP_ASSIGNED';
$version = 'v1';
$time = $3;
$account = $1;
$assigned_ip = $6;
$source_ip = $2;
$authentication_result = 'SUCCESS';
$authentication_target = $5;
}
</Exec>
</Input>
<Output local_file>
Module om_file
Exec to_json();
File "C:\Testing-logs\Parsed.txt"
</Output>
<Route NM_to_file>
Path NetMotion => local_file
</Route>After that was not producing anything I decided to rip the whole thing out and simply do a “parse_syslog” like below but still no luck.panic SOFT
define INSTALLDIR C:\Program Files\nxlog
define LOGDIR %INSTALLDIR%\data
define MYLOGFILE %LOGDIR%\nxlog.log
LogLevel DEBUG
LogFile %MYLOGFILE%
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input NetMotion>
Module im_file
File "C:\Testing-logs\NetMotion.txt"
Exec parse_syslog();
</Input>
<Output local_file>
Module om_file
Exec to_json();
File "C:\Testing-logs\Parsed.txt"
</Output>
<Route NM_to_file>
Path NetMotion => local_file
</Route>I've done similar things before and have never really had an issue but this is throwing me for a loop. The nxlog.log shows no errors and actually says that the routes are being processed. Even when I was applying the custom regex it showed the regex being applied and everythign workign but there were still no lines being written to the Parsed.txt file. Can anyone see anythign blatantly obvious that I'm missing that could stop this from working?
jhartman
permanent download link gone
AutoNick
Hi folks,how can we download the latest agent version without going true the manual download page.Since the change of your webpage, the previous links do not work anymore.This crucial, to have the latest agents in place.Thank youNick
AutoNick
Sending TLS Syslog over from Trellix ePO to NXLOG CE
blackwat3rr
Hello, I'm having trouble forwarding Logs from my ePO instance to nxlog. ePO will say Syslog connection success under test connection, however, nxlog.log will say “Error Module ssl coulden't read the input; invalid header received by Syslog_TLS input reader, input is not RFC 5425 compliant.” It seems like nxlog is having trouble decrypting due to maybe a certificate issue but im not sure. Any help would be greatly appreciated.
blackwat3rr
Unable to download CE v3.1.2319 for Windows
Dave Small
Getting “Ajax request cannot be executed” error when downloading CE-3.1.2319.msi file from nxlog.co.
Dave Small
Unable to download Community addition
damiany@terrane.net
I havent been able to download the files for any of the community edition agents. https://nxlog.co/downloads/nxlog-ce#nxlog-community-edition , select the OS version and click Download. 5 sec later I get a pop up stating “Undefined”. I have tried on different Browsers, platforms, and workstations and there is no change in the behavior.
damiany@terrane.net
NXLog-CE Question
greg.smith
Hello,This is not a installation question.Using wget, as I have done for past 6 years was grab a NXLog-CE installation and install on my Linux core servers. Yesterday 11/22/2022 I was unable to do this. I also noticed the Web Site has changed for downloading community versions and now I need to make account. I'm assuming at this point, Steps needed are install NXLog on any core servers I need to make account on NXLog site, Download the package needed. Transfer the NXLog package to a closed environment that we have, Upload NXLog package to a internal repo and distribute it as needed? I'm also assuming this is a security procedure taken by NXLog? If anyone could enlighten me on the new changes that would be great.Thanks-Greg
greg.smith
Exclude Windows logs based on process name
Alper Demir
I am sending Windows logs to Graylog via nxlog community edition, but certain processes are generating so much logs that I'd rather not send at all, so I'm trying to figure out how to modify nxlog config to exclude logs with specific terms or generated by a specific process using the “ProcessName” field for example. any help would be appreciated.
Alper Demir
NXLogAgent: Sometimes cannot forwarding log to FortiSIEM (Agent stop running)
Sunat Praphanwong
I would like to ask, in some circumstances NXLogAgent on Windows, the agent cannot forwarding log to FortiSIEM (sometimes the agent was stopped by itself), I need to manual restart the agent to make the agent running again, in this situation is it abnormal or not?Another question would be about the log format can be parsed by FortiSIEM or I need to custom parser to parse this log format or someone can provide this parser to me?Best Regards,
Sunat Praphanwong
NXLOG configuration to work with GRAYLOG
José Manuel
Hi the pronblem is that all works but I don´t receive any log.Graylog version 4.3 in debian 11. Sidecar graylog 1.2 and NXLOG 3.0 if my memory doesn´t fail.What can i do?Thanks and happy new year.
José Manuel