Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Nxlog Error 1067: the process terminated unexpectedly
parint@lhbank.co.th
We have the issue Nxlog Error 1067: the process terminated unexpectedly.Is there a way to fix this without reinstalling?Does re-installation require a reboot?
parint@lhbank.co.th
nxlog does not send log file to graylog
Tadeusz
Hi, can you help with the problem of nxlog not sending loose txt files to graylog?My nxlog.conf snippet about sending loose txt files<Input zpliku>Module im_fileFile "D:\file.log"</Input><Route messages_to_udp>Path zpliku => out</Route I have output defined for Graylog as GELF and the other section sending eventlog works correctly. Only sending loose files doesn't work here
Tadeusz
Integrate Windows NXLog Agent with google Chronicle
rodrigo1413
I need help to integrate my Windows Server with Nxlog Agent installed to forward events/logs to Google Chronicle. I read the documentation of NxLog of this integrations, but the topic that explains how to use nxlog to direct structure logs to chronicle, he talk to edit a XML file, but i dont know what file I need to edit.
rodrigo1413
IIS logs in Graylog
Nunzio Brandi
hello everyone,I am configuring nxlog to send IIS logs to Graylog.I managed to configure everything correctly but I would like to make the logs more readable on Graylog.I currently display them like this:would it be possible to somehow get the login name and ip of the user who logged in out of the “message” field?my current nxlog configuration is this: Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf\nxlog.d
define LOGDIR %ROOT%\data
include %CONFDIR%\*.conf
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Extension _fileop>
Module xm_fileop
# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>
# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
<Extension _gelf>
Module xm_gelf
</Extension>
<Extension _json>
Module xm_json
</Extension>
#######################################################################
IIS NXLOG
#######################################################################
<Extension w3c>
Module xm_csv
Fields $date, $time, $s_ip, $cs_method, $cs_uri_stem, $cs_uri_query, $s_port, $cs_username, $c_ip, $cs_User_Agent, $cs_Referer, $sc_status, $sc_substatus, $sc_win32_status, $time_taken
FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string
Delimiter ' '
QuoteChar '"'
EscapeControl FALSE
UndefValue -
</Extension>
<Input iis>
Module im_file
File "C:\inetpub\logs\LogFiles\W3SVC*\u_ex*"
SavePos TRUE
Exec $ShortMessage = $raw_event;
Exec if $raw_event =~/^#/ drop();\
else\
{\
w3c->parse_csv();\
$EventTime = parsedate($date + " " + $time);\
$EventTime = parsedate($date + " " + $time + "Z");\
$SourceName = "IIS";\
$raw_event = to_json();\
}
</Input>
#######################################################################
/IIS NXLOG
#######################################################################
Snare compatible example configuration
Collecting event log
<Input in>
Module im_msvistalog
</Input>
Converting events to Snare format and sending them out over TCP syslog
<Output out>
Module om_tcp
Host ha-centlog-vip.xxxxxxxx
Port 12201
Exec to_json();
OutputType GELF_TCP
Exec $Hostname = hostname_fqdn();
Exec $raw_event =$Hostname + ' IIS-NXLOG ' + $raw_event;
#Use the following line for debugging (uncomment the fileop extension above as well)
#Exec file_write("C:\Program Files (x86)\nxlog\data\nxlog_output.log", $raw_event);
</Output>
<Route iis-to-graylog>
Path iis => out
</Route>
Connect input 'in' to output 'out'
<Route 1>
Path in => outThanks
Nunzio Brandi
NXLog - read logs from a local .csv file on Windows Server
aleksta
Hi!I would like to have some help with my NXLog confiugration. I dont get any errors or so but in the SIEM I don't receive any logs at all from the source. So I'm guessing that there Is some issues reading logs from the .csv file. Or It could be something else. So my purpose with this Is to ship logs in a local .csv file to a SIEM. My thought was that NXLog should be a great solutions with this due to all extensions and so on. My NXLog configuration file Is based on these modules,xm_csvxm_syslogxm_jsonim_file (pointing out the local .csv file)out_ssl (for shipping logs through tls encryption)Been following along with this guide, Delimiter-Separated Values (xm_csv) | NXLog DocsHas anyone done this before? Thanks
aleksta
Problem using im_odbc (SQLDescribeParam Invalid parameter number)
Floss
Hi I'm getting a problem were the im_odbc module connects successfully to the DB via odbc but then straight away disconnects and give the error INFO [im_odbc|sccm_alerts] im_odbc successfully connected to the databaseWARNING [im_odbc|sccm_alerts] im_odbc detected a disconnection, attempting to reconnect in 10 secondsERROR [im_odbc|sccm_alerts] SQLDescribeParam failed, 07009:2:0:[Microsoft][ODBC Driver 17 for SQL Server]Invalid parameter number (odbc error code: -1) This is running on windows server 2019 and using MS SQL server 2017 (64bit)I have tried the ODBC driver 13/17/18 and tried just basic SQL query's to retrieve a single table (of just a couple or rows and columns ) The current DSN again works with PowerShell fine I've tried making my own and using system ones All permutations work using all ODBC drivers and SQL query's with PowerShell no problem <Extension _json>
Module xm_json
</Extension>
<Input sccm_alerts>
Module im_odbc
ConnectionString DSN=test;Trusted_Connection=yes;
SQL SELECT ID,TypeID,TypeInstanceID,Name,FeatureArea,
ObjectWmiClass,Severity FROM V_SMS_Alert
</Input>
<Output outfile>
Module om_file
File 'C:\scripts\out.log'
Exec to_json();
</Output>
<Route sccm>
Path sccm_alerts, sccm_alerts => outfile
</Route>It was al installed as a “standard” SCCM install which is working fine Many thanks for any help, please let me know if you needs any more information Kind regards
Floss
NXLog Manager support for RHEL
billychua
Hi,Can check if the current NXLog Manager can support RHEL 8.8?And any roadmap to support RHEL9 in future? ThanksRegards, Billy
billychua
NXLOG for windows server 2019
אסף ל
hi,First of all I would like to know if NXLOG compatible with Windows server 2019. I have trouble with the performance of nxlog on windows 2019 while on 2012 everything is fine.The logs do not go up instantly even though the configuration is the same and there is no issue with the network.Does anyone knows what the issue might be?Thank you for your answers.
אסף ל
Seeking Assistance with NXLog Agent Installation aws ec2 linux instance
siuolkl
Hi Experts,I am reaching out to seek assistance with the installation of the NXLog Agent on my AWS EC2 Linux instance. I have encountered some challenges during the installation process and would greatly appreciate your guidance and support.The AWS EC2 instance I am using is running Amazon Linux version 2023. Here are the details of my Linux distribution:- Name: Amazon Linux- Version: 2023- ID: amzn- ID_LIKE: fedora- Version_ID: 2023- PLATFORM_ID: platform:al2023- PRETTY_NAME: Amazon Linux 2023- ANSI_COLOR: 0;33- CPE_NAME: cpe:2.3:o:amazon:amazon_linux:2023- HOME_URL: [https://aws.amazon.com/linux/](https://aws.amazon.com/linux/)- BUG_REPORT_URL: [https://github.com/amazonlinux/amazon-linux-2023](https://github.com/amazonlinux/amazon-linux-2023)- SUPPORT_END: 2028-03-01I kindly request your assistance in determining the correct RPM package I should download for this particular distribution and version of Linux.https://nxlog.co/downloads/nxlog-ce#nxlog-community-editionis there any documentation or resources that I can refer to for guidance on the installation process.Appreciate any help.
siuolkl
Systemd and open files limit
billychua
Would like to check where should i change the file for RHEL 8? I found the below link but doesn't work for rhel 8. Common issues :: NXLog Documentation This scenario requires edits to the service file or an override. To check NXLog system limits, use the following command:$ cat /proc/$(cat /opt/nxlog/var/run/nxlog/nxlog.pid)/limitsOn Systems not using /proc, check the system’s open file limit:$ sysctl kern.maxfilesTo adjust limits for nxlog, create /etc/systemd/system/nxlog.service.d/override.conf and add the following definition:[Service]
LimitNOFILE=100000Update the service settings with:$ systemctl daemon-reload
billychua
Multiple log in Windows Events Log
santiagonahuel.sarchetti@bbva.com
Hi, I have a problem trying to send the raw event of Windows Server 2016. I have this configuration in nxlog.conf: Panic Soft
#NoFreeOnExit TRUE
define ROOT C:\Program Files\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf\nxlog.d
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
LogFile %LOGFILE%
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _leef>
Module xm_leef
</Extension>
<Extension xml>
Module xm_xml
</Extension>
<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>
<Extension _exec>
Module xm_exec
</Extension>
<Extension rewrite>
Module xm_rewrite
Keep EventXML
</Extension>
<Input argentina>
Module im_msvistalog
<QueryXML>
<QueryList>
<Query Id="0" Path="Security">
<Select Path="Security">*[System[band(Keywords,13510798882111488)]]</Select>
</Query>
</QueryList>
</QueryXML>
</Input>
<Processor buffer1>
Module pm_buffer
MaxSize 102400
Type Mem
</Processor>
<Output qradar>
Module om_tcp
Host XXX.XXX.XXX.XXX:514
Exec $raw_event = $EventXML;
Exec delete_all();
</Output>
<Route r1>
Path argentina => buffer1 => qradar
</Route>but in our SIEM i see this output (every line is a diferent log):I used "tcpdump" to saw if every log are diferent packets but i saw that it's only one packet but it has a special character that separete the line (i thought) .Could someone help to solve this? maybe using “replace” or changing the encoding. Thanks
santiagonahuel.sarchetti@bbva.com
Google Chronicle (om_chronicle) with multiple input and output error [HELP]
billychua
Anyone has encounter error before or have issue when doing multiple input and output for chronicle.If i list 1 chronicle setting and with the below configuration will have no error. define BASE_URL https://abc.defgh.ijkhlmndefine ENDPOINT unstructuredlogentriesdefine API_KEY key=ABCdefghiJKLNMOP<Input listener1> Module im_tcp Host 1.1.1.1:10000</Input><Output to_chronicle_out> Module om_chronicle URL %BASE_URL%%ENDPOINT%?%API_KEY% HTTPSCAFile \opt\cert.crt LogType WINEVTLOG ChronicleBatchSize 1024</Output><Route route_chronicle> Path listener1 => to_chronicle_out</Route> If i list 2 or more chronicle setting and i do encounter errordefine BASE_URL https://abc.defgh.ijkhlmndefine ENDPOINT unstructuredlogentriesdefine API_KEY key=ABCdefghiJKLNMOP<Input listener1> Module im_tcp Host 1.1.1.1:10000</Input><Input listener2> Module im_tcp Host 1.1.1.1:10001</Input><Output to_chronicle_out> Module om_chronicle URL %BASE_URL%%ENDPOINT%?%API_KEY% HTTPSCAFile \opt\cert.crt LogType WINEVTLOG ChronicleBatchSize 1024</Output><Output to_chronicle_out1> Module om_chronicle URL %BASE_URL%%ENDPOINT%?%API_KEY% HTTPSCAFile \opt\cert.crt LogType WINEVTLOG ChronicleBatchSize 1024</Output><Route route_chronicle> Path listener1 => to_chronicle_out</Route><Route route_chronicle1> Path listener2 => to_chronicle_out1</Route>The error i'm getting is. ERROR [CORE|main] can't initialize logger: already initialized
billychua
nxlog manager push policy to nxlog agent
billychua
Hi, I have the following error. I'm using agent to manager using agent-ca.pem. I have encounter error when trying to push down the managed.conf file. Below is the error. Would like to check can i just do agent-ca.pem for manager and agent communication or i need to use mutual authentication in order to push down the managed.conf file?2023-06-28 17:37:08,547 INFO 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Reloading agent: Server1 2023-06-28 17:37:08,549 INFO 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Agent configuration exported: Server1 2023-06-28 17:37:08,551 ERROR 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Failed to reconfigure agent: Server1 [Unable to perform requested lazy initialization [com.nxsec.log4ensics.data.model.certdb.Certificate.cer] - no session and settings disallow loading outside the Session] 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.s.CommAgents] - Agent Server1 set to OFFLINE state 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.s.CommAgents] - Agent Server1 removed from opened connections 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.CommAgent] - Closing the socket for agent Server1: Unable to perform requested lazy initialization [com.nxsec.log4ensics.data.model.certdb.Certificate.cer] - no session and settings disallow loading outside the Session 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.s.CommAgents] - Agent Server1's connection closed 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.CommAgent] - Agent Server1 connection has been reset until execute 'putFile' 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Agent reconfigured: Server1 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.c.s.CommAgents] - Agent Server1 connection has been reset until execute RestartServerTask 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Agent restarted: Server1 2023-06-28 17:37:08,563 INFO 1.1.1.17 unknown [c.n.l.s.a.AgentManagementService] - Agent reloaded: Server1 2023-06-28 17:37:08,568 INFO 1.1.1.17 unknown [c.n.l.s.a.s.n.MultiReactor] - Agent manager accepted agent connection Server1 from 1.1.1.48 2023-06-28 17:37:15,812 INFO 1.1.1.17 unknown [c.n.l.s.a.c.CommAgent] - getServerInfo from agent server2 succeeded. 2023-06-28 17:37:15,812 INFO 1.1.1.17 unknown [c.n.l.s.a.c.CommAgent] - getServerInfo from agent server3 succeeded. 2023-06-28 17:37:16,618 WARN 1.1.1.17 unknown [c.n.l.s.a.s.AgentSslVerifier] - Agent manager failed to accept agent connection from 1.1.1.25 [EOF during handshake with peer 1.1.1.205/Server1]
billychua
Adding Source IP to Messages
russeller
I am receiving some logs from network devices that don't include the source IP or host in the log message. How do I add $MessageSourceAddress to the message so I can identify its source?I've tried Exec $raw_event = $raw_event +$MessageSourceAddress;But that does nothing. What is the correct syntax for this?
russeller
nxlog agent to nxlog-manager
billychua
I have encountered an issue that in nxlog agent i can connected to the nxlog manager. But in the nxlog manager i couldn't see any agent in the list.
billychua
Google Chronicle (om_chronicle) with amazon Linux2
billychua
I have install the amzon linux2 agent into EC2. After that we realized the om_chronicle is not available in the /opt/nxlog/modules/output/ directory. We copy the om_chronicle from Redhat and place it into the below directory. When restart the service and it have the below error.Error [Core|main] Failed to load module from /opt/nxlog/modules/output/om_chronicle.so, libssl.so.1.1: cannot open shared object file. No such file or directory; DSO load/failed
billychua
NXLOG for Parrot OS
Roland9494
Hi. Will there be a Nxlog EE for Parrot OS? If so are there an expected date.RegardsRoland
Roland9494
Multiple include.conf files cause a Warning in logs in version 5
sinisa
Hi,We recently upgraded our NXlog agents to version 5*, and have noticed the following warning in the logs on some servers:2023-05-16 08:03:32 WARNING [CORE|main] not starting unused module logfile_IMAP42023-05-16 08:03:32 WARNING [CORE|main] not starting unused module logfile_POP3…We figured out that this happens when we have more then one _include.conf file on top of the main nxlog.conf file. It seems that it reads one of them, and then spews out this warning for all the modules/inputs in the other include.conf files. This is a new behavior. We use the following syntax at the bottom of the main nxlog.conf file:include C:\\Program Files\\nxlog\\conf\\*_include.confIt used to work till the upgrade to version 5. Anyone else has this issue?
sinisa
im_maculs does not get logs seen with "log stream"
simtom
Hi,we've configured im_maculs and have noticed, that it does not handle expected ULS logs (which are seen with log stream command).We then configured im_exec module, to run log stream and have compared configurations head-to-head, the input with im_exec receives expected logs, while im_maculs does not.Here is configuration:<Input m_uls>Module im_maculs<Exec># Filterif ($subsystem == 'com.apple.launchservices' and $category == 'open'){$Hostname = hostname();} else{drop();}to_json();</Exec></Input> <Input m_logstream>Module im_execCommand /usr/bin/logArg streamArg --style=ndjsonArg --type=log<Exec>if $raw_event =~ /^{/{# Filterif ($subsystem == 'com.apple.launchservices' and $category == 'open'){$Hostname = hostname();} else{drop();}to_json();} else{# Fix ERROR [im_exec|m_logstream] failed to parse json string, lexical error: invalid char in json text.; Filtering the log data using "t; (right here) ------^; [Filtering the log data using "type == 1024"]# Since first log stream output line is not a json log entry, but informational messagedrop();}</Exec></Input>The m_logstream Input produces log message every time a graphical application is openned in macOS, while the m_uls - does not.
simtom
NXlog agent not collecting the windows security event from Forwarded Events.
jilin
HiWe are not receiving the Windows security logs via the nxlog agent. We noticed that nxlog.log successfully connected the destination IP, and port.2023-04-12 08:18:57 INFO [CORE|main] nxlog-5.7.7898 (68bb24e7e@REL_v5.7) started on Windows2023-04-12 08:18:57 INFO [om_udp|syslogout] connecting to 10.129.5.20:5212023-04-12 08:18:57 INFO [om_udp|syslogout] successfully connected to 10.129.5.20:521Nxlog. conf file configuration## Please set the ROOT to the folder your nxlog was installed into,## otherwise it will not start.define ROOT C:\Program Files (x86)\nxlogdefine CERTDIR %ROOT%\certdefine CONFDIR %ROOT%\confdefine LOGDIR %ROOT%\datadefine LOGFILE %LOGDIR%\nxlog.logLogFile %LOGFILE%Moduledir %ROOT%\modulesCacheDir %ROOT%\dataPidfile %ROOT%\data\nxlog.pidSpoolDir %ROOT%\data<Extension syslog>Module xm_syslog</Extension><Input eventlog>Module im_msvistalog# ReadFromLast True<QueryXML><QueryList><Query Id="0"><Select Path="ForwardedEvents">*</Select></Query></QueryList></QueryXML></Input><Processor eventlog_transformer>Module pm_transformer# OutputFormat syslog_rfc5424</Processor><Processor buffer>Module pm_buffer# 100 MB disk bufferMaxSize 102400Type disk</Processor>########################OUTPUTS##########################<Output syslogout>Module om_udpHost 10.129.5.20:521########################PUTS EVENT IN IETF FORMAT############Exec to_syslog_snare();</Output><Route 1>Path eventlog => eventlog_transformer => syslogout</Route>Note: We are using the nxlog agent version 5.7.7898 and security events are available in Forwarded events
jilin