Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
NxLog CE version 2.X statement of Supporting windows server 2022
Ken1 created
Hi,We are using NXLog-CE version 2.10.2150 and we prepare to deploy NX log agent in Windows server 2022.Is Nxlog 2.10.2150 supporting log collection for Windows server 2022?If the version is not supported, what is the minimum NX log agent to support Windows server 2022?Thanks.
Ken1 created
nxlog platform start up issue
EH_272573 created
Have installed nx onprem on ubuntu 22.04.5 LTS and cant login after the install finishes. Not sure how many pods are suppose to run but I see a postgress and vault 1c9df1fc6f5d nxlogacr.azurecr.io/vault:1.13.3 3 hours ago Up 2 hours ago (healthy) nxlog-1_2_2-vault-1eef5bec91376 nxlogacr.azurecr.io/postgres:16.3-alpine -c config_file=/e... 3 hours ago Up 2 hours ago (healthy) nxlog-1_2_2-postgres-1 Below is what shows listening and I don't see any web services. dp UNCONN 0 0 10.89.0.1:53 0.0.0.0:* users:(("dnsmasq",pid=1291,fd=4)) udp UNCONN 0 0 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=627,fd=13)) udp UNCONN 0 0 [fe80::d433:a2ff:fe02:9e0d]%cni-podman1:53 [::]:* users:(("dnsmasq",pid=1291,fd=10)) tcp LISTEN 0 32 10.89.0.1:53 0.0.0.0:* users:(("dnsmasq",pid=1291,fd=5)) tcp LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* users:(("systemd-resolve",pid=627,fd=14)) tcp LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=750,fd=3)) tcp LISTEN 0 32 [fe80::d433:a2ff:fe02:9e0d]%cni-podman1:53 [::]:* users:(("dnsmasq",pid=1291,fd=11)) tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd"I see the following error in nxp.logHA Mode standbyActive Node Address <none>Raft Committed Index 31Raft Applied Index 31Error authenticating: error looking up token: Error making API request.URL: GET http://0.0.0.0:8200/v1/auth/token/lookup-selfCode: 500. Errors:I got the following during the install[2024-09-25 00:28:55] [INFO] Vault container ID: 62f8bd5e1e00[2024-09-25 00:28:55] [INFO] Executing command (/init/bootstrap.sh) in container 62f8bd5e1e00...make: *** [Makefile:231: seed-vault] Error 2
EH_272573 created
NXLog Platform - vault container stuck in bootloop
nervevector created
Hello,I'm attempting to install the NXLog Platform on-prem on a Ubuntu 24.04 LTS VM, but I am running into the following error on the nxlog-1_2_2-vault-1 container:fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later)
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)
ERROR: unable to select packages:
supervisor (no such package):
required by: world[supervisor]
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later)
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)
ERROR: unable to select packages:
supervisor (no such package):
required by: world[supervisor]
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
When accessing this link from my host machine, I am able to download the .tar.gz, so it does not seem to be a network issue. Additionally, from the VM I am able to reach the internet perfectly fine to, e.g., run updates and I can cURL the URL from the VM as well.Any assistance on this would be appreciated!Thanks
nervevector created
Replace function to remove a string
Jay1 created
HiHere is my configuration. However, only Sysmon events are not working because they contain "/operational: " at the beginning of the message, which causes the events to be parsed incorrectlySo I want to know how to remove "/operational: "<Extension syslog> Module xm_syslog</Extension><Input in> Module im_msvistalog ReadFromLast True <QueryXML> <QueryList> <Query Id="0"> <Select Path="Application">*</Select> <Select Path="System">*</Select> <Select Path="Security">*</Select> <Select Path="Windows PowerShell">*</Select> <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select> </Query> </QueryList> </QueryXML> Exec $UnixTime = integer($EventTime)/1000; </Input><Output devo_relay> Module om_tcp Host 192.168.29.133Port 13000 I tried Exec if ($SourceName =~ /Microsoft-Windows-Sysmon\/Operational/) { $Message = replace($Message, "/operational: ", ""); }and Exec if ($Message =~ /\/operational: /) { $Message = replace($Message, "/operational: ", ""); }But all did not work
Jay1 created
Read a log with yesterdays date in the filename
James created
I'm successfully using this config format with im_file to read logs with todays date in the filename:'\\server.domain\Logs\IN' + strftime(now(), "%y%m%d") + '.log'One of our services writes its log for the previous day at 3am on the next day. The filename has yesterdays date. What's the easiest/neatest/most efficient way of reading this log please?
James created
Is it possible to "reset the baseline" for the File Integrity module?
elazur@ecampus.com created
We currently have NXLog running on Windows 2019 with the file integrity module that monitors files in sub directories under a main directly. I.E. Say we have about 20 subdirectories for files under a directory called c:\code. If any file is changed/deleted/added under that directory, an alert message is sent out via the OM_UDP module to our siem. The problem I am looking to resolve, is that I will be copying over about 10 new directories with hundreds of files in each directory, and I don't want NXLog to generate hundreds of alerts because it found new files and directories. Is there a way to “reset” or “refresh” the baseline after the new files/folders are copied over so it will know those files should be there?
elazur@ecampus.com created
NXLog Manager 5.7.5
billychua created
Hi,Could you please advise on how to replace the NXLog-Manager's self-signed certificate for the HTTPS console?The steps in the following article do not apply to version 5.7.5:https://docs.nxlog.co/manager/current/installation/https.htmlPlease help. Thanks.Regards, Billy
billychua created
Nxlog Manager integrate with LDAPS
billychua created
I'm trying to integrate with LDAPS and do have some issue. I'm using the below document and below is the command to import LDAPS certificate into the cacerts keystore. keytool -keystore <PATH_TO_JRE>/lib/security/cacerts -import -alias \ certificate -file <PATH_TO_CERTIFICATE>/certificate.cerWhen i try to import the certificate and they are asking for the keystore password. I assume this keystore was created during installation not sure what is the password.Need to check whether anyone know what is the keystore password?https://docs.nxlog.co/manager/current/users/index.html
billychua created
Parse log with unicode characters hanging out
cschelin created
I'm attempting to parse a Cerberus FTP log file. What I wind up with:{
"EventReceivedTime": "2024-08-01 16:11:37",
"SourceModuleName": "cerberus_log",
"SourceModuleType": "im_file",
"message": "[\u00002\u00000\u00002\u00004\u0000-\u00000\u00008\u0000-\u00000\u00001\u0000 \u00001\u00006\u0000:\u00001\u00001\u0000:\u00003\u00006\u0000]\u0000:\u0000C\u0000O\u0000N\u0000N\u0000E\u0000C\u0000T\u0000 \u0000[\u00001\u00005\u00002\u00004\u00009\u00002\u0000]\u0000 \u0000-\u0000 \u0000C\u0000o\u0000n\u0000n\u0000e\u0000c\u0000t\u0000i\u0000o\u0000n\u0000 \u0000t\u0000e\u0000r\u0000m\u0000i\u0000n\u0000a\u0000t\u0000e\u0000d\u0000"
}I've tried this, to no avail:<Input cerberus_log>
Module im_file
File "C:\ProgramData\Cerberus LLC\Cerberus FTP Server\log\server.1.log"
<Exec>
$message = convert($raw_event, "utf-8", "iso8859-2"); if $message =~ s/(.)\\u0000// $message = $1;
to_json();
</Exec>
</Input>How can I properly parse the log to remove the \u0000 characters before it goes out?
cschelin created
Logs from centralised solution does not pass host field in some instances
esky created
We have a distributed solution and a centralised solutionBoth send events to Splunk (I'm the Splunk Admin)100% of the distributed events have the host field present.About 50% of the centralised events have the host field missing and show :Hostname: ?Any idea why this would be? is this a misconfiguration on the centralised host somewhere? or on the agentless side?Module:SourceModuleName: in_audit_pipe SourceModuleType: im_pipeThanks.
esky created
im_msvistalog --> Exec if or drop statement understanding problem
hkrischeu created
Hi,I am using this code snipping<Input EventLOG>Module im_msvistalogExec if ($TargetUserName == 'SYSTEM') OR ($EventType == 'VERBOSE') drop();</Input>This is working fine with my nx-Client at Windows 11. It push notification at my debian server with installed rsyslog server.But I want to have some specific ID´s from Windows Eventlog.( e.g. 5013, 10016, 4616, 6869)Can anybody point me to the right way? Thanks in Advance for every help. Heinz
hkrischeu created
im_azure using proxy to connect to Azure environment
mwaszut created
Hello,We are setting up log collection from Azure Log Analytics workspace but the connection is not possible without B2B proxy. I see that Proxy setup is possible only with om_azure module. We need to read and collect the logs from the Azure environment and not to send it there. What should we do to make it happen? Without proxy the connection is not possible and we can't use nxlog for our new service. Is there any other module which could set the proxy by default etc. ? Thank you,Martin W.
mwaszut created
One Input Multiple Outputs (AlienVault/Nxlog)
seppic created
We currently have a central logging server for our Windows Servers collecting and forwarding with NXLog to AlienVault. We have a working config file currently that I would like to modify to be able to send two of the Windows Event ID's that are being collected to our NDR (Vectra). What is the best route to be able to configure multiple sources and/or outputs? Currently we have two working configs, one for Vectra and one for AlienVault, but I'd like to “combine” them in a way that allows us to be able to send relevant data to their needed sources.AlienVault uses a patterndb.xml file for what events it wants to collect, Vectra just needs two specific event ID's that it calls out in the log file with the below line. It then outputs to an IP. This seems like it should be pretty straight forward but I'd like to have it sort of configured/figured out before I bring down my AlienVault feed.We are also currently using the Nxlog community edition if that matters.<Input eventlog> Module im_msvistalog<QueryXML> <QueryList> <Query Id="0" Path="ForwardedEvents"> <Select Path="ForwardedEvents">*[System[(EventID=4768 or EventID=4769)]]</Select> </Query></QueryList></QueryXML></Input>
seppic created
Windows Event Log - Drop/Filter
jacob.omara@doubleline.com created
All,Hopefully an easy question.I am currently collecting Windows event logs on a dedicated forwarding server (using native WEF) in a dedicated event log (named “Forwarded Events”). I have NxLog installed on this server and logs are being sent properly to my SIEM.Currently I am having difficulties filtering events where the SubjectUserName field ends with “$” symbol (logs are still reaching my SIEM). Below is a snippet of my configuration. I am running NXLog Enterprise version 6.2.Please advise.<Extension json>
Module xm_json
</Extension>
<Input windows_security_eventlog>
Module im_msvistalog
ReadFromLast True
SavePos True
<QueryXML>
<QueryList>
<Query Id="0">
<Select Path="ForwardedEvents">*</Select>
</Query>
</QueryList>
</QueryXML>
<Exec>
###################################################
# Drop noisy machine object access (4662) events.#
###################################################
if $EventID == 4662 AND ($SubjectUserName =~ /(.)$/) drop();
</Exec>
</Input>
jacob.omara@doubleline.com created
Running Powershell script on a schedule to retrieve and format Windows Event logs and send to logstash output
MCon30318 created
I have a Powershell script that is retrieving events from Windows Event logs that are written by a certain application. It then parses the exception info from the Event_Data portion into separate fields and combines them into JSON (the exception info is written in one big block of text, but each line has fields that we want to separate out into distinct fields so that the data is easier to filter in Kibana. I have an input im_exec module in nxlog.conf that runs that script and a route to send that input to an om_tcp output for our logstash instance. This works fine when you restart the service, however I'm having a problem getting this to pick up events after the initial restart. I've tried adding a schedule both to the input module and adding an xm_exec module to restart the input module on a scheduled basis. Anyone have any ideas on how to get this script to run repeatedly so that i can pick up and format new events? I've looked through the schedule documentation and tried some of the suggestions on other discussions that look a bit similar, but none seem to have worked so far. This is what I have in the conf file currently: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO <Extension json> Module xm_json </Extension> define LogFile %ROOT%\data\nxlog.log <Input input_logs_powershell> Module im_exec Restart true Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "-ExecutionPolicy" Arg "Bypass" Arg "-NoProfile" # This specifies the path to the PowerShell script. Arg "-File" Arg "D:\Temp\events_parser.ps1"<Exec> # Parse JSON parse_json(); </Exec></Input> <Output logs_to_Kibana> Module om_tcp Host logstash_hostname Port 6710 exec to_json(); </Output><Route input_logs_powershell>PATH input_logs_powershell =>logs_to_Kibana</Route>
MCon30318 created
Regarding PaloAlto Panorama (syslog) Logs
jacob.omara@doubleline.com created
New Enterprise NxLog customer here…..hopefully any easy question.Today I am ingesting syslog messages from my PaloAlto Panorama instance into a dedicated syslog (Ununtu) server running syslog-ng. I am using syslog-ng to parse the incoming logs into 3 distinct log files (traffic, threat, and system). I am then using “logrotate” and “cron" to rotate, gzip, and retain the logs.I figure I have 2 options in terms of the log files themselves now that I am an nxlog customer.Option 1: Keep things as-is (since it is working now) and just use “im_file”.Option 2: Use nxlog to do the same things I am with syslog-ng. Being new to nxlog, not sure how to best do this.If I want to go with Option #2, does anyone have a working configuration they would be willing to share on how they parsed the incoming syslog messages from Palo Alto into those 3 distinct files (or came up with a better alternative)? Thank you.
jacob.omara@doubleline.com created
Errors when obtaining logs from Office365.
oscar.cerna@threatshieldsecurity.com created
Hello team.We have followed the steps from this link: https://docs.nxlog.co/refman/v5.10/im/ms365.html#config-certkeyfile to obtain the logs from Office365.The following permissions have been applied:However, I am encountering the following errors when trying to extract the information: 2024-05-01 18:59:31 WARNING [im_ms365|microsoft_365] Retrying request Audit.Exchange, attempt 33 failed, error: {"error":{"code":"AF20055","message":"Start time and end time must both be specified (or both omitted) and must be less than or equal to 24 hours apart, with the start time prior to end time and start time no more than 7 days in the past. StartTime:2024-05-01T22:48:58Z, EndTime:2024-05-01T21:21:47Z"}}. (Retrying in 200 seconds).
2024-05-01 18:59:34 WARNING [im_ms365|microsoft_365] Retrying request HealthOverviewsWithServiceHealthIssues, attempt 33 failed, error: {"error":{"code":"UnknownError","message":"","innerError":{"date":"2024-05-01T22:59:34","request-id":"20d6e12b-eb61-4b2b-bffa-b69f8f5c4847","client-request-id":"20d6e12b-eb61-4b2b-bffa-b69f8f5c4847"}}}. (Retrying in 200 seconds).
2024-05-01 18:59:36 WARNING [im_ms365|microsoft_365] Retrying request AuditEvents, attempt 33 failed, error: {"error":{"code":"Forbidden","message":"{\r\n \"_version\": 3,\r\n \"Message\": \"Application is not authorized to perform this operation. Application must have one of the following scopes: DeviceManagementApps.Read.All, DeviceManagementApps.ReadWrite.All - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: e188947b-1412-4a09-9b64-548de1c1f6a6 - Url: https://fef.amsua0602.manage.microsoft.com/StatelessAuditingFEService/deviceManagement/auditEvents?api-version=5022-08-30&$filter=activityDateTime+ge+2024-04-01T21%3a21%3a47Z+and+activityDateTime+lt+2024-05-01T21%3a21%3a47Z&$top=50\",\r\n \"CustomApiErrorPhrase\": \"\",\r\n \"RetryAfter\": null,\r\n \"ErrorSourceService\": \"\",\r\n \"HttpHeaders\": \"{}\"\r\n}","innerError":{"date":"2024-05-01T22:59:36","request-id":"e188947b-1412-4a09-9b64-548de1c1f6a6","client-request-id":"e188947b-1412-4a09-9b64-548de1c1f6a
2024-05-01 18:59:44 WARNING [im_ms365|microsoft_365] Retrying request SignIns, attempt 33 failed, error: {"error":{"code":"Authentication_RequestFromNonPremiumTenantOrB2CTenant","message":"Neither tenant is B2C or tenant doesn't have premium license","innerError":{"date":"2024-05-01T22:59:44","request-id":"6eb5223c-948f-42af-b28b-bbf3fbea96fb","client-request-id":"6eb5223c-948f-42af-b28b-bbf3fbea96fb"}}}. (Retrying in 200 seconds).
2024-05-01 19:00:05 WARNING [im_ms365|microsoft_365] Retrying request ReportingWebService/MessageTrace, attempt 33 failed, error: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">; <html xmlns="http://www.w3.org/1999/xhtml">; <head>; <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>; <title>401 - Unauthorized: Access is denied due to invalid credentials.</title>; <style type="text/css">; <!--; body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}; fieldset{padding:0 15px 10px 15px;} ; h1{font-size:2.4em;margin:0;color:#FFF;}; h2{font-size:1.7em;margin:0;color:#CC0000;} ; h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} ; #header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;; background-color:#555555;}; #content{margin:0 0 0 2%;position:relative;}; .content-container{background:#FFF;width:96%;margin-top:8px;padding:1 Could you help me understand these errors?
oscar.cerna@threatshieldsecurity.com created
nxlog could not be stopped - error during installation
sa250367 created
I uninstalled the community version and trying to install the enterprise version, I am getting this error - Service nxlog could not be stopped, verify that you have sufficient privileges to stop system services. I am a global admin. Please help
sa250367 created
Are there any sample log files for NXLog Manager available to test?
JW created
Looking to test some ingest into a data lake to test searches adn dashboards.
JW created
DROP messages, if contains STRING SOMEWHERE
denny.fuchs@inatec.com created
hi,I try to DROP all messages, if they contains somewhere “/connection_status” or “/status”, but what ever I try, the filter won't fit on Nxlog, while it works in RegexTesterExample log:Apr 25 11:15:11 nomad-cde cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981: {"Hostname":"nomad-cde","ShortMessage":{"message":"Matched route \"status\".","context":{"route":"status","route_parameters":{"_route":"status","_controller":"App\\Controller\\Monitoring\\MonitoringController::getStatus"},"request_uri":"https://hpp.example.com/status","method":"GET"},"level":200,"level_name":"INFO","channel":"request","datetime":"2024-04-25T09:15:10.999734+00:00","extra":{}},"EventTime":"2024-04-25T11:15:11.000000+02:00","SeverityValue":6,"command":"/home/app/entrypoint.sh start php-fpm","container_id":"2baaa8302059bbcb881aac339807d429b1d91e1a117659e37cb4edfb7bb1eeca","container_name":"cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981","image_id":"sha256:7ddb43de759bd8ef93975cf10b25289a4f8628587b6d31cbe3d16e8d39443572","image_name":"harbor.example.com/testing/hpp/test:433e16c7","tag":"production","EventReceivedTime":"2024-04-25T11:15:11.001689+02:00","SourceModuleName":"container","SourceName":"cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981","SyslogFacility":6}orApr 25 11:15:09 nomad-cde cde_iframes-3709bf3d-86a1-0264-fe9f-150ac2b14cdd: {"Hostname":"nomad-cde","ShortMessage":"172.16.0.40 - - [25/Apr/2024:09:15:09 +0000] \"GET /status HTTP/1.0\" 200 2 \"-\" \"-\" \"172.16.1.40\"","EventTime":"2024-04-25T11:15:09.479000+02:00","SeverityValue":6,"command":"/home/app/entrypoint.sh start php-fpm","container_id":"c8b7c9357b1bc195f6d88d09e4c329627bfe165debc09cfe4bbfd556fdab966c","container_name":"cde_iframes-3709bf3d-86a1-0264-fe9f-150ac2b14cdd","image_id":"sha256:be421273041ffa5d7b8be4963f91c0376d9829ba942b86341413c59105ae671c","image_name":"harbor.example.com/testing/iframes/test:3cb57629","tag":"production","EventReceivedTime":"2024-04-25T11:15:09.524068+02:00","SourceModuleName":"container","SourceName":"cde_iframes-3709bf3d-86a1-0264-fe9f-150ac2b14cdd","SyslogFacility":6}orApr 25 11:15:09 nomad-cde cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981: {"Hostname":"nomad-cde","ShortMessage":{"message":"Matched route \"connection_status\".","context":{"route":"connection_status","route_parameters":{"_route":"connection_status","_controller":"App\\Controller\\Monitoring\\MonitoringController::getStatusDB"},"request_uri":"https://web:4433/connection_status","method":"GET"},"level":200,"level_name":"INFO","channel":"request","datetime":"2024-04-25T09:15:09.603963+00:00","extra":{}},"EventTime":"2024-04-25T11:15:09.605000+02:00","SeverityValue":6,"command":"/home/app/entrypoint.sh start php-fpm","container_id":"2baaa8302059bbcb881aac339807d429b1d91e1a117659e37cb4edfb7bb1eeca","container_name":"cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981","image_id":"sha256:7ddb43de759bd8ef93975cf10b25289a4f8628587b6d31cbe3d16e8d39443572","image_name":"harbor.example.com/testing/hpp/test:433e16c7","tag":"production","EventReceivedTime":"2024-04-25T11:15:09.634920+02:00","SourceModuleName":"container","SourceName":"cde_hpp-c7d794e5-6297-e29a-81d4-c284f52e8981","SyslogFacility":6}I tried in the end the simplest one:if $raw_event =~ /.*status.*/ drop(); but it does not match. The config looks like this:
<Output syslog-container-server>
Module om_udp
Host ${user.logserver}
Port 514
<Exec>
if $raw_event =~ /.*status.*/ drop();
$Hostname = "nomad-cde";
$message =~ s/-p[^\s]+/-pXXX/;
delete($SourceModuleType);
delete($MessageSourceAddress);
delete($version);
delete($created);
$SourceName = $container_name;
$SyslogFacility = $SeverityValue;
to_json();
to_syslog_bsd();
</Exec>
</Output>Can someone give me an hint, where I have to look ?
denny.fuchs@inatec.com created