For event 4688 I can only extract the parent process id out of the message body but there is no field in JSON containing only the parent process id. In im_msvistalog this field is properly added to JSON. Example in im_wseventing:

{
"EventTime":"2020-11-23 00:25:57",
"Hostname":"...",
"Keywords":-9214364837600034816,
"EventType":"AUDIT_SUCCESS",
"SeverityValue":2,
"Severity":"INFO",
"EventID":4688,
"SourceName":"Microsoft-Windows-Security-Auditing",
"ProviderGuid":"{ANONYMIZED}",
"Version":2,
"Task":13312,
"OpcodeValue":0,
"RecordNumber":14699970,
"ProcessID":4,
"ThreadID":7924,
"Channel":"Security",
"Message":"A new process has been created.\r\n\r\nCreator Subject:\r\n\tSecurity ID:\t\tS-1-5-18\r\n\tAccount Name:\t\ANON$\r\n\tAccount Domain:\t\ANON\r\n\tLogon ID:\t\tANON\r\n\r\nTarget Subject:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\t-\r\n\tAccount Domain:\t\t-\r\n\tLogon ID:\t\t0x0\r\n\r\nProcess Information:\r\n\tNew Process ID:\t\t0x114c\r\n\tNew Process Name:\tC:\\Windows\\System32\\cmd.exe\r\n\tToken Elevation Type:\tTokenElevationTypeDefault (1)\r\n\tCreator Process ID:\t0x1f8c\r\n\tProcess Command Line:\t\r\n\r\nToken Elevation Type indicates the type of token that was assigned to the new process in accordance with User Account Control policy.\r\n\r\nType 1 is a full token with no privileges removed or groups disabled. A full token is only used if User Account Control is disabled or if the user is the built-in Administrator account or a service account.\r\n\r\nType 2 is an elevated token with no privileges removed or groups disabled. An elevated token is used when User Account Control is enabled and the user chooses to start the program using Run as administrator. An elevated token is also used when an application is configured to always require administrative privilege or to always require maximum privilege, and the user is a member of the Administrators group.\r\n\r\nType 3 is a limited token with administrative privileges removed and administrative groups disabled. The limited token is used when User Account Control is enabled, the application does not require administrative privilege, and the user does not choose to start the program using Run as administrator.",
"Category":"Process Creation",
"Opcode":"Info",
"SubjectUserSid":"ANON",
"SubjectUserName":"ANON",
"SubjectDomainName":"ANON",
"SubjectLogonId":"ANON",
"NewProcessId":"0x114c",
"NewProcessName":"C:\\Windows\\System32\\cmd.exe",
"TokenElevationType":"%%1936",
"TargetUserSid":"S-1-0-0",
"TargetUserName":"-",
"TargetDomainName":"-",
"TargetLogonId":"0x0",
"EventReceivedTime":"2020-11-23 02:13:00",
"SourceModuleName":"security_event_collect",
"SourceModuleType":"im_msvistalog"}

Please add the missing field.

Best regards Theo

When I check the Windows "Services", it is running, but the logs are not transferred. After restarting the service, it works fine for a few days. After a few days, the logs will not be transferred. I think that "★" is because the transfer destination server is temporarily offline. If I can't connect even once, can I connect after that?

2020-11-12 14:41:17 INFOnxlog-ce-2.10.2150 has started ★ 2020-11-12 14:41:17 Error 10.17.140.209 Failed to connect to udp socket: 514; An attempt was made to perform a socket operation on an unreachable host. 2020-11-2009: 56: 06 Warning nxlog service outage 2020-11-2009: 56: 06 Warning nxlog-ce received end request signal and ended ... 2020-11-20 09:56:08 INFOnxlog-ce-2.10.2150 has started

Hello everyone. I'm new to NXLog but I'm glad to be here and to learn.

We have the Enterprise edition and I'm trying to work out how the template structure works. The user doc (135.3. Creating Templates) references the image below.

However when I try and create a template, I don't have the label or drop down to select MASTER / SUB as a template type as shown in the screenshot. --> Missing Template Option

Is there something in the config to turn that function on and off that I might be missing? For reference, we're using version 5.5.5398 of the NXLog Manager via Docker image.

I'm looking at a slightly unusual application logging which has turned out quite challenging to handle with NXLog, as is, and for that I've been experimenting of running PS scripts using NXLog. In principle, I'd like to know if it is possible to build the following scenario using NXLog Enterprise agent.

Running of PS script (using NXLog) to fetch log files at interval from variable directories and putting them into another (a copy of logs not older than 1 hour, PS script would manage this, but needs to be invoked by NXLog agent). Reading selected events from the fetched logs and dispatching them to another system (note, this is completed in another scenario already so I know this would work). Deleting of all logs from the import directory after they have been read. This could be managed with the xm_fileop, I believe.

I have been experimenting of running PS scripts, unsuccesfully so far, but I'm going through the docs and examples to understand how would one execute a (any) script using the NXLog agent.

Any advice will be highly appreciated.

Hi team, Can someone please help me with parsing the below tomcat log (contains multiple lines) ? **tomcat log snippet: ** 19-Aug-2020 12:39:51.412 INFO [Catalina-utility-2] org.apache.catalina.startup.HostConfig.undeploy Undeploying context [/front] 19-Aug-2020 12:40:31.462 INFO [Catalina-utility-2] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [D:\Tomcat\webapps\front] 19-Aug-2020 12:40:32.813 SEVERE [Catalina-utility-2] org.apache.catalina.startup.HostConfig.deployDirectory Error deploying web application directory [D:\Tomcat\webapps\front] java.lang.IllegalStateException: Error starting child at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:720) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705) at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1133) at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:1866) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) I am trying to use the same example (url below) from nxlog official guide, but didn't help. https://nxlog.co/documentation/nxlog-user-guide/apache-tomcat.html **Please see my conf file content below: ** define REGEX /(?x)^(?\d{2}\-\d{3}\-\d{4}\ \d{2}\:\d{2}\:\d{2}).\d{3}\ \ (?\S+)\ \[(?\S+)\]\ \(?[\s\S]+)/ Module xm_multiline HeaderLine %REGEX% Module im_file File 'D:\Tomcat\logs\catalina.*.log' SavePos TRUE InputType multiline Exec if $raw_event =~ %REGEX% $EventTime = parsedate($EventTime); log_info($raw_event); log_info($Message); Exec $Message = 'TOMCAT_Catalina ' + $raw_event ; $SyslogFacilityValue = 1; $SyslogSeverityValue=5; **And getting below errors: ** 2020-11-13 14:25:54 ERROR failed to compile regular expression '(?x)^(?\d{2}\-\d{3}\-\d{4}\ \d{2}\:\d{2}\:\d{2}).\d{3}\ (?\S+)\ \[(?\S+)\]\ \(?[\s\S]+)', error at position 136: unmatched parentheses 2020-11-13 14:25:54 ERROR invalid expression in 'HeaderLine' at C:\Program Files (x86)\nxlog\conf\custom\\tomcatCatalina.conf:7 2020-11-13 14:25:54 ERROR Invalid InputType 'multiline' at C:\Program Files (x86)\nxlog\conf\custom\\tomcatCatalina.conf:19 2020-11-13 14:25:54 ERROR module 'tomcat_catalina_log' has configuration errors, not adding to route '4' at C:\Program Files (x86)\nxlog\conf\custom\\tomcatCatalina.conf:53 2020-11-13 14:25:54 ERROR route 4 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\custom\\tomcatCatalina.conf:53 Please help. Thanks in advance!

Hi,

I posted a question on september regarding the release of nxlog for debian buster:

https://nxlog.co/question/6073/nxlog-community-edition-package-debian-buster

Raf answered that the release will be available soon, it has been 2 months, have you any idea of the release date ?

Thanks in advance, Best regards.

Paul

Hello,

I'm using NXlog CE 2.10.2150 on a Win2016 server to collect "Forwarded Events" and send to a syslog server as snare formatted. However, some events only contain their System segment, missing their entire EventData.

Here's my configuration (EventData Missing):

<Input eventlog> Module im_msvistalog Query <QueryList>
<Query Id="">
<Select Path="ForwardedEvents">*</Select>
</Query>
</QueryList> </Input>

<Output out> Module om_tcp Host ip_syslogs_server Port 514 Exec to_syslog_snare(); </Output>

<Route 1> Path eventlog => out </Route>

Resulting in (tcpdump): <14>Nov 11 10:50:37 server2.domain MSWinEventLog 1 Security 2189 Wed Nov 11 10:50:37 2020 4768 Microsoft-Windows-Security-Auditing N/A N/A Success Audit server2.domain Kerberos Authentication Service N/A 998061427

Here's my configuration (EventData Included):

<Input eventlog> Module im_msvistalog Query <QueryList>
<Query Id="">
<Select Path="ForwardedEvents">*</Select>
</Query>
</QueryList> Exec $Message =~ s/(\t|\R)/ /g; Exec $Message = to_json(); </Input>

<Output out> Module om_tcp Host ip_syslogs_server Port 514 Exec to_syslog_snare(); </Output>

<Route 1> Path eventlog => out </Route>

Resulting in (tcpdump): <14>Nov 11 10:49:36 server1 MSWinEventLog 1 Security 1593 Wed Nov 11 10:49:36 2020 4768 Microsoft-Windows-Security-Auditing N/A N/A Success Audit server1 Kerberos Authentication Service {"EventTime":"2020-11-11 10:49:36","Hostname":"server1","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":4768,"SourceName":"Microsoft-Windows-Security-Auditing","ProviderGuid":"{54849625-5478-4994-A5BA-3E3B0328CXXX}","Version":0,"Task":14339,"OpcodeValue":0,"RecordNumber":953757340,"ProcessID":708,"ThreadID":11484,"Channel":"Security","ERROR_EVT_UNRESOLVED":true,"Category":"Kerberos Authentication Service","Opcode":"Info","TargetUserName":"User1","TargetDomainName":"domain","TargetSid":"S-1-5-21-3493186346-123456789-198542525-123456","ServiceName":"krbtgt","ServiceSid":"S-1-5-21-123456789-2449186506-123456525-502","TicketOptions":"0x40000000","Status":"0x0","TicketEncryptionType":"0x12","PreAuthType":"2","IpAddress":"10.xxx.xxx.xxx","IpPort":"33925","EventReceivedTime":"2020-11-11 10:49:38","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog"} 953757340

As you can see NXLog fills the EventData with "N/A" in my first example and wiht JSON Encoded Full EventData in my Second Example. The Problem I have is my Parser on the syslogserver does not expect JSON Encoded Data.

Any Hints? Thx Johannes

Hello, I'm logging event logs from a custom c++ app to a server and am trying to setup file cycling for both the application event logs and NXLog log file. When I add a second Output to my Route, I start to see the event logs showing up in the NXLog log file, which I wasn't expecting. I'm not sure how to approach this ... should I be setting up a second path for the NXLog cycling? Here's what my config file is looking like:

Keep 2 weeks of app log files

<Output app_log_cycle> Module om_file File 'C:/Users/Jeremy/Documents/myApp/myApp.log' <Schedule> When @daily <Exec> file_cycle(file_name(), 7); app_log_cycle->reopen(); </Exec> </Schedule> </Output>

Keep 2 weeks of nxlog log files

<Output nxlog_log_cycle> Module om_file File 'C:/Program Files (x86)/nxlog/data/nxlog.log' <Schedule> When @daily <Exec> file_cycle(file_name(), 14); nxlog_log_cycle->reopen(); </Exec> </Schedule> </Output> <Route 1> Path watchfile => syslogout, app_log_cycle, nxlog_log_cycle </Route> Thanks++ for any tips! Jeremy

Hey everyone!

I'm attempting to use the om_ssl module on an NXLog Community Edition but checking the logs at "C:\Program Files (x86)\nxlog\data\nxlog" showed the following message: "ERROR invalid keyword: CAThumbprint"

After not finding anything about the error above, I decided to use a combination of CAFile, CertFile, CertKeyFile and KeyPass on the Output configuration which worked but I'd rather use the CAThumbprint directive. What am I doing wrong?

Output tag from the "C:\Program Files (x86)\nxlog\conf\nxlog.conf" looked like this: <Output out> Module om_ssl CAThumbprint xxxxxxxxxxxxxxxxxxxxxxxxxxxx # numbers and letters, without spaces Host 10.0.0.10 # representative IP Port 1514 Exec to_syslog_bsd(); </Output>

Hi, I have a log source (Cisco ironport) that sends some multi-line logs via syslog and I want to join only one log when the ICID is the same in the first case and join only one log when the MID is the same in the second case . But I want to remove part of the log. Example below. Is it possible to do this?

1. join when ICID is equal <22>Oct 07 22:16:47 TextMailLogs: Info: New SMTP ICID 929637136 interface Public_SMTP (yy.yy.yy.yy) address xx.xx.xx.xx reverse dns host host@isp.com verified yes <22>Oct 07 22:16:47 TextMailLogs: Info: ICID 929637136 RELAY SG policy match .isp.com SBRS not enabled <22>Oct 07 22:16:47 TextMailLogs: Info: SMTP Auth: (ICID 929637136) succeeded for user: user@isp.com using AUTH mechanism: PLAIN with profile: SMTP_AUTH_CP <22>Oct 07 22:16:48 TextMailLogs: Info: ICID 929637136 lost <22>Oct 07 22:16:48 TextMailLogs: Info: ICID 929637136 close

example result: <22>Oct 07 22:16:47 TextMailLogs: Info: New SMTP ICID 929637136 interface Public_SMTP (yy.yy.yy.yy) address xx.xx.xx.xx reverse dns host host@isp.com verified yes RELAY SG policy match .isp.com SBRS not enabled succeeded for user: user@isp.com using AUTH mechanism: PLAIN with profile: SMTP_AUTH_CP lost close

2. join when MID is equal <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 From: <from@isp.com> <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 RID 0 To: <to@isp.com> <22>Apr 12 12:00:36 TextMailLogs: Info: MID 354020393 Subject '=?UTF-8?B?V29tZW7'
   <22>Apr 12 12:00:36 TextMailLogs: Info: MID 354020393 ready 92209 bytes from <from@isp.com> <22>Apr 12 12:00:53 TextMailLogs: Info: Message finished MID 354020393 done

example result: <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 From: <from@isp.com> RID 0 To: <to@isp.com> Subject '=?UTF-8?B?V29tZW7' ready 92209 bytes from <from@isp.com> done

Thanks

Hey guys,

We tested nxlog on a few servers and everything worked fine, no problem at all. Now when we deployed it to more, after 2 hours, the clients just keep disconnecting and reconnecting. I'm really not sure what's happening. We're using a self signed cert, made with opeenssl. Not sure what would you need to identify my problem. If needed I'll copy over the debug log or conf file.

Environment is windows.

Any help is appreciated!

hi all Fairly simple one i think we send way too many "informational" Messages from a few of our Windows servers to Loggly i want to simply stop sending those "informational" messages any help would be appreciated Dale

Q1. Hello I have one problem about mysql to graylog server

i want my db (version:mariadb 5.5 OS centos 7 ) "SELECT eventlog" to my graylog server

but my nxlog log have error the message is: ERROR im_dbi failed to execute SQL statement. 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE id > -1 LIMIT 10' at line 1

my nxlog.conf

<Extension _syslog> Module xm_syslog </Extension>

<Input in> Module im_dbi Driver mysql Option host 127.0.0.1 Option username root Option password librenms Option dbname librenms SQL SELECT eventlog </Input>

<Output out> Module om_udp Host 192.168.1.20 Port 10514 Exec to_syslog_snare(); </Output>

<Route 1> Path in => out </Route>

so how can i do or where are reference about nxlog for mysql(mariadb) ?

Q2. can i install nxlog-ce to CentOS8?

How would I go about statically assigning the value of x-forward-for the end of an IIS message that is being read through IM_FIle?

I am doing testing of IIS Log files and I want to take a value from an internal system and add the x-forward-for field to the message.

I have tried adding it to the Input on im_file

Exec $Message = $x_forwarded_for + ' ' + 'x.x.x.x';

I have tried it as

$raw_event = $raw_event + 'x.x.x.x'

Either the value never shows up or it ends up outside of the message. What I need is for it to be inside the message. Essentially where the time-taken is on this example message I would end up with my static x-forwarded-for entry.

#Software: Microsoft Internet Information Services 10.0 #Version: 1.0 #Date: 2017-10-02 17:11:27 #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken 2017-10-02 17:11:27 fe80::b5d8:132c:cec9:daef%6 RPC_IN_DATA /rpc/rpcproxy.dll 1d4026cb-6730-43bf-91eb-df80f41c050f@test.com:6001&CorrelationID=<empty>;&RequestId=11d6a78a-7c34-4f43-9400-ad23b114aa62&cafeReqId=11d6a78a-7c34-4f43-9400-ad23b114aa62; 80 TEST\HealthMailbox418406e fe80::b5d8:132c:cec9:daef%6 MSRPC - 500 0 0 7990 2017-10-02 17:12:57 fe80::a425:345a:7143:3b15%2 POST /powershell clientApplication=ActiveMonitor;PSVersion=5.1.14393.1715 80 - fe80::a425:345a:7143:3b15%2 Microsoft+WinRM+Client - 500 0 0 11279

Using one dedicated NXlog node as master collector that forwards logs to SIEM, I also dump all logs into a file and I rotate that file weekly, keeping 4 weeks worth of logs on the collector; this very clean setup is taken from the documentation and I like it - it works well (see Log rotate in documentation, example 103). Problem is: This consumes much disk, so I'll need to compress the rotated file. Can I add compression to that output to compress the rotate-copy? I'd prefer not using the example 113 in said log-rotation documentation page.

I'm fairly new to this, but wanted to get some pointers. I've been provided with a syslog.conf file, and am trying to determine what events are being logged. Can someone please let me know what I should be looking for? Alternatively if the syslog.conf file doesn't contain this info, where can I find what events are being logged? On the other hand, is there a standard list of event types that are automatically logged?

Hi,

I am new to NXLog and I am setting up relay log servers in IP ranges to collect and forward log events from hosts (different OS) with dynamically assigend IP addresses.

While doing so, I figured out that the forwarded log events missing important details about the orginating systems and I like to enrich the log events while receiving (accepting ) them with the following details:

• hostname as SOURCE_HOSTNAME
• ipv4 address as SOURCE_HOSTIP

How is the best practice to do so?

Rgds, Thorsten

hi who i can get iis web server logs and then send them to ossim log server?

Hello Guys, Im currently trialing nxlog enterprise version and specifically interested in agentless version of it. While working on it i have come across a blockade which seems to not forward logs from windows server machine to linux windows event collector(nxlog enterprise version is running). Im running nxlog version 5.1.6133 on ubuntu. Here are the details (SERVER details w.r.t im_wseventing module documentation) NAME="Ubuntu" VERSION="18.04.5 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.5 LTS" VERSION_ID="18.04" Below are client details which forwards logs to server above using agentless method OS Name: Microsoft Windows Server 2019 Datacenter OS Version: 10.0.17763 N/A Build 17763 OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Server Im getting following error message while trying to forward logs. Seeing this error in eventviewer under eventlog forwarding plugin. I have followed the documentation and generated certs and certs seems to be valid and working. Log Name: Microsoft-Windows-Forwarding/Operational Source: Microsoft-Windows-Forwarding Date: 10/28/2020 2:37:05 AM Event ID: 105 Task Category: None Level: Error Keywords: User: NETWORK SERVICE Computer: computer_name Description: The forwarder is having a problem communicating with subscription manager at address HTTPS://private_ip_address:5986/wsman/. Error code is 2150858819 and Error Message is 鿰柣ƴ. Event Xml: 105 0 2 0 0 0x8000000000000000 438 Microsoft-Windows-Forwarding/Operational computer_name HTTPS://private_ip_address:5986/wsman/ 2150858819 鿰柣ƴ I have tried even port 5985 and it also faces similar error. I have checked whether the server which is the ubuntu is reachable or not, did GET request using postman api tool.POST request seems to not work which might be the cause which might be behind this problem. Below is my nxlog config Module im_wseventing ListenAddr 0.0.0.0 Port 5986 Address https://private_ip_address:5986/wsman HTTPSCertFile %CERTDIR%/server-cert.pem HTTPSCertKeyFile %CERTDIR%/server-key.pem HTTPSCAFile %CERTDIR%/ca-cert.pem * * * * * * Module om_file File "/opt/nxlog/var/log/nxlog/windows_events.log" Path wseventing => file Let me know how do i overcome this issue if possible. Thanks.

Hello

windows ---> nxlog-------->graylog&splunk(syslog)

how can i do?
this conf is right?

p.s. graylog 192.168.1.20 splunk 192.168.1.21

------------------------------------My conf------------------------------------------------------------------ Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_gelf

</Extension>

<Extension _syslog> Module xm_syslog

</Extension>

<Input in>

Module im_msvistalog

</Input>

<Output out> Module om_udp Host 192.168.1.20 Port 10554 OutputType GELF </Output>

<Output out1> Module om_udp Host 192.168.1.21 Port 10554 Exec to_syslog_snare(); </Output>

<Route 1> Path in => out </Route>

<Route 2> Path in => out1 </Route>