I've been tasked to roll out nxlog on all of our Macs. I have it working in the sense that logs are being uploaded to our syslog server. However I've been given a list from https://www.iansresearch.com/resources/all-blogs/post/security-blog/2021/04/29/best-practices-for-macos-logging-monitoring and told to implement it. How the heck do I find out what eventtypes to filter so that I can capture the list of logs that is on this webpage. And is this list even the right one to follow? It mentions using Consolation 3, but I have no idea how that's supposed to help me figure this out. What is your goto source for this type of info?

It is possible to create dynamic directories per host I would like to separate all logs by host then year then month day I can create dynamic file names base on hostname $Hostname + "_" + month(now()) Thank you

Hello, i have the following problem and no Idea how to solve it: I have a Logline from our VPN Server which looks like this: 2021-11-11 16-56-00, connect, PartnerIP=1.1.1.1, VPNIP=172.16.123.123, User="Computername.Domain.de" My Problem is, that our SIEM System does not accept Computer Accounts for VPN Connections. It only allows User Accounts. Thats why i need to transform it into: 2021-11-11 16-56-00, connect, PartnerIP=1.1.1.1, VPNIP=172.16.123.123, User="Username@Domain.de" The connection between Computer and User is stored in a MS SQL Database or in a CSV/TXT File. Is it possible to replace the Computer Entry with the corresponding Username and then send it to a Syslog server? I did not found a possibility to load something like a key-Value List.

At the moment, I need a solution to a particular problem. I need to use a syslog relay agent to receive syslog data at the NXLog host, and relay it to another collector. This is to reduce the number of firewall rules needed to get syslogs from more secure areas of the network out to our SIEM. I would especially like to be able to have the source of the syslog be the same as the original source host - i.e. I don't want all 5 hosts for which I am relaying syslog data, to appear in the SIEM as if the source is the NXLog relay host. I am just beginning to look into this so any assistance is appreciated. Paul

Hi all, I am configuring NXLog CE to use Perl on Windows. I understand that NXLog requires Strawberry Perl 5.28.1 to be installed on Windows host. Quote from document: "To use the xm_perl module on Windows, a separate Perl environment must be installed. Currently, the only environment supported is a specific version of Strawberry Perl, 5.28.2.1. Newer versions will not work." I also have found in a user guide a requirement to include a lib path as a fisrt line of Perl script. Quote from document: "On Windows, the Perl script invoked by the PerlCode directive must define the Perl library paths at the beginning of the script to provide access to the Perl modules. nxlog-windows.pl" use lib 'c:\Program Files\nxlog\data'; I installed Strawberry Perl 5.28.1. I tried to put different folders as Perl lib folders, I tried use lib 'C:\Strawberry\perl\lib'; or use lib 'C:\Strawberry\perl\bin'; or use lib 'c:\Program Files\nxlog\data'; (doesn't contain libs, right?). However I still get this when running NXLog: 2021-11-09 14:45:54 ERROR Failed to load module from C:\Program Files (x86)\nxlog\modules\extension\xm_perl.dll, The specified module could not be found. ; The specified module could not be found. So what are actually steps to get Perl (xm_perl) on Windows working?

Hello, We are currently working on collecting the logs from McAfee EPO (without pooling the database ) using the agent as the syslog server . In the EPO we can configure a syslog server to send our events , in that case the syslog server is our agent We try some configuration but still not working <Extension clean_fields_mcafee_epo> Module xm_rewrite Keep time, host, source, sourcetype, event, classification, os, dc, severity, module, stanza, collector_hostname, collector_received_at </Extension> <Input i_mcafee_epo_raw> Module im_ssl ListenAddr 127.0.0.1:1572 CAFile "McAfeeEpo\Apache2\conf\ssl.crt\ca.cer" CertFile "McAfeeEpo\Apache2\conf\ssl.crt\cert.crt" CertKeyFile "McAfeeEpo\Apache2\conf\ssl.crt\cert.key" AllowUntrusted TRUE FlowControl TRUE <Exec> parse_syslog(); $source = "mcafee_epo"; log_info("event : " +$event); </Exec> </Input> <Output onul> Module om_null </Output> <Route r_raw_epo> Path i_mcafee_epo_raw => onul </Route> I was wondering I we need to use the module xm_syslog as the module

We have a process that writes log info to a txt file. The process writes a blank line before and after its processing. What can be done to drop these lines from being included when nxlog sends to the log aggregator that we use?

Wee need to collect logs from Internet Explorer and Powershell from a Windows Server Event Viewer. We use the im_vistalog but only the System, Application, and Security channels are showing up in our SIEM solution.

I'm using nxlog-ce-2.11.2190.msi (Community Edition) on Windows 2016 to send Windows Logs to a syslog server. If I put more than one Host to the <Output> section like <Output to_splunk> Module om_tcp Host abc1.corp.net Host abc2.corp.net Host abc3.corp.net Port 514 Exec to_syslog_ietf(); </Output> I get the following error when validating the configuration C:\Windows\system32>"C:\Program Files (x86)\nxlog\nxlog.exe" -v 2021-10-29 10:12:03 ERROR host is already defined at C:\Program Files (x86)\nxlog\conf\nxlog.conf:67 With just one Host defined, it works. What is false? Is the featuren not supported by the Community Edition? Regards, Martin

Windows Server 2019 NXLog: nxlog-ce 2.11.2190 Running the Community version to test /trial a SEIM platform (Enterprise will be acquired if the current PoC is selected). From 132.4.2. Example monitoring configurations I copied the code block in Example 644 into my nxlog.conf. After a bit of frustration, I pulled the current Server 2019 EventLog item list via PS> Get-WinEvent -ListLog and found that there were a few updates /changes. I made those changes and yet when I run nxlog, no logs are sent. Turning on DEBUG for LogLevel, I see: ERROR [im_msvistalog.c:1320/im_msvistalog_start()] -; [im_msvistalog.c:1285/im_msvistalog_start()] failed to subscribe to msvistalog events,the Query is invalid: The operation completed successfully.; [error code: 15001] After a bit of web searching for this error (completely fruitless) and reviewing the code block again, I can't see anywhere that a non-existent EventLog is being Selected, nor any common "typo" errors. Can anyone help me to figure out what "error code 15001" means and /or spot where I goofed in my nxlog.conf file? define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog-events.log LogLevel DEBUG #Load Extensions <Extension _syslog> Module xm_syslog </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _json> Module xm_json </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> # define Account Usage Events define AccountUsage 300, 1511, 1518, 4624, 4625, 4634, 4648, 4672, \ 4704, 4720, 4722, 4725, 4726, 4728, 4731, 4732, \ 4733, 4735, 4740, 4756, 4765, 4766, 4767, 4776, \ 4781, 4782, 4793, 5376, 5377 # define Application Crash Events define AppCrashes 1000, 1001, 1002 # define Application Whitelisting Events define AppWhitelisting 865, 866, 867, 868, 882, 4688, 4689, 8002, \ 8003, 8004, 8005, 8006, 8007, 8020, 8023 # define Boot Events define BootEvents 12, 13 # define Certificate Services Events define CertServices 95, 1001, 1002, 1003, 1004, 1006, 1007, 4870, \ 4873, 4874, 4885, 4886, 4887, 4890, 4896, 4899 # define Clearing Event Logs Events define ClearingLogs 104, 1100, 1102 # define DNS and Directory Services Events define DNSDirectoryServ 3008, 3020, 5136, 5137, 5138, 5139, 5141 # define External Media Detection events define ExtMedia 400, 410 # define Group Policy Error Events define GroupPolicyError 112, 1001, 1125, 1126, 1127, 1129 # define Software Service Installation Events define Installation 2, 6, 19, 800, 903, 904, 905, 906, 907, 908, \ 1022, 1033, 7000, 7045 # define Kernel Driver Signing Events define KernelDriver 219, 3001, 3002, 3003, 3004, 3010, 3023, 5038, \ 6281 # define Microsoft Cryptography API Events define MSFTCryptoAPI 11, 70, 90 # define Mobile Device Activities define MobileDeviceEvents 10000, 10001 # define Network Host Activities define NetworkHost 1024, 4706, 4713, 4714, 4716, 4719, 4769, 4778, \ 4779, 4897, 5140, 5142, 5144, 5145, 5632, 6272, \ 6273, 6274, 6275, 6276, 6277, 6278, 6279, 6280 # define PowerShell Activities define PowerShell 169, 800, 4103, 4104, 4105, 4106 # define Printing Services Events define PrintingServices 307 # define System Integrity Events define SystemIntegrity 1, 2, 5, 8, 9, 4616, 4657 # define System or Service Failure Events define SystemServiceFail 7022, 7023, 7024, 7026, 7031, 7032, 7034 # define Windows Defender Activities define WinDefender 1005, 1006, 1007, 1008, 1009, 1010, 1116, 1117, \ 1118, 1119, 2001, 2003, 2004, 3002, 5008 # define Windows Firewall Events define WinFirewall 2005, 2006, 2009, 2033 # define Windows Update Error Events define WinUpdateError 20, 25, 31, 34, 35 <Input Events> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Application">*</Select> <Select Path="Security">*</Select> <Select Path="System">*</Select> <Select Path="Windows PowerShell">*</Select> <Select Path="Microsoft-Windows-AppLocker/EXE and DLL">*</Select> <Select Path="Microsoft-Windows-AppLocker/MSI and Script">*</Select> <Select Path="Microsoft-Windows-AppLocker/Packaged app-Deployment">*</Select> <Select Path="Microsoft-Windows-AppLocker/Packaged app-Execution">*</Select> <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Assistant">*</Select> <Select Path="Microsoft-Windows-Application-Experience/Program-Compatibility-Troubleshooter">*</Select> <Select Path="Microsoft-Windows-Application-Experience/Program-Inventory">*</Select> <Select Path="Microsoft-Windows-Application-Experience/Program-Telemetry">*</Select> <Select Path="Microsoft-Windows-Application-Experience/Steps-Recorder">*</Select> <Select PATH="Microsoft-Windows-Backup">*</Select> <Select Path="Microsoft-Windows-CAPI2/Operational">*</Select> <Select Path="Microsoft-Windows-CertificateServicesClient-Lifecycle-System/Operational">*</Select> <Select Path="Microsoft-Windows-CodeIntegrity/Operational">*</Select> <Select Path="Microsoft-Windows-DNS-Client/Operational">*</Select> <Select Path="Microsoft-Windows-GroupPolicy/Operational">*</Select> <Select PATH="Microsoft-Windows-Kernel-IO/Operational">*</Select> <Select Path="Microsoft-Windows-Kernel-PnP/Configuration">*</Select> <Select Path="Microsoft-Windows-LSA/Operational">*</Select> <Select Path="Microsoft-Windows-NTLM/Operational">*</Select> <Select Path="Microsoft-Windows-NetworkProfile/Operational">*</Select> <Select Path="Microsoft-Windows-PowerShell/Admin">*</Select> <Select Path="Microsoft-Windows-PowerShell/Operational">*</Select> <Select Path="Microsoft-Windows-PrintService/Admin">*</Select> <Select Path="Microsoft-Windows-PrintService/Operational">*</Select> <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select> <Select Path="Microsoft-Windows-TerminalServices-RemoteConnectionManager/Operational">*</Select> <Select Path="Microsoft-Windows-Time-Service/Operational">*</Select> <Select Path="Microsoft-Windows-User Profile Service/Operational">*</Select> <Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select> <Select Path="Microsoft-Windows-Windows Defender/WHC">*</Select> <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurity">*</Select> <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/ConnectionSecurityVerbose">*</Select> <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/Firewall">*</Select> <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallDiagnostics">*</Select> <Select Path="Microsoft-Windows-Windows Firewall With Advanced Security/FirewallVerbose">*</Select> <Select Path="Microsoft-Windows-WindowsUpdateClient/Operational">*</Select> <Select Path="OpenSSH/Operational">*</Select> <Select Path="PowerShellCore/Operational">*</Select> </Query> </QueryList> </QueryXML> <Exec> if ($EventID NOT IN (%AccountUsage%)) and ($EventID NOT IN (%AppCrashes%)) and ($EventID NOT IN (%AppWhitelisting%)) and ($EventID NOT IN (%BootEvents%)) and ($EventID NOT IN (%CertServices%)) and ($EventID NOT IN (%ClearingLogs%)) and ($EventID NOT IN (%DNSDirectoryServ%)) and ($EventID NOT IN (%ExtMedia%)) and ($EventID NOT IN (%GroupPolicyError%)) and ($EventID NOT IN (%Installation%)) and ($EventID NOT IN (%KernelDriver%)) and ($EventID NOT IN (%MSFTCryptoAPI%)) and ($EventID NOT IN (%MobileDeviceEvents%)) and ($EventID NOT IN (%NetworkHost%)) and ($EventID NOT IN (%PowerShell%)) and ($EventID NOT IN (%PrintingServices%)) and ($EventID NOT IN (%SystemIntegrity%)) and ($EventID NOT IN (%SystemServiceFail%)) and ($EventID NOT IN (%WinDefender%)) and ($EventID NOT IN (%WinFirewall%)) and ($EventID NOT IN (%WinUpdateError%)) drop(); </Exec> </Input> <Processor buffer> Module pm_buffer MaxSize 102400 Type disk </Processor> <Output out> Module om_tcp Host 0.0.0.0 Port 514 Exec to_json(); $Message = $raw_event; to_syslog_ietf(); </Output> <Route 1> Path Events => buffer => out </Route>

Hi, I am trying to create a regex for parsing log lines from an application. The issue is that the regex works fine in other applications, regex makers, etc. But when used by NXLog it won't find any matches, so I'm afraid there may be some NXLog specific regex syntax. Before I deconstruct this large regex and restart my service repeatedly, I wanted to ask if there is anything immediately obvious that is wrong... or is there a way to create these patterns in a reliable way? I saw some documentation about NXLog manager being able to create patterns, but I don't have access to this tool at the moment. /\[.*?\] (\d+\s\w+\s\d+\s\d+\:\d+\:\d+\,\d+)\s(\S+)\s+(\S+)\s+\W+BusinessApplication:(.+?(?=\|))\|Component:(.+?(?=\|))\|Service:(.+?(?=\|))\|Operation:(.+?(?=\|))\|HttpMethod:(.+?(?=\|))\|Version:(.+?(?=\|))\|Client:(.+?(?=\|))\|ResponseTime:(.+?(?=\|))\|HttpStatus:(.+?(?=\|))\|Status:(.+?(?=\|))\|Severity:(.+?(?=\|))\|StatusDescription:(.+?(?=\|))\|MessageID:(.+?(?=\|))\|PE:(.+?(?=\|))\|CorrelationID:(.+?(?=\|))\|RelativeURI:(.+?(?=\|))\|Region:(.+?(?=\\n))\\n&quot;,&quot;stream&quot;:&quot;(.+?(?=&quot;))&quot;,&quot;time&quot;:&quot;(.+?(?=&quot;))\&quot;}/ sample log line {&quot;log&quot;:&quot;[http-nio-8080-exec-5] 10 Sep 2021 22:59:16,420 INFO PerfLog [{}]: BusinessApplication:NA|Component:NA|Service:Account Search|Operation:NA|HttpMethod:POST|Version:1|Client:enterpriseapi-2e900c67f3b948a09b0209306c64aa47|ResponseTime:1132|HttpStatus:200|Status:0|Severity:INFO|StatusDescription:SUCCESS|MessageID:MONARCH-afbd568f-e4b7-4a52-9150-26c730077c8e|PE:2718deb1806c4d6fa54efd4bf10a1abf|CorrelationID:MONARCH-2e387962-ec0c-4a11-90b3-7ea834dda252|RelativeURI:/yyyyyy-44444-333/private/25886/auto/accounts/search|Region:prod-west\n&quot;,&quot;stream&quot;:&quot;stdout&quot;,&quot;time&quot;:&quot;2021-09-10T22:59:16.421453207Z&quot;} Thank you

I've tried to deploy NXLog manager, but when I enter the credentials in Web UI, the output says "Cannot communicate with database server." I use NXLog-manager v5.5, MySQL v8.0.26 log/nxlog-manager.err output: java.sql.SQLException: Could not connect: Client does not support authentication protocol requested by server; consider upgrading MySQL client at org.mariadb.jdbc.internal.SQLExceptionMapper.get(SQLExceptionMapper.java:149) at org.mariadb.jdbc.internal.SQLExceptionMapper.throwException(SQLExceptionMapper.java:106) at org.mariadb.jdbc.Driver.connect(Driver.java:115) at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220) at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206) at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203) at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138) at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125) at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44) at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870) at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696) Caused by: org.mariadb.jdbc.internal.common.QueryException: Could not connect: Client does not support authentication protocol requested by server; consider upgrading MySQL client at org.mariadb.jdbc.internal.mysql.MySQLProtocol.checkErrorPacket(MySQLProtocol.java:571) at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:501) at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:722) at org.mariadb.jdbc.internal.mysql.MySQLProtocol.<init>(MySQLProtocol.java:281) at org.mariadb.jdbc.Driver.connect(Driver.java:111) ... 9 more nxlog-manager.conf NXLOG_MANAGER_PATH=/opt/nxlog-manager BIN_PATH=${NXLOG_MANAGER_PATH}/bin WEBAPP_PATH=${NXLOG_MANAGER_PATH}/webapps/nxlog-manager LOGS_PATH=${NXLOG_MANAGER_PATH}/log/nxlog-manager CONTEXT_PATH=/nxlog-manager JROBIN_PATH=${NXLOG_MANAGER_PATH}/data/rrd JETTY_RUNNER_PATH=${NXLOG_MANAGER_PATH}/lib/jetty-runner-9.0.7.v20131107.jar JETTY_LIB=${NXLOG_MANAGER_PATH}/lib JETTY_CONFIG_PATH=${NXLOG_MANAGER_PATH}/conf/jetty-config.xml JETTY_RUN_CMD="-cp $JETTY_RUNNER_PATH org.eclipse.jetty.runner.Runner" # This uses standalone mode, for distributed mode use 'distributed-manager' INSTANCE_MODE=nxlog-manager JVM_OPTS="-Xms1g -Xmx2g -XX:PermSize=64m -XX:MaxPermSize=256m -Xss2m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Dorg.apache.commons.collections.enableUnsafeSerialization=true -Dorg.eclipse.jetty.annotations.maxWait=120 -Dnxlog.manager.jrobin.path=$JROBIN_PATH -Dlogs.root=$LOGS_PATH -Dnxlog.manager.path=$NXLOG_MANAGER_PATH -Dnxlog.manager.contextpath=$CONTEXT_PATH -Dnxlog.manager.webapp.path=$WEBAPP_PATH -Dspring.profiles.active=$INSTANCE_MODE" PIDFILE=${NXLOG_MANAGER_PATH}/data/nxlog-manager.pid JAVA=/usr/bin/java PORT=9090 NXLOG_MANAGER_USER=nxlog-mgr NXLOG_MANAGER_GROUP=nxlog-mgr

Has anybody installed NxLog-Manager on Vmware's Photon OS with Docker? Any issues?

Hello I have to use nxlog to read SAS log file from SAS Guide. It's about more than 1.000 users in my company. Nxlog is installed on these 1.000 laptops, and the SAS Log files are under a directory like c:\Users\A012345\Logs*.log -> I can't change this directory ... -> A012345 is an exemple of the employee number. So, I have one different directory for all 1.000 laptops. I have to find a solution to bypass these employee numbers variable and find a generic expression to access to this directory. So, I can't use a wildcard directly in log directory (c:\Users*\Logs*.log), nxlog can't do that. I have a solution by using a Windows Shortcut. I create a shortcut "D:\Temp\SAS-Logs&quot; with a link to my official directory c:\Users\A012345\Logs. And my nxlog input with im_file works perfectly. Now, I'm faced to another problem. I must manage the SAS logs file retention. I'm testing the file_remove function. When I specify my official directory (c:\Users\A012345\Logs*.log), it works perfectly. When I'm using the same shortcut (which works with my Im_file), nxlog says he can't find the directory. What is a the problem, the file_remove function uses another library ? Do you any idea to suppress my "old" sas log file with nxlog ? Thanks Arnaud

Hi, I'm using Graylog as logging platform and in the error logs I'm facing an issue related to the way Nxlog is forwarding messages to Graylog. Here the stacktrace followed by the nxlog configuration: 2021-10-19T17:00:24.464+02:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=49ba6ee1-30ed-11ec-a554-005056a17083, journalOffset=140463071838, codec=gelf, payloadSize=249, timestamp=2021-10-19T15:00:24.398Z, remoteAddress=/10.181.8.23:39018} java.lang.IllegalArgumentException: GELF message <49ba6ee1-30ed-11ec-a554-005056a17083> (received from <10.181.8.23:39018>) has empty mandatory "short_message" field. at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:258) ~[graylog.jar:?] at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:140) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:149) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292] 2021-10-19T17:00:24.464+02:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=49ba47dc-30ed-11ec-a554-005056a17083, journalOffset=140463071836, codec=gelf, payloadSize=249, timestamp=2021-10-19T15:00:24.397Z, remoteAddress=/10.181.8.23:39018} on input <581344a7a0474e1cf763f7f4>. 2021-10-19T17:00:24.469+02:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=49bb3232-30ed-11ec-a554-005056a17083, journalOffset=140463071886, codec=gelf, payloadSize=249, timestamp=2021-10-19T15:00:24.403Z, remoteAddress=/10.181.8.23:39018} java.lang.IllegalArgumentException: GELF message <49bb3232-30ed-11ec-a554-005056a17083> (received from <10.181.8.23:39018>) has empty mandatory "short_message" field. at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:258) ~[graylog.jar:?] at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:140) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:149) ~[graylog.jar:?] at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:90) [graylog.jar:?] at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:47) [graylog.jar:?] at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?] at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_292] @ define ROOT /usr/bin #define REGEX /^(?x)^[(?<EventTime>\d{4}-\d{2}-\d{2}\ \d{2}:\d{2}:\d{2}),\d{3}]/ define REGEX /(?x)^[(?<EventTime2>\d{4}-\d{2}-\d{2}\ \d{2}:\d{2}:\d{2}),\d{3}]\ (?<Severity>\S+)\ (?<Class>\S+)\ -\ (?<Message>[\s\S]+)/ define REGEX_CATALINA /(?x)^(?<EventTime>\d{2}-\w{3}-\d{4}\ \d{2}:\d{2}:\d{2}).\d{3}\ (?<Message>[\s\S]+) | (?x)^(?<EventTime2>\w{3}\ \w{3}\ \d{2}\ \d{2}:\d{2}:\d{2})\ (?<Message2>[\s\S]+)/ <Extension gelfExt> Module xm_gelf Avoid truncation of the short_message field to 64 characters. ShortMessageLength 65536 </Extension> <Extension syslogExt> Module xm_syslog </Extension> <Extension multiline> Module xm_multiline HeaderLine %REGEX% </Extension> User nxlog Group sfd #Moduledir /usr/lib/nxlog/modules Moduledir /usr/libexec/nxlog/modules CacheDir /var/spool/nxlog PidFile /var/run/nxlog/nxlog.pid LogFile /var/log/nxlog/nxlog.log LogLevel INFO <Input file_catalina1> Module im_file File '/opt/tomcat/logs/catalina.out' PollInterval 1 SavePos True ReadFromLast True Recursive False RenameCheck False Exec $FileName = file_name(); # Send file name with each message </Input> <Input file_catalina2> Module im_file File '/opt/tomcat8081/logs/catalina.out' PollInterval 1 SavePos True ReadFromLast True Recursive False RenameCheck False Exec $FileName = file_name(); # Send file name with each message </Input> <Input file_catalina3> Module im_file File '/opt/tomcat8082/logs/catalina.out' PollInterval 1 SavePos True ReadFromLast True Recursive False RenameCheck False Exec $FileName = file_name(); # Send file name with each message </Input> #<Input file_catalina> Module im_file File "/opt/tomcat/logs/catalina.out" InputType multiline <Exec> if $raw_event =~ %REGEX_CATALINA% $EventTime = parsedate($1); </Exec> #</Input> <Input log4j1> Module im_file File "/opt/tomcat/logs/error.log" InputType multiline <Exec> if $raw_event =~ %REGEX% $EventTime = parsedate($1); </Exec> </Input> <Input log4j2> Module im_file File "/opt/tomcat8081/logs/error.log" InputType multiline <Exec> if $raw_event =~ %REGEX% $EventTime = parsedate($1); </Exec> </Input> <Input log4j3> Module im_file File "/opt/tomcat8082/logs/error.log" InputType multiline <Exec> if $raw_event =~ %REGEX% $EventTime = parsedate($1); </Exec> </Input> #<Input file_logs> Module im_file File '/opt/tomcat8081/logs/error.log' PollInterval 1 SavePos True ReadFromLast True Recursive False RenameCheck False Exec $FileName = file_name(); # Send file name with each message #</Input> #<Input syslog-udp> Module im_udp Host 127.0.0.1 Port 514 Exec parse_syslog_bsd(); #</Input> <Output gelf> Module om_udp Host graylog.intern.ti8m.ch Port 1515 OutputType GELF <Exec> # These fields are needed for Graylog $gl2_source_collector = 'fb04f873-aed9-441a-b3a9-7545d4c728f6'; $collector_node_id = 'app-01.prd.sfd.intern.ti8m.ch'; $Hostname = 'app-01.prd.sfd.intern.ti8m.ch'; </Exec> </Output> <Route route-1> Path file_catalina1 => gelf </Route> <Route route-2> Path log4j1 => gelf </Route> <Route route-3> Path file_catalina2 => gelf </Route> <Route route-4> Path log4j2 => gelf </Route> <Route route-5> Path file_catalina3 => gelf </Route> <Route route-6> Path log4j3 => gelf </Route> #<Route route-7> Path syslog-udp => gelf #</Route> Hope you can help solving the issue. Thank you for your time. Best regards, Stefano Paci

Hi, I want to use nxlog to collect the windows eventlog, gzip the log and transform it to logstash for further processing. Is it possible to include all the steps into one pipeline? e.g., Input module -> Collect eventlog (im_msvistalog) Output module -> Compress the log (xm_zlib) and send it to Logstash (om_tcp) Thank you. Best regards, Steven

Hi, I can collect Security Events Windows in XML format and send them to my SIEM like i wanted the result is here for an event 4624 <Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>4624</EventID><Version>2</Version><Level>0</Level><Task>12544</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2021-10-08T09:48:30.960813800Z'/><EventRecordID>457357</EventRecordID><Correlation/><Execution ProcessID='620' ThreadID='4724'/><Channel>Security</Channel><Computer>test</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>S-1-0-0</Data><Data Name='SubjectUserName'>-</Data><Data Name='SubjectDomainName'>-</Data><Data Name='SubjectLogonId'>0x0</Data><Data Name='TargetUserSid'>S-1-5-18</Data><Data Name='TargetUserName'>test$</Data><Data Name='TargetDomainName'>test.com</Data><Data Name='TargetLogonId'>0xeb6a3</Data><Data Name='LogonType'>3</Data><Data Name='LogonProcessName'>Kerberos</Data><Data Name='AuthenticationPackageName'>Kerberos</Data><Data Name='WorkstationName'>-</Data><Data Name='LogonGuid'>{49b0407a-e478-e673-1f20-942e0965289d}</Data><Data Name='TransmittedServices'>-</Data><Data Name='LmPackageName'>-</Data><Data Name='KeyLength'>0</Data><Data Name='ProcessId'>0x0</Data><Data Name='ProcessName'>-</Data><Data Name='IpAddress'>::1</Data><Data Name='IpPort'>49805</Data><Data Name='ImpersonationLevel'>%%1833</Data><Data Name='RestrictedAdminMode'>-</Data><Data Name='TargetOutboundUserName'>-</Data><Data Name='TargetOutboundDomainName'>-</Data><Data Name='VirtualAccount'>%%1843</Data><Data Name='TargetLinkedLogonId'>0x0</Data><Data Name='ElevatedToken'>%%1842</Data></EventData></Event> but now i want some magic and i'm not Harry potter lol i dont know if it's possible but i would like to use xm_resolver to resolve SID from the SubjectUserSid ? rename <Data Name='TargetUserName'>test$</Data> into <Data Name='computer'>test$</Data> if the eventid = 4624 ? send the final result my siem in LEEF or JSon ? i don't find any good exemple on internet and all i'm trying is fail.. if someone can help ? thx ;)

Hello, My NXLog-ce-2.11.2190 stops sending logs after network disconnect and reconnect. Last logs: 2021-10-05 11:40:09 INFO nxlog-ce-2.11.2190 started 2021-10-05 11:53:29 INFO reconnecting in 1 seconds 2021-10-05 11:53:29 ERROR om_udp apr_socket_send failed; Został dostarczony nieprawidłowy argument. 2021-10-05 11:53:30 ERROR couldn't connect to udp socket on 10.2.1.159:9000; Próba przeprowadzenia operacji, wykonywanej przez gniazdo, na nieosiągalnej sieci. Config: <Extension _gelf> Module xm_gelf </Extension> <Input in> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Security"></Select> <Select Path="Microsoft-Windows-UniversalTelemetryClient/Operational"></Select> </Query> </QueryList> </QueryXML> </Input> <Output out> Module om_udp Host 10.2.1.159 Port 9000 OutputType GELF </Output> <Route 1> Path in => out </Route> System: Windows 10 Pro version 2004 on VMware vSphere 7.0 It is exactly the same question like here: https://www.mail-archive.com/nxlog-ce-users@lists.sourceforge.net/msg00970.html but I'm using the latest version of NXLog-CE. Any ideas, please...?

Hi, We have the use case to get the specific service's status hourly and it could be done via some PowerShell commands. However, we don't want maintain the script in users' hosts and want to integrate the checking into nxlog's configuration. Does nxlog agent has the capability to run such command hourly? Or will there be any alternative method? Many thanks in advance ! Best regards, Steven

Hi, The decoders used in Netwitness. The job of a decoder is to select a parser to parse log files. The Netwitness LogDeconder shows Service Type as unknown but I was expecting to see winevent_snare. My NXLog config uses the Exec $Message =~ s/(\t|\R)/ /g; to_syslog_snare(); to send windows log data to the NETWITNESS collector/decoder. The problem is the decoder is using unknown or rxlinux as service type not winevent_snare to parse my windows log files. I was looking for the decoder to use winevent_snare but it is not. Does anyone have a working NXLog config file to collect windows event logs to Netwitness? Thanks for you assistance, Jim