Hi! We have nxlog ce running in a Windows machine. It works ok. If time is changed to the future, it continues forwarding logs. However, if time is changed to the past, logs are not forwarded anymore. This affects to logs from windows events, from a text file, etc. It seems that nxlog is filtering the logs and that logs with a previous time than others received are discarded. Logs are forwarded again if nxlog service is restarted (this seems to be doing a 'reset' on expected time) Do you know how could we avoid this?

NXLOG version: NXLog CE 3.0.2272 OS version: Windows 2019 server Issue: file_name() returns "unknown" in im_file module Config: <Input in_AppABC> Module im_file <Exec> log_info('Filename is' + file_name()); </Exec> File "C:\logs\AppABC.log" </Input>

On our Graylog server we have GELF over TCP enabled. I use the following as a prototype Windows Server config file, with all relevant log paths defined for various services. We then just erase the lines we dont' want. I don't think I've seen a sample template, so this would have been useful when I was first building. Important to note, we didn't find any useful logs in event log for sharepoint, sccm, SQL Server, IIS, or Dynamics CRM, they log separately: Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension gelf> Module xm_gelf </Extension> <Input inWindowsAudit> Module im_msvistalog ReadFromLast True Query <QueryList> <Query Id="0">\ Delete Unwanted Rows Standard Server Logs &lt;Select Path=&quot;Security&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;System&quot;&gt;*[System/Level=4]&lt;/Select&gt;\ &lt;Select Path=&quot;Application&quot;&gt;*[Application/Level=2]&lt;/Select&gt;\ &lt;Select Path=&quot;Setup&quot;&gt;*[System/Level=3]&lt;/Select&gt;\ Hardware Logs &lt;Select Path=&quot;HardwareEvents&quot;&gt;*&lt;/Select&gt;\ Key Management &lt;Select Path=&quot;Key Management Service&quot;&gt;*&lt;/Select&gt;\ Windows Powershell &lt;Select Path=&quot;Windows PowerShell&quot;&gt;*&lt;/Select&gt;\ Internet Explorer &lt;Select Path=&quot;Internet Explorer&quot;&gt;*&lt;/Select&gt;\ Active Directory &lt;Select Path=&quot;Active Directory Web Services&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;DFS Replication&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;Directory Service&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;DNS Server&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;File Replication Service&quot;&gt;*&lt;/Select&gt;\ Server Manager &lt;Select Path=&quot;Microsoft-ServerManagementExperience&quot;&gt;*&lt;/Select&gt;\ Exchange Logs &lt;Select Path=&quot;EWS Monitoring Events&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;MSExchange Management&quot;&gt;*&lt;/Select&gt;\ VAMT &lt;Select Path=&quot;Volume Activation Management Tool&quot;&gt;*&lt;/Select&gt;\ Lync/Skype &lt;Select Path=&quot;Lync Server&quot;&gt;*&lt;/Select&gt;\ Blank Template &lt;Select Path=&quot;&quot;&gt;*&lt;/Select&gt;\ &lt;/Query&gt;\ &lt;/QueryList&gt; # For windows 2003 and earlier use the following: # Module im_mseventlog Exec $CustomerID = 'my_customer'; Exec $LogType = 'Windows Audit'; </Input> <Output outGraylog> Module om_tcp Host ## GRAYLOG SERVER IP ## Port 12201 OutputType GELF_TCP </Output> <Route 1> Path inWindowsAudit => outGraylog </Route>

Hello, I parameterized as seen in the examples the nxlog configuration file for the logs of my Windows 2016 servers, but when I restart the services with them. In the nxlog files I find this: nxlog failed to start: Expected </Extension_gelf> but saw </Extension> at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48 nxlog failed to start: Expected </Extension_gelf> but saw </Extension> at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48 nxlog failed to start: Expected </Extension2> but saw </Extension> at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48 Do you have ideas to list the errors? thank you in advance

Hello everyone, I have a window server that receives logs from other windows hosts (log collector) and from this last one, events are sent to a Fortisiem. The problem is that in SIEM the IP that appears is always the collector's IP and all host events are identified by that IP. Is it possible to keep the original IP of each host? My out config: <Output out> Module om_tcp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $Message = replace($Message, "\t", " "); $Message = replace($Message, "\n", " "); $Message = replace($Message, "\r", " "); Exec $Message = to_json(); to_syslog_snare(); </Output> Thanks

Hi everyone! You many help me, thanks a lot. I hope you kind to help me now. My NXLog clients don't collect Windows System logs. And now I often see in my logs this message: 2019-06-04 17:49:50 INFO nxlog-4.3.4308 started 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown. <QueryList> <Query Id='1'> <Select Path='System'>*</Select> </Query> </QueryList> <QueryList> <Query Id='1'> <Select Path='Application'>*</Select> </Query> </QueryList> 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events [error code: 1717]; The interface is unknown. My config: define ROOT C:\nxlog define NXLOGLOGFILE %ROOT%\data\nxlog.log define CERTDIR %ROOT%\cert PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval 0 CacheSync TRUE <Input winapp> Module im_msvistalog ReadFromLast TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='Application'>*</Select> </Query> </QueryList> </QueryXML> Exec $FileName = 'winapp.log'; Exec $EventTime = $EventReceivedTime; </Input> <Input winsys> Module im_msvistalog ReadFromLast TRUE <QueryXML> <QueryList> <Query Id='1'> <Select Path='System'>*</Select> </Query> </QueryList> </QueryXML> Exec $FileName = 'winsys.log'; Exec $EventTime = $EventReceivedTime; </Input> <Output out> BufferSize 9500000 Module om_batchcompress Host 192.168.100.100 Port 1514 UseSSL true AllowUntrusted TRUE CAFile %CERTDIR%\cacert.pem CertFile %CERTDIR%\clientcert.pem CertKeyFile %CERTDIR%\clientkey.pem </Output> <Route client> Path winapp, winsys => out </Route> After restart service nothing new. Any ideas, please!

Hello, I have recently been trying up a syslog-ng server for various devices and have tried a couple of things for sending Windows Events to the server. Finally decieded that NXLog will do what I need and I have gotten sent some events over without much configuration, but when trying filter within the .conf file, it always fails. I can't really find much good information as to why it might be failing, as it seems that it should be correct.(to me anyway) # Windows Event Log, <Input s_eventlog> Module im_msvistalog Exec if $EventID == 4734 or $EventID == 4624 drop(); Exec $Message = to_json(); </Input> I have narrowed it down to this block, since the log says nxlog failed to start: </Input> without matching <Input> section at C:\Program Files (x86)\nxlog\conf\nxlog.conf:43 Which is where this block ends? I can't really make sense of this, so if anyone has some guidance please tell me.

I am trying to install nxlog on Windows server 2000. However, I get the error "Installation directory must be on a local hard drive." I have tried using administrative command prompt, Same Error. Can anyone help me out here?

Hi, I'm using the im_msvistalog input to grab events from the Windows security log however the important information is being ignored. This is one my Windows events: <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="AD FS Auditing" /> <EventID Qualifiers="0">411</EventID> <Level>0</Level> <Task>3</Task> <Keywords>0x8090000000000000</Keywords> <TimeCreated SystemTime="2018-11-06T09:22:29.086191400Z" /> <EventRecordID>85712874</EventRecordID> <Channel>Security</Channel> <Computer>server1</Computer> <Security UserID="S-8-8-88-8888-8888-8888-8888" /> </System> <EventData> <Data>00000000-0000-0000-0000-000000000000</Data> <Data>http://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName</Data> <Data>user1@domain.com</Data> <Data>System.IdentityModel.Tokens.SecurityTokenValidationException: user1@domain.com at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateToken(SecurityToken token)</Data> <Data>8.8.8.8</Data> </EventData> <RenderingInfo Culture="en-US"> <Message>Token validation failed. See inner exception for more details. Additional Data Activity ID: 00000000-0000-0000-0000-000000000000 Token Type: http://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName Client IP: 8.8.8.8 Error message: user1@domain.com Exception details: System.IdentityModel.Tokens.SecurityTokenValidationException: user1@domain.com at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateToken(SecurityToken token)</Message> <Level>Information</Level> <Task /> <Opcode>Info</Opcode> <Channel /> <Provider /> <Keywords><Keyword>Audit Failure</Keyword><Keyword>Classic</Keyword> </Keywords> </RenderingInfo> </Event> As you can see, the relevant information is between EventData and Message tags. But this information does not appear in the output message: {"EventTime":"2018-11-06 09:22:29" ,"Hostname":"server1" ,"Keywords":-9182839640208441344 ,"EventType":"AUDIT_FAILURE" ,"SeverityValue":4 ,"Severity":"ERROR" ,"EventID":411 ,"SourceName":"AD FS Auditing" ,"Task":3 ,"RecordNumber":85712874 ,"ProcessID":0 ,"ThreadID":0 ,"Channel":"Security" ,"Domain":"domain.com" ,"AccountName":"service1" ,"AccountType":"User" ,"EventReceivedTime":"2018-11-06 09:22:31" ,"SourceModuleName":"eventlog" ,"SourceModuleType":"im_msvistalog" } This is my nxlog config: <Input eventlog> Module im_msvistalog Channel ForwardedEvents Exec $Message = to_json(); </Input> <Output graylog> Module om_tcp Host graylog.server.com Port 1111 OutputType GELF_TCP </Output> <Route 1> Path eventlog => graylog </Route> According to the docs, Data between EvenData tags is automatically extracted if it is named, but it isn't in my case. Can data be extracted manually somehow? I'm running nxlog CE 2.9. Thanks

hi, I'm working on monitoring a log file using nxlog. I have the File set to "C:\Program Files\test1.log" but it's saying that the "input file does not exist". I tried running a python script to check the file using the os module import os test = os.listdir('C:\Program Files\test1.log') print(test) This will return an error "FileNotFoundError: The system cannot find the path specified" I noticed that this error has been encountered before but none of the solutions I tried work. any help is much appreciated. Thanks, skawt

Hello, Where can I find Nxlog-ce source code for Windows?

It seems I have a problem with Nxlog-ce and Windows eventlog after power resume/reconnect to the network. On the high level we won't get any logs from a a machine before we restart the nxlog service. It shows as runnig but sends no logs. As soon as you restart it, the logs are sent. I Enabled debug logging and got the following 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 26 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG executing statements 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:3 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:4 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:5 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:6 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:7 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:8 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:9 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:10 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:11 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:12 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 27 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 ERROR Exception was caused by "apr_sockaddr_info_get(&sa, omconf->host, APR_INET, omconf->port, 0, pool)" at om_udp.c:279/om_udp_connect(); [om_udp.c:279/om_udp_connect()] apr_sockaddr_info failed for Myhost.mydomain.XX:12235; Det begärda namnet är giltigt men data för den begärda typen kunde inte hittas. 2017-11-27 08:02:40 DEBUG worker 2 processing event 0x27a5078 2017-11-27 08:02:40 DEBUG PROCESS_EVENT: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG om_udp_write 2017-11-27 08:02:40 DEBUG module eventlogOUT is not running, not reading any more data 2017-11-27 08:02:40 DEBUG worker 2 waiting for new event 2017-11-27 08:02:40 DEBUG executing statements my NXlog.conf looks like this Nxlog.conf Created: 10/12/2017 15:21:54 LogLevel DEBUG define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> Include plug-in directory include %ROOT%\conf\add-on\*.conf and I have an include file for the eventlog that looks like this <Input eventlogIN> Module im_msvistalog </Input> <Output eventlogOUT> Module om_udp Host myhost.mydomain.xx Port 12235 OutputType GELF </Output> <Route eventlog> Path eventlogIN => eventlogOUT </Route> Has anyone seen this before or got some ideas?

Hi , Trying to create an ODBC connect for NXLog to connect to.   NXLog is installed on the same Windows 2012 server as the SQL Server 2008R2 instance.   Scenario 1: 32-bit ODBC is setup as a System DSN with a SQL Server account that has DBO access to the desired database NXLog service is setup to run under the System account.      - I've tried both drivers available on the system ("SQL Server Native Client 10.0"  and "SQL Server") - get the same result in the error log for each: ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][SQL Server Native Client 10.0][SQL Server]Login failed for user ''. (odbc error code: -1)and ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)   Scenario 2: Same ODBC, but with a Windows account that has full Admin access to the desired databases, and is the same account logged into Windows NXLog service is setup to run under this same account. Goal is to have the same user account accessing everything, in the hope of getting it to connect.   Same error messages as above.   Login failed for user ' '.     Since the error messages don't show the user that is failing to login, I'm having trouble narrowing down where the failure is at.   NXLOG.conf file: <Input call_logs> Module im_odbc ConnectionString DSN=SIEM_NXLog;database=recorder; SQL SELECT ident as id ,at.audit_time as EventTime ,am.audit_module_name as Message FROM mytables...  WHERE at.ident>? SavePos TRUE </Input>   There's one line in the documentation that has me scratching my head: SECTION 6.2.18 (ODBC) The data source must be accessible by the user which nxlog is running under.   I'm not sure if this means that the NTService account needs database access? Or, if the service must be under a Windows account user that has database access? Or, by using a ODBC->System DSN , shouldn't the ODBC already be accessible to all users on the system?   Any thoughts or insight would be helpful. Thanks in advance.   Cheers,  Peter          

Scenario: I have NXLog EE installed on a host in Windows domain. I need to read DHCP logs from the DC(s), UNC path: \\<server name>\C$\Windows\System32\dhcp\DhcpSrvLog-*.log Since it is not possible to specify alternate credentials for accessing remote files (as it is for eventlog, i.e. im_msvistalog module), nxlog has to be started using an account with special privileges on the DC's file system - 4 options:  1. for nxlog service, use domain admin account (local admin role does not exist on DC)      - nxlog.conf - use UNC path: `\\<server name>\C$\Windows\System32\dhcp\DhcpSrvLog-*.log`  2. for nxlog service, use local admin account on the agent's host + share C:\Windows\System32\dhcp\ on the DC, enabling read only permissions for nxlog account only      - nxlog.conf - use share name: `\\<server name>\dhcp\DhcpSrvLog-*.log`  3. install nxlog agent on the DC, run nxlog as a service, use local admin account  4. smaller footprint ? -> install http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#nxlog_processor on the DC None of these options are win-wins for customer production environment, as they require opening the restricted environment of the DC. My question is: are there any nxlog configuration options, which would enable me to fetch the file remotely, similar to these for DC's Security event log?: <Input dc1> Module im_msvistalog RemoteServer <ip> Remoteuser <user> RemotePassword <pwn> RemoteDomain <domain> Query <QueryList><Query Id="0" Path="Security"><Select Path="Security">*</Select></Query></QueryList> </Input>

Here is a sample event when using to_syslog_snare() in the nxlog.conf: <14>Jan 27 10:03:39 event_computer MSWinEventLog        1        Security        32630749        Wed Jan 27 10:03:39 2016        4624        Microsoft-Windows-Security-Auditing        N/A        N/A        Success Audit        event_computer        Logon                An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Impersonation    New Logon:   Security ID:  S-1-5-21-2705889813-1605608894-1661845433-43745   Account Name:  account_name   Account Domain:  account_domain   Logon ID:  0x23820B882   Logon GUID:  {00000000-0000-0000-0000-000000000000}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: workstation_name   Source Network Address: source_address   Source Port:  54241    Detailed Authentication Information:   Logon Process:  NtLmSsp    Authentication Package: NTLM   Transited Services: -   Package Name (NTLM only): NTLM V2   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.        35284558 My issue is that I would NOT want to collect the "informational text" representing the event - in this case everything starting from the string "This event is generated---" all the way up until "--was requested." Before I go any deeper into this, let me state that in the logs of this format I call the "<14>Jan 27 10:03:39 event_computer MSWinEventLog        1        Security        32630749        Wed Jan 27 10:03:39 2016        4624        Microsoft-Windows-Security-Auditing        N/A        N/A        Success Audit        event_computer        Logon" portion of the whole log message the HEADER, and the rest is called MESSAGE. Putting it another way, I would like to forward the message using syslog in a format constructed according to the pseudocode below: parse fields from windows event /* e.g. SubjectUserName, LogonType, IpAddress, etc. */ /* print the header "as is" already in the to_syslog_snare() format, i.e. from "<14>---" until and including "---Logon" print HEADER /* e.g. event_time,event_computer,event_type,event_id,... */ for all fields parsed     print "'field_name=field_value'" /* e.g. SubjectUserName=value,LogonType=value,IpAddress=value,... /*      The reason I would like to do this is that the informational text, which gets appended to some Windows events (not all, it seems), takes a lot of space, and we do not really need this information text for anything. Another way to do this would be to statically list all the fields POSSIBLY found in an Windows event and construct the message that way, but this would often leave me with a lot of empty key-value pairs. THUS I would only like to print out those fields that were found in that specific log message while leaving out the informational message. I do acknowledge, though, that especially Application and System events might not contain most or any of the fields that are present in a Security log event. Take for example the following System log event: <14>Jan 27 11:09:21 event_computer MSWinEventLog        1        System        32633951        Wed Jan 27 11:09:21 2016        7036        Service Control Manager        N/A        N/A        Information        event_computer        N/A                The Remote Registry service entered the stopped state.        319889 In the example above, the "header" portion of the whole message only contains the string "The Remote Registry service entered the stopped state." I do hope, though, that the variable where this string is stored is actually the same that hosted the string "An account was successfully logged on.", which would mean that my approach in the pseudocode would still work (i.e. the array or list of fields that is iterated and printed would only contain one field. The HEADER portion of the field is exactly the same in all messages. The description of to_syslog_snare() in the nxlog documentation states: "Create a SNARE Syslog formatted log message in $raw_event. Uses the following fields to construct $raw_event: $EventTime, $Hostname, $SeverityValue, $FileName, $EventID, $SourceName, $AccountName, $AccountType, $EventType, $Category, $Message." Thus when reflecting back to what I said, it seems that what I call the HEADER includes all the fields from $EventTime to (and including) $Category - this I would like to keep as it is. But according to the documentation, the $Message variable actually then holds all the other information in the log, or what I call the MESSAGE portion. So I guess the question is that can the contents of the $Message variable be further filtered, as it obviously is constructed from e.g. EventData's Data fields listed below. I would like to only change the $Message contents so that it would never contain the informational text if there exists such a message in a given log message, and that preferably the Data fields inside $Message would be formatted using key-value pairs instead of the to_syslog_snare format seen in the first example (one or more whitespace as delimiter).

Hello nxlog world, Shamed to say, I've spent entire yesterday trying to figure out how to read Windows DHCP log files and ship the events to ElasticSearch. Problem was with using direct path for folder C:\Windows\System32\dhcp\. Managed to get nxlog to read by sharing the folders (read-only permissions) to the user account used for nxlog service account logon. As the events were not showing in ES, I'm stuck with trying to write the events into another file, in order to confirm that the source files are being read correctly.   OS: Win Srv 2008 R2 Ent nxlog: v 2.9.1347 Here is the nxlog.conf: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log # debugging only: LogLevel DEBUG NoCache TRUE <Input msdhcp> Module im_file File '\\DC5\dhcp\DhcpSrvLog-*.log' SavePos TRUE InputType LineBased Exec if $raw_event =~ /^#/ drop(); Exec $message = $raw_event; </Input> <Input dns> Module im_file File '\\DC5\dns\dns.log' SavePos TRUE InputType LineBased Exec if $raw_event =~ /^#/ drop(); Exec $message = $raw_event; </Input> <Output file_test> Module om_file File 'C:\Program Files (x86)\nxlog\data\test_file_output.txt' # Sync TRUE OutputType LineBased </Output> <Route test> Path msdhcp,dns => file_test </Route> As a result, only DNS events are written in the output file: 21.1.2016. 11:34:00 A6A8 PACKET 0000000003B27E90 UDP Snd 192.168.105.12 3f0d R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0) 21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Rcv 192.168.105.12 3c32 Q [0001 D NOERROR] A (8)PLANKING(3)lab(5)rador(0) 21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Snd 192.168.105.12 3c32 R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0) ..but only the new ones, i.e. as the source DNS log file is being appended. I have tried modifying the SavePos parameter to FALSE of both input modules, but to no avail - same result.   Questions: 1. What would be the correct configuration of global NoCache and module specific SavePos parameters, in order to read and output the complete source file, regardless of prior attempts? 2. What is the reason DHCP logs (using wildcard) are not being read (or at least written in the output), as opposed to the same configuration for DNS logs?     I will provide nxlog debug level log if needed. No visible errors there.   Any help greatly appreciated!

I'm trying to pass only Warning / Error / Critical level Application Logs through NXLog to my ELK stack. When I have this configuration <Input EventLog_In> Module im_msvistalog <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ </Query>\ </QueryList> Exec to_json(); </Input> everything works fine, and I'm collecting all levels of Application logs. I tried putting in a parameter on the <Select Path> line like this <Select Path="Application">*[Application/Level=1]</Select>\ And it craps itself and I get nothing. NXLog isn't reporting any issue, and I'm not seeing anything on the logstash side of things. I got the information about Event Viewer querying from this thread and adapted it to my use case: https://serverfault.com/questions/543494/query-specific-logs-from-event-log-using-nxlog

I'm attempting to build nxlog with some updated libraries: Latest APR (1.5.2) Non-Heartbleed vulnerable OpenSSL sources PCRE 8.37 Zlib 1.2.8 After building all the dependencies I'm a little stuck on getting nxlogs to build, specifically I'm stuck on the step where I run ./configure At first it couldn't find apr-1-config, so I added /local/apr/bin to the path. Then it couldn't fine libapr-1 so I added /local/apr/lib to the path, this is where the problems started. When APR built there wasn't a "libapr-1" file in /local/apr/lib, only libapr-1.a, libapr-1.la, libapr-1.dll.a. Did I build APR incorrectly? I'm trying to build this on windows List of steps to get where I am: 1. Install MINGW using MinGW Installation Manager Add packages: mingw-developer-toolkit mingw-base mingw-expat bin mingw32-libexpat dev msys-libopenssl dev msys-automake msys-autoconf Setup msys fstab (c:/mingw     /mingw) 2. Install Python (2.5) 3. Add Python and mingw to system path (C:\Python25;C:\MinGW\bin;C:\MinGW\msys\1.0\bin) 3. Get and build APR source (I could not get APR iconv to compile) Download: http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz http://mirror.nexcess.net/apache//apr/apr-1.5.2-win32-src.zip http://mirror.nexcess.net/apache//apr/apr-util-1.5.4-win32-src.zip http://sourceforge.net/projects/pcre/files/pcre/8.37/pcre-8.37.zip/download http://zlib.net/zlib128.zip Build: Extract all files to c:\mingw\msys\1.0\src Compile libiconv cd libiconv-1.11 ./configure CFLAGS="-O2 -s -mms-bitfields -march=i686" CXXFLAGS="-O2 -s -mms-bitfields -march=i686" make && make install Compile APR cd apr ./buildconf   ./configure CFLAGS="-O0 -s -mms-bitfields -march=i686" CXXFLAGS="-O0 -s -mms-bitfields -march=i686" make && make install cd .. Compile APR-UTIL cd apr-util-1.5.4 ./buildconf --with-apr=/usr/src/apr-1.5.2 ./configure CFLAGS="-O2 -s -mms-bitfields -march=i686" CXXFLAGS="-O2 -s -mms-bitfields -march=i686" --with-apr=/usr/src/apr-1.5.2 make && make install cd .. Compile PCRE cd pcre-.37 ./configure make && make install (make threw an error corrected with make clean, autoconf -i --force, started back at step 1) cd .. Compile ZLIB cd zlib-1.2.8 make -f win32/Makefile.gcc Compile nxlog cd nxlog-ce-2.8.1248 ./configure This is where the problems began. First it couldn't find apr-1-config. Fixed by adding /local/apr/bin to path. Now it can't find libapr-1, addint /local/apr/lib to the path doesn't help. There is no libapr-1 file in the MinGW directory tree. Ideas?   -pacmanwa  

Are there any numbers about how nxLog performs when it is processing a high rate of messages being placed into a log file? Right now we have a couple of incidents which resulted in a few thousand messages being logged per second. I assume this is more than nxLog can handle but am wondering about any performance testing that has been run

I've seen some posts from about a year ago that NXLog is unable to parse attributes using xm_xml, I just wanted to check if this is still true? I am running NXLog as a service on Windows machines and want to be able to parse the following message, is it possible? <log4j:event logger="com.sentry.test.LogContextListener" timestamp="1437661699866" level="TRACE" thread="localhost-startStop-1"> <log4j:message><![CDATA[This is a trace message about how we should use C#]]></log4j:message> </log4j:event>