Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.

file_name() returns unknown in im_file in Windows

``` NXLOG version: NXLog CE 3.0.2272

OS version: Windows 2019 server

Issue: filename() returns "unknown" in imfile module

Config:

<Input in_AppABC>
  Module im_file
  <Exec>
    log_info('Filename is' + file_name());
  </Exec>
  File "C:\logs\AppABC.log"
</Input>

```


mitchfloresswi
Replies: 5
View post »
gahorvath
logs are not forwarded if windows time-date is changed backwards

``` Hi!

We have nxlog ce running in a Windows machine. It works ok.
- If time is changed to the future, it continues forwarding logs. - However, if time is changed to the past, logs are not forwarded anymore. This affects to logs from windows events, from a text file, etc.

It seems that nxlog is filtering the logs and that logs with a previous time than others received are discarded. Logs are forwarded again if nxlog service is restarted (this seems to be doing a 'reset' on expected time) Do you know how could we avoid this? ```


juanjo
Replies: 2
View post »
juanjo
My example nxlog.conf file for all windows services we monitor.

``` On our Graylog server we have GELF over TCP enabled. I use the following as a prototype Windows Server config file, with all relevant log paths defined for various services. We then just erase the lines we dont' want. I don't think I've seen a sample template, so this would have been useful when I was first building. Important to note, we didn't find any useful logs in event log for sharepoint, sccm, SQL Server, IIS, or Dynamics CRM, they log separately:

Panic Soft

NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

Module xm_syslog

Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32

Module xm_exec

Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
                (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

Module xm_gelf

Module im_msvistalog ReadFromLast True Query \ \

Delete Unwanted Rows

Standard Server Logs

        <Select Path="Security">*</Select>\
        <Select Path="System">*[System/Level=4]</Select>\
        <Select Path="Application">*[Application/Level=2]</Select>\
        <Select Path="Setup">*[System/Level=3]</Select>\

Hardware Logs

        <Select Path="HardwareEvents">*</Select>\

Key Management

        <Select Path="Key Management Service">*</Select>\

Windows Powershell

        <Select Path="Windows PowerShell">*</Select>\

Internet Explorer

        <Select Path="Internet Explorer">*</Select>\

Active Directory

        <Select Path="Active Directory Web Services">*</Select>\
        <Select Path="DFS Replication">*</Select>\
        <Select Path="Directory Service">*</Select>\
        <Select Path="DNS Server">*</Select>\
        <Select Path="File Replication Service">*</Select>\

Server Manager

        <Select Path="Microsoft-ServerManagementExperience">*</Select>\

Exchange Logs

        <Select Path="EWS Monitoring Events">*</Select>\
        <Select Path="MSExchange Management">*</Select>\

VAMT

        <Select Path="Volume Activation Management Tool">*</Select>\

Lync/Skype

        <Select Path="Lync Server">*</Select>\

Blank Template

        <Select Path="">*</Select>\

    </Query>\
</QueryList>
# For windows 2003 and earlier use the following:
# Module im_mseventlog
Exec $CustomerID = 'my_customer';
Exec $LogType = 'Windows Audit';

Module omtcp Host ## GRAYLOG SERVER IP ## Port 12201 OutputType GELFTCP

Path inWindowsAudit => outGraylog ```


surfrock66
Replies: 1
View post »
Arkadiy
Windows logs can'

``` Hello, I parameterized as seen in the examples the nxlog configuration file for the logs of my Windows 2016 servers, but when I restart the services with them. In the nxlog files I find this:

nxlog failed to start: Expected but saw at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48

nxlog failed to start: Expected but saw at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48

nxlog failed to start: Expected but saw at C:\Program Files (x86)\nxlog\conf\nxlog.conf:48

Do you have ideas to list the errors? thank you in advance

```


feujj
Replies: 3
View post »
Zhengshi
Windows Logs

``` Hello everyone, I have a window server that receives logs from other windows hosts (log collector) and from this last one, events are sent to a Fortisiem. The problem is that in SIEM the IP that appears is always the collector's IP and all host events are identified by that IP. Is it possible to keep the original IP of each host?

My out config: Module omtcp Host %OUTPUTDESTINATIONADDRESS% Port %OUTPUTDESTINATIONPORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $Message = replace($Message, "\t", " "); $Message = replace($Message, "\n", " "); $Message = replace($Message, "\r", " "); Exec $Message = tojson(); tosyslogsnare();

Thanks ```


egas84
Replies: 1
View post »
b0ti
NXLog 4.3.4308 is failed to subscribe to msvistalog events

``` Hi everyone!

You many help me, thanks a lot. I hope you kind to help me now.

My NXLog clients don't collect Windows System logs. And now I often see in my logs this message: ``` 2019-06-04 17:49:50 INFO nxlog-4.3.4308 started 2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown.
2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events using bookmark: The interface is unknown.

   <QueryList>
     <Query Id='1'>         
       <Select Path='System'>*</Select>
     </Query>
   </QueryList>

   <QueryList>
     <Query Id='1'>
       <Select Path='Application'>*</Select>      
     </Query>
   </QueryList>

2019-06-04 17:49:50 ERROR failed to subscribe to msvistalog events [error code: 1717]; The interface is unknown.
```

My config: ``` define ROOT C:\nxlog define NXLOGLOGFILE %ROOT%\data\nxlog.log define CERTDIR %ROOT%\cert

PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval 0 CacheSync TRUE

Module im_msvistalog ReadFromLast TRUE
Exec $FileName = 'winapp.log'; Exec $EventTime = $EventReceivedTime;

Module im_msvistalog ReadFromLast TRUE
Exec $FileName = 'winsys.log'; Exec $EventTime = $EventReceivedTime;

BufferSize 9500000 Module om_batchcompress Host 192.168.100.100 Port 1514 UseSSL true AllowUntrusted TRUE CAFile %CERTDIR%\cacert.pem CertFile %CERTDIR%\clientcert.pem CertKeyFile %CERTDIR%\clientkey.pem

Path winapp, winsys => out ``` After restart service nothing new.

Any ideas, please! ```


hatula
Replies: 1
View post »
Zhengshi
Windows event filtering not working? Or something else

``` Hello, I have recently been trying up a syslog-ng server for various devices and have tried a couple of things for sending Windows Events to the server.

Finally decieded that NXLog will do what I need and I have gotten sent some events over without much configuration, but when trying filter within the .conf file, it always fails. I can't really find much good information as to why it might be failing, as it seems that it should be correct.(to me anyway)

# Windows Event Log, 
<Input s_eventlog>
    Module im_msvistalog
    Exec if $EventID == 4734 or $EventID == 4624 drop(); 
    Exec $Message = to_json();
</Input>

I have narrowed it down to this block, since the log says

nxlog failed to start: </Input> without matching <Input> section at C:\Program Files (x86)\nxlog\conf\nxlog.conf:43

Which is where this block ends?

I can't really make sense of this, so if anyone has some guidance please tell me.

```


DamnPeggy
Replies: 2
View post »
mikep
nxlog in Windows server 2000

``` I am trying to install nxlog on Windows server 2000. However, I get the error "Installation directory must be on a local hard drive." I have tried using administrative command prompt, Same Error.

Can anyone help me out here?

```


BibekShrestha
Replies: 1
View post »
Zhengshi
Windows EventData not captured

``` Hi,

I'm using the im_msvistalog input to grab events from the Windows security log however the important information is being ignored.

This is one my Windows events:

  <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
  <Provider Name="AD FS Auditing" /> 
  <EventID Qualifiers="0">411</EventID> 
  <Level>0</Level> 
  <Task>3</Task> 
  <Keywords>0x8090000000000000</Keywords> 
  <TimeCreated SystemTime="2018-11-06T09:22:29.086191400Z" /> 
  <EventRecordID>85712874</EventRecordID> 
  <Channel>Security</Channel> 
  <Computer>server1</Computer> 
  <Security UserID="S-8-8-88-8888-8888-8888-8888" /> 
  </System>
  <EventData>
  <Data>00000000-0000-0000-0000-000000000000</Data> 
  <Data>http://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName</Data> 
  <Data>user1@domain.com</Data> 
  <Data>System.IdentityModel.Tokens.SecurityTokenValidationException: user1@domain.com at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateToken(SecurityToken token)</Data> 
  <Data>8.8.8.8</Data> 
  </EventData>
  <RenderingInfo Culture="en-US">
  <Message>Token validation failed. See inner exception for more details. Additional Data Activity ID: 00000000-0000-0000-0000-000000000000 Token Type: http://schemas.microsoft.com/ws/2006/05/identitymodel/tokens/UserName Client IP: 8.8.8.8 Error message: user1@domain.com Exception details: System.IdentityModel.Tokens.SecurityTokenValidationException: user1@domain.com at Microsoft.IdentityServer.Service.Tokens.MSISWindowsUserNameSecurityTokenHandler.ValidateToken(SecurityToken token)</Message> 
  <Level>Information</Level> 
  <Task /> 
  <Opcode>Info</Opcode> 
  <Channel /> 
  <Provider /> 
  <Keywords><Keyword>Audit Failure</Keyword><Keyword>Classic</Keyword> 
  </Keywords>
  </RenderingInfo>
  </Event>

As you can see, the relevant information is between EventData and Message tags. But this information does not appear in the output message:

{"EventTime":"2018-11-06 09:22:29"
,"Hostname":"server1"
,"Keywords":-9182839640208441344
,"EventType":"AUDIT_FAILURE"
,"SeverityValue":4
,"Severity":"ERROR"
,"EventID":411
,"SourceName":"AD FS Auditing"
,"Task":3
,"RecordNumber":85712874
,"ProcessID":0
,"ThreadID":0
,"Channel":"Security"
,"Domain":"domain.com"
,"AccountName":"service1"
,"AccountType":"User"
,"EventReceivedTime":"2018-11-06 09:22:31"
,"SourceModuleName":"eventlog"
,"SourceModuleType":"im_msvistalog"
}

This is my nxlog config:

<Input eventlog>
    Module im_msvistalog
    Channel ForwardedEvents
    Exec $Message = to_json();
</Input>
<Output graylog>
    Module      om_tcp
    Host        graylog.server.com
    Port        1111
    OutputType  GELF_TCP
</Output>
<Route 1>
    Path eventlog => graylog
</Route>

According to the docs, Data between EvenData tags is automatically extracted if it is named, but it isn't in my case. Can data be extracted manually somehow?

I'm running nxlog CE 2.9. Thanks ```


traz
Replies: 1
View post »
traz
input file does not exist

``` hi,

I'm working on monitoring a log file using nxlog. I have the File set to "C:\Program Files\test1.log" but it's saying that the "input file does not exist". I tried running a python script to check the file using the os module

import os

test = os.listdir('C:\Program Files\test1.log') print(test)

This will return an error "FileNotFoundError: The system cannot find the path specified"

I noticed that this error has been encountered before but none of the solutions I tried work.

any help is much appreciated.

Thanks, skawt ```


skawt
Replies: 1
View post »
skawt
Nxlog-ce source code for Windows

``` Hello,

Where can I find Nxlog-ce source code for Windows? ```


lukasz
Replies: 1
View post »
Zhengshi
Issue with sending eventlogs.

``` It seems I have a problem with Nxlog-ce and Windows eventlog after power resume/reconnect to the network.

On the high level we won't get any logs from a a machine before we restart the nxlog service. It shows as runnig but sends no logs. As soon as you restart it, the logs are sent.

I Enabled debug logging and got the following

2017-11-27 08:02:40 DEBUG before nxlogqueuepush, size: 26 2017-11-27 08:02:40 DEBUG nxeventtojobqueue: DATAAVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG executing statements 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:3 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:4 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:5 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:6 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:7 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:8 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:9 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:10 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:11 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlogclient.conf:12 2017-11-27 08:02:40 DEBUG before nxlogqueuepush, size: 27 2017-11-27 08:02:40 DEBUG nxeventtojobqueue: DATAAVAILABLE (eventlogOUT) 2017-11-27 08:02:40 ERROR Exception was caused by "aprsockaddrinfoget(&sa, omconf->host, APRINET, omconf->port, 0, pool)" at omudp.c:279/omudpconnect(); [omudp.c:279/omudpconnect()] aprsockaddrinfo failed for Myhost.mydomain.XX:12235; Det begärda namnet är giltigt men data för den begärda typen kunde inte hittas.
2017-11-27 08:02:40 DEBUG worker 2 processing event 0x27a5078 2017-11-27 08:02:40 DEBUG PROCESSEVENT: DATAAVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG omudpwrite 2017-11-27 08:02:40 DEBUG module eventlogOUT is not running, not reading any more data 2017-11-27 08:02:40 DEBUG worker 2 waiting for new event 2017-11-27 08:02:40 DEBUG executing statements

my NXlog.conf looks like this

Nxlog.conf

Created: 10/12/2017 15:21:54

LogLevel DEBUG define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

Module xm_gelf

Include plug-in directory

include %ROOT%\conf\add-on\*.conf

and I have an include file for the eventlog that looks like this Module im_msvistalog

Module om_udp Host myhost.mydomain.xx Port 12235 OutputType GELF

Path eventlogIN => eventlogOUT

Has anyone seen this before or got some ideas?

```


mats
Replies: 2
View post »
AlienVault
NXLog and ODBC

```

Hi ,
Trying to create an ODBC connect for NXLog to connect to.   NXLog is installed on the same Windows 2012 server as the SQL Server 2008R2 instance.
 
Scenario 1:
32-bit ODBC is setup as a System DSN with a SQL Server account that has DBO access to the desired database
NXLog service is setup to run under the System account.   
 
- I've tried both drivers available on the system ("SQL Server Native Client 10.0"  and "SQL Server")
- get the same result in the error log for each:
  • ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][SQL Server Native Client 10.0][SQL Server]Login failed for user ''. (odbc error code: -1)and
  • ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)
 
Scenario 2:
Same ODBC, but with a Windows account that has full Admin access to the desired databases, and is the same account logged into Windows
NXLog service is setup to run under this same account.
Goal is to have the same user account accessing everything, in the hope of getting it to connect.
 
Same error messages as above.   Login failed for user ' '.
 
 
Since the error messages don't show the user that is failing to login, I'm having trouble narrowing down where the failure is at.
 
NXLOG.conf file:
<Input call_logs>
Module im_odbc
ConnectionString DSN=SIEM_NXLog;database=recorder;
SQL SELECT ident as id ,at.audit_time as EventTime ,am.audit_module_name as Message FROM mytables...  WHERE at.ident>?
SavePos TRUE
</Input>
 
There's one line in the documentation that has me scratching my head:
SECTION 6.2.18 (ODBC)
The data source must be accessible by the user which nxlog is running under.
 
I'm not sure if this means that the NTService account needs database access?
Or, if the service must be under a Windows account user that has database access?
Or, by using a ODBC->System DSN , shouldn't the ODBC already be accessible to all users on the system?
 
Any thoughts or insight would be helpful. Thanks in advance.
 

Cheers, 
Peter

 
 
 
 
 

```


pbechard
Replies: 2
View post »
owenh000
Remote collection of (restricted) file

```

Scenario:

I have NXLog EE installed on a host in Windows domain.

I need to read DHCP logs from the DC(s), UNC path: \\<server name>\C$\Windows\System32\dhcp\DhcpSrvLog-*.log

Since it is not possible to specify alternate credentials for accessing remote files (as it is for eventlog, i.e. im_msvistalog module), nxlog has to be started using an account with special privileges on the DC's file system - 4 options:
 1. for nxlog service, use domain admin account (local admin role does not exist on DC)
     - nxlog.conf - use UNC path: `\\<server name>\C$\Windows\System32\dhcp\DhcpSrvLog-*.log`
 2. for nxlog service, use local admin account on the agent's host + share C:\Windows\System32\dhcp\ on the DC, enabling read only permissions for nxlog account only
     - nxlog.conf - use share name: `\\<server name>\dhcp\DhcpSrvLog-*.log`
 3. install nxlog agent on the DC, run nxlog as a service, use local admin account
 4. smaller footprint ? -> install http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html#nxlog_processor on the DC

None of these options are win-wins for customer production environment, as they require opening the restricted environment of the DC.

My question is: are there any nxlog configuration options, which would enable me to fetch the file remotely, similar to these for DC's Security event log?:

<Input dc1>
    Module      im_msvistalog
    RemoteServer  <ip>
    Remoteuser <user>
    RemotePassword <pwn>
    RemoteDomain  <domain>
    Query        <QueryList><Query Id="0" Path="Security"><Select Path="Security">*</Select></Query></QueryList>
</Input>

```


djontra
Selective logging of Windows Event Log fields when forwarding to SIEM - exclude information text from the end of the log message

```

Here is a sample event when using to_syslog_snare() in the nxlog.conf:

<14>Jan 27 10:03:39 event_computer MSWinEventLog        1        Security        32630749        Wed Jan 27 10:03:39 2016        4624        Microsoft-Windows-Security-Auditing        N/A        N/A        Success Audit        event_computer        Logon                An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Impersonation    New Logon:   Security ID:  S-1-5-21-2705889813-1605608894-1661845433-43745   Account Name:  account_name   Account Domain:  account_domain   Logon ID:  0x23820B882   Logon GUID:  {00000000-0000-0000-0000-000000000000}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: workstation_name   Source Network Address: source_address   Source Port:  54241    Detailed Authentication Information:   Logon Process:  NtLmSsp    Authentication Package: NTLM   Transited Services: -   Package Name (NTLM only): NTLM V2   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.        35284558

My issue is that I would NOT want to collect the "informational text" representing the event - in this case everything starting from the string "This event is generated---" all the way up until "--was requested."

Before I go any deeper into this, let me state that in the logs of this format I call the "<14>Jan 27 10:03:39 event_computer MSWinEventLog        1        Security        32630749        Wed Jan 27 10:03:39 2016        4624        Microsoft-Windows-Security-Auditing        N/A        N/A        Success Audit        event_computer        Logon" portion of the whole log message the HEADER, and the rest is called MESSAGE.

Putting it another way, I would like to forward the message using syslog in a format constructed according to the pseudocode below:

parse fields from windows event /* e.g. SubjectUserName, LogonType, IpAddress, etc. */
/* print the header "as is" already in the to_syslog_snare() format, i.e. from "<14>---" until and including "---Logon"
print HEADER /* e.g. event_time,event_computer,event_type,event_id,... */
for all fields parsed
    print "'field_name=field_value'" /* e.g. SubjectUserName=value,LogonType=value,IpAddress=value,... /*
    
The reason I would like to do this is that the informational text, which gets appended to some Windows events (not all, it seems), takes a lot of space, and we do not really need this information text for anything.

Another way to do this would be to statically list all the fields POSSIBLY found in an Windows event and construct the message that way, but this would often leave me with a lot of empty key-value pairs. THUS I would only like to print out those fields that were found in that specific log message while leaving out the informational message.

I do acknowledge, though, that especially Application and System events might not contain most or any of the fields that are present in a Security log event. Take for example the following System log event:

<14>Jan 27 11:09:21 event_computer MSWinEventLog        1        System        32633951        Wed Jan 27 11:09:21 2016        7036        Service Control Manager        N/A        N/A        Information        event_computer        N/A                The Remote Registry service entered the stopped state.        319889

In the example above, the "header" portion of the whole message only contains the string "The Remote Registry service entered the stopped state." I do hope, though, that the variable where this string is stored is actually the same that hosted the string "An account was successfully logged on.", which would mean that my approach in the pseudocode would still work (i.e. the array or list of fields that is iterated and printed would only contain one field. The HEADER portion of the field is exactly the same in all messages.

The description of to_syslog_snare() in the nxlog documentation states:

"Create a SNARE Syslog formatted log message in $raw_event. Uses the following fields to construct $raw_event: $EventTime, $Hostname, $SeverityValue, $FileName, $EventID, $SourceName, $AccountName, $AccountType, $EventType, $Category, $Message."

Thus when reflecting back to what I said, it seems that what I call the HEADER includes all the fields from $EventTime to (and including) $Category - this I would like to keep as it is. But according to the documentation, the $Message variable actually then holds all the other information in the log, or what I call the MESSAGE portion. So I guess the question is that can the contents of the $Message variable be further filtered, as it obviously is constructed from e.g. EventData's Data fields listed below. I would like to only change the $Message contents so that it would never contain the informational text if there exists such a message in a given log message, and that preferably the Data fields inside $Message would be formatted using key-value pairs instead of the to_syslog_snare format seen in the first example (one or more whitespace as delimiter).

```


tsigidibam
Replies: 1
View post »
adm
KISS: beginner's problems with im_file and om_file

```

Hello nxlog world,

Shamed to say, I've spent entire yesterday trying to figure out how to read Windows DHCP log files and ship the events to ElasticSearch.

Problem was with using direct path for folder C:\Windows\System32\dhcp\. Managed to get nxlog to read by sharing the folders (read-only permissions) to the user account used for nxlog service account logon.

As the events were not showing in ES, I'm stuck with trying to write the events into another file, in order to confirm that the source files are being read correctly.

 


OS: Win Srv 2008 R2 Ent

nxlog: v 2.9.1347

Here is the nxlog.conf:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

# debugging only:
LogLevel DEBUG
NoCache TRUE

  <Input msdhcp>
      Module      im_file
      File        '\\DC5\dhcp\DhcpSrvLog-*.log'
      SavePos     TRUE
      InputType   LineBased
      Exec if $raw_event =~ /^#/ drop();
      Exec $message = $raw_event;
  </Input>

  <Input dns>
      Module      im_file
      File        '\\DC5\dns\dns.log'
      SavePos     TRUE
      InputType   LineBased
      Exec if $raw_event =~ /^#/ drop();
      Exec $message = $raw_event;
  </Input>

  <Output file_test>
      Module      om_file
      File        'C:\Program Files (x86)\nxlog\data\test_file_output.txt'
#      Sync    TRUE
      OutputType   LineBased
  </Output>

<Route test>
     Path  msdhcp,dns => file_test
</Route>


As a result, only DNS events are written in the output file:

21.1.2016. 11:34:00 A6A8 PACKET  0000000003B27E90 UDP Snd 192.168.105.12  3f0d R Q [8085 A DR  NOERROR] A      (8)PLANKING(3)lab(5)rador(0)

21.1.2016. 11:34:00 A6A8 PACKET  0000000003EDA2C0 UDP Rcv 192.168.105.12  3c32   Q [0001   D   NOERROR] A      (8)PLANKING(3)lab(5)rador(0)

21.1.2016. 11:34:00 A6A8 PACKET  0000000003EDA2C0 UDP Snd 192.168.105.12  3c32 R Q [8085 A DR  NOERROR] A      (8)PLANKING(3)lab(5)rador(0)


..but only the new ones, i.e. as the source DNS log file is being appended.

I have tried modifying the SavePos parameter to FALSE of both input modules, but to no avail - same result.

 

Questions:

1. What would be the correct configuration of global NoCache and module specific SavePos parameters, in order to read and output the complete source file, regardless of prior attempts?

2. What is the reason DHCP logs (using wildcard) are not being read (or at least written in the output), as opposed to the same configuration for DNS logs?

 

 

I will provide nxlog debug level log if needed. No visible errors there.

 

Any help greatly appreciated!

```


djontra
Replies: 1
View post »
adm
Issue selecting specific levels of windows application logs in NXLog

```

I'm trying to pass only Warning / Error / Critical level Application Logs through NXLog to my ELK stack. When I have this configuration

<Input EventLogIn> Module immsvistalog

<QueryList>\ <Query Id="0">\

<Select Path="Application"></Select>\

</Query>\ </QueryList>

Exec to_json(); </Input>

everything works fine, and I'm collecting all levels of Application logs. I tried putting in a parameter on the <Select Path> line like this

<Select Path="Application">[Application/Level=1]</Select>\

And it craps itself and I get nothing. NXLog isn't reporting any issue, and I'm not seeing anything on the logstash side of things.

I got the information about Event Viewer querying from this thread and adapted it to my use case: https://serverfault.com/questions/543494/query-specific-logs-from-event-log-using-nxlog

```


pcort42
Replies: 1
View post »
pcort42
Attempting to build nxlog with updated libraries, stuck at libapr-1 running ./configure

```

I'm attempting to build nxlog with some updated libraries:

  • Latest APR (1.5.2)
  • Non-Heartbleed vulnerable OpenSSL sources
  • PCRE 8.37
  • Zlib 1.2.8

After building all the dependencies I'm a little stuck on getting nxlogs to build, specifically I'm stuck on the step where I run ./configure

At first it couldn't find apr-1-config, so I added /local/apr/bin to the path.

Then it couldn't fine libapr-1 so I added /local/apr/lib to the path, this is where the problems started. When APR built there wasn't a "libapr-1" file in /local/apr/lib, only libapr-1.a, libapr-1.la, libapr-1.dll.a.

Did I build APR incorrectly?

I'm trying to build this on windows

List of steps to get where I am:

1. Install MINGW using MinGW Installation Manager

Add packages:

  • mingw-developer-toolkit
  • mingw-base
  • mingw-expat bin
  • mingw32-libexpat dev
  • msys-libopenssl dev
  • msys-automake
  • msys-autoconf

Setup msys fstab (c:/mingw     /mingw)

2. Install Python (2.5)

3. Add Python and mingw to system path (C:\Python25;C:\MinGW\bin;C:\MinGW\msys\1.0\bin)

3. Get and build APR source (I could not get APR iconv to compile)

Download:

  • http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz
  • http://mirror.nexcess.net/apache//apr/apr-1.5.2-win32-src.zip
  • http://mirror.nexcess.net/apache//apr/apr-util-1.5.4-win32-src.zip
  • http://sourceforge.net/projects/pcre/files/pcre/8.37/pcre-8.37.zip/download
  • http://zlib.net/zlib128.zip

Build:

  1. Extract all files to c:\mingw\msys\1.0\src
  2. Compile libiconv
    1. cd libiconv-1.11
    2. 
      ./configure CFLAGS="-O2 -s -mms-bitfields -march=i686" CXXFLAGS="-O2 -s -mms-bitfields -march=i686"
    3. make && make install
  3. Compile APR
    1. cd apr
    2. ./buildconf
    3.  
      
      ./configure CFLAGS="-O0 -s -mms-bitfields -march=i686" CXXFLAGS="-O0 -s -mms-bitfields -march=i686"
    4. make && make install
    5. cd ..
  4. Compile APR-UTIL
    1. cd apr-util-1.5.4
    2. ./buildconf --with-apr=/usr/src/apr-1.5.2
    3. 
      ./configure CFLAGS="-O2 -s -mms-bitfields -march=i686" CXXFLAGS="-O2 -s -mms-bitfields -march=i686" --with-apr=/usr/src/apr-1.5.2
    4. make && make install
    5. cd ..
  5. Compile PCRE
    1. cd pcre-.37
    2. ./configure
    3. make && make install
    4. (make threw an error corrected with make clean, autoconf -i --force, started back at step 1)
    5. cd ..
  6. Compile ZLIB
    1. cd zlib-1.2.8
    2. make -f win32/Makefile.gcc
  7. Compile nxlog
    1. cd nxlog-ce-2.8.1248
    2. ./configure

This is where the problems began. First it couldn't find apr-1-config.

Fixed by adding /local/apr/bin to path.

Now it can't find libapr-1, addint /local/apr/lib to the path doesn't help. There is no libapr-1 file in the MinGW directory tree. Ideas?

 

-pacmanwa

 

```


pacmanwa
Replies: 1
View post »
pacmanwa
Performance statistics/measurements of nxLog on Windows

```

Are there any numbers about how nxLog performs when it is processing a high rate of messages being placed into a log file?

Right now we have a couple of incidents which resulted in a few thousand messages being logged per second.

I assume this is more than nxLog can handle but am wondering about any performance testing that has been run

```


J_Grieb
Replies: 1
View post »
JasonHuxLey
NXLog Parsing XML

```

I've seen some posts from about a year ago that NXLog is unable to parse attributes using xm_xml, I just wanted to check if this is still true?

I am running NXLog as a service on Windows machines and want to be able to parse the following message, is it possible?

<log4j:event logger="com.sentry.test.LogContextListener" timestamp="1437661699866" level="TRACE" thread="localhost-startStop-1"> <log4j:message><![CDATA[This is a trace message about how we should use C#]]></log4j:message> </log4j:event>

```


Jakauppila
Replies: 1
View post »
adm