define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules

CacheDir %ROOT%\data

Pidfile %ROOT%\data\nxlog.pid

SpoolDir %ROOT%\data

LogFile %ROOT%\data\nxlog.log

#### Extensions - load and configure global extendsions

<Extension eW3c>

    Module xm_csv

    Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $sc-status, $sc-substatus, $sc-win32-status, $time-taken

    FieldTypes string, string, string, string, string, string, integer, string, string, string, integer, integer, integer, integer

    Delimiter ' '

    QuoteChar '"'

    EscapeControl FALSE

    UndefValue -

</Extension>

<Extension eFileOp>

    Module xm_fileop

</Extension>

####

#### Inputs

<Input iIIS-fulfillment.Api.1>

    Module    im_file

    File     "D:\\APShared\\Log\\W3SVC1\\\\*"

    SavePos    TRUE

    

#    Exec if $raw_event =~ /^#/ drop();

    Exec file_write("C:\\temp\\nxlog_RawIIS.log", $raw_event+"some text man"); \

        eW3c->parse_csv(); \

        $SourceName = "IIS-fulfillment.Api"; \

        file_write("C:\\temp\\nxlog_IIS-csv.log", $raw_event);

</Input>

####

#### Processor

####

#### Outputs

<Output oIISFile>

    module    om_file

    File    "c:\\temp\\nx_iis.out"

</Output>

####

#### Routes

<Route rIISFile>

    path     iIIS-fulfillment.Api.1 => oIISFile

</Route>

NXLOG.log

2015-03-04 12:13:45 INFO nxlog-ce-2.8.1248 started

2015-03-04 12:14:11 ERROR procedure 'parse_csv' failed at line 40, character 94 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 14, got 0 in input ''

"C:\\temp\\nxlog_RawIIS.log"

contains: some text man

C:\\temp\\nxlog_IIS-csv.log - File not created.

Help.  Where's my $raw_event data?

I have xml style messages that can have thousands of <CUID>xxx</CUID> statements in them so I need to cut out from the middle of the message: from the first string of <CUID> to the last </CUID>. Is this possible?

Hello,
I have followed instructions to install nxlog in a Windows Server Core 2012 machine (no UI only command prompt).

However, when I try to run the service I get: service could not be started

Do you know what I could try to fix this?

This Windows is the instance created automatically by Amazon Beanstalk and without a UI I find this difficult to debug.. so any suggestion is appreciated

Thanks

Dear NXlog community,

I am using nxlog on a windows 2003 environment and i am having some problems with windows failed authentication events.  All entry's with EventID 675 contain the AccountName "SYSTEM" in stead of the username that the failed authentication is for. I couldnt get it to work with pattern matching in nxlog but as i have never used this before i am probably doing something wrong. I would really like to get some statistics of this and get the user name in the AccountName field.

For example kibana is reporting:

AccountName        SYSTEM
AccountType        User
Category        Account Logon
CategoryNumber        9
Domain            NT AUTHORITY
EventID            675
EventType        AUDIT_FAILURE
FileName        Security
Hostname        SomeHostName
Severity        ERROR
SeverityValue        4
SourceModuleName    eventlog
SourceModuleType    im_mseventlog
SourceName        Security
host            SomeHostName.SomeDomain
message            Pre-authentication failed:
             User Name: [username]
             User ID: %{some user id}
             Service Name: krbtgt/office Pre-Authentication
             Type: 0x0 Failure Code: 0x19
             Client Address: [ip address]

Any help is appreciated!

Hello,

I am trying to connect through SSL to ship logs but nxlog only attempts to connect through SSLv3, not TLS. Any way to force TLS 1.1 or 1.2 handshakes?

Thank you

I'm noticing something weird with my NXLog endpoints. If I have my NXLog agents pull windows event logs and transfer them over the network it loses the event time somewhere along the way. My current setup looks like this:

Remote Agent: Windows Event Log -> im__mseventlog -> om_SSL (Type Binary) (to Log Collector)

Log Collector: im_SSL (Type Binary) -> om_UDP (Type GELF) (to Graylog server)

When Graylog is offline it will buffer correctly, but after Graylog comes back online, NXlog will send the logs with the current time.

Hi guys, because the event log doesn't subscribe to debug logs and because I have an app that mangles it's entries I was wondering if you can run a powershell command encapsulated in an if.

eg:

if $SourceName == "IHateThisApp" \

{ \

$Message = *encapsulated powershell command* ;  }

Of if someone know a way to get the debug info out and parsed into nxlog.

Hi

I read the help but still don't understand how to send this log that would like to call impaxauditlog from a windowsbox text file (that is overwritten) to my syslog server. Below are some examples from the log and my current nxlog config setting.

Basically the entries take two lines one starts with !SYS and the other with <IHEYr4>. So based on the example the "!SYS 2015-02-23 11:51:49,478"  is the time the event was logged and "<TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp>" is the real event datetime that I would like use as syslog message datetime (you can ask AGFA for why the difference:) if this is not possible then of course I can settle with the 2015-02-23 11:51:49,478.

In essence the message itself should be everything that is between <IHEYr4></IHEYr4> tags BUT there is one big but - I need to parse the message a little and cut out everything that is between fist <CUID> and last </CUID> as there can be thousands of these and this is just wasting my diskspace. So the syslog message should look somethng like this:

2015-02-23T10:08:59+02:00 windowsbox impaxauditlog <IHEYr4><DicomQuery><CUID></CUID><SyntaxUID>LittleIndianImplicit</SyntaxUID></DicomQuery><Host>xx.xx.xx.xx</Host><TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp></IHEYr4>

Example:

!SYS 2015-02-23 11:51:49,478 - veebr 23 10:08:59 172.16.10.33 <?xml version="1.0" encoding="UTF-8" ?>
<IHEYr4><DicomQuery><CUID>1.3.12.2.1107.5.1.4.65588.30000015021704415668700013901</CUID><CUID>1.3.12.2.1107.5.1.4.65588.30000015021706453421800094763</CUID><SyntaxUID>LittleIndianImplicit</SyntaxUID></DicomQuery><Host>xx.xx.xx.xx</Host><TimeStamp>2015-02-23T10:08:59+02:00</TimeStamp></IHEYr4>

Main part of my current config:

<Input in>

    Module      im_msvistalog
</Input>

<Extension syslog>
Module xm_syslog
</Extension>

<Output sslout>
Module om_ssl
Host xx.xx.xx.xx
Port 10514
Exec to_syslog_ietf();
CAFile %CERTDIR%/ca.cert.pem
CertFile %CERTDIR%/cert.pem
CertKeyFile %CERTDIR%/key.pem
AllowUntrusted TRUE
OutputType Syslog_TLS
</Output>

<Route 1>
    Path        in => sslout
</Route>

Hello,

Having an issue with using om_ssl from a Windows NXlog client to a Linux ELK server.

The issue lies with the encryption part, the connection is established on the associated destination SSL port I set but no SSL traffic can be see/captured.

For information here is the logstash config and the NXlog config, I believe the issue lies with the certs.

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
  tcp {
    port => 5001
    type => syslog
    ssl_cacert => "/etc/pki/tls/certs/rootCA.pem"
    ssl_cert => "/etc/pki/tls/certs/logstash.crt"
    ssl_key => "/etc/pki/tls/private/logstash.key"
    ssl_enable => true

nxlog.conf:
<Output syslogout>
#This is for TCP non-SSL traffic
  Module om_tcp
  Host 192.168.0.20
  Port 5000
</Output>

<Output omsslout>
#This is for SSL traffic only, omit all "#"
  Module          om_ssl
  Host            192.168.0.20
  Port            5001
  CAFile          %CERTDIR%\rootCA.pem
  OutputType LineBased
  AllowUntrusted FALSE
</Output>

<Route 1>
  Path eventlog => eventlog_transformer => omsslout
</Route>

Only included relative SSL parts as everything else works over TCP and UDP.

Many thanks

Hello, is there any chance that nxlog supports Output om_ssl with OutputType GELF? As descriped here https://www.mail-archive.com/nxlog-ce-users@lists.sourceforge.net/msg00397.html currently NXLog only supports GELF via om_udp .

It would be great, because Graylog now (19. February 2015) supports TLS for all standard TCP and AMQP inputs.

I am getting an error message to nxlog.log. 2015-02-17 08:16:23 INFO nxlog-ce-2.8.1248 started 2015-02-17 08:16:35 ERROR Couldn't read next event, corrupted eventlog?; The data is invalid. And after this error no more events or log messages are generated. However I can read event log with event viewer and I can see new events. These are messages in my graylog2: 2015-02-17 08:16:35.000 wintoosa Couldn't read next event, corrupted eventlog? The data is inval 2015-02-17 08:16:23.000 wintoosa nxlog-ce-2.8.1248 started

Seems that nxlog is running but it can't handle events after this error. How can I fix this ?

My very basic nxlog.conf:

define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log #LogLevel DEBUG

<Extension syslog> Module xm_syslog </Extension>

<Extension gelf> Module xm_gelf </Extension>

<Input internal> Module im_internal </Input>

<Input eventlog> Module im_msvistalog

For windows 2003 and earlier use the following:

Module im_mseventlog

</Input>

<Output out> Module om_udp Host 10.0.0.103 Port 12900 outputType GELF </Output>

<Route 1> Path internal, eventlog => out </Route>

Botond,

I would like to write logs out to a message queue such as kafka or redis or rabbitmq etc. Is there a road map for that? Alternately would a workaround such as writing out in JSON format to a om_tcp destination work?

Can I also track the state of a tcp connection and send output to an alternate destination? I could write an external shell that would track the nxlog.log for "connection lost/made" and decision off that but I suspect that would be very expensive.

Thanks

Ash 

Hi everyone,

We have an internal application on a windows box which logs in a way which is slightly mangled. All other windows logs come through nxlog but on these messages the entire message field gets dropped.

Is there a way to parse using ifs? eg:

if $service == "Homegrown app" {

}

So that I can either change the output format or parse them untouched into a file etc...

Does anyone know if there are any characters that don't get escaped properly that might mangle the json output?

Sorry for not giving a huge amount of information, I have to keep this as agnostic as I can.

Is there a way to use the Community Edition to log to MS SQL Server? I have tried writing a Perl script to dump the text log files into the database, but have had no luck. I understand that you can use im_odbc and om_odbc in the Enterprise version, but not the Community Edition (CE). Has anyone had any luck doing this in the CE?

Hi,

I am trying to parse a log4net file into json.

Here's my sample log4net:

----------------

2015-01-27 01:06:18,859 [7] ERROR Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider [(null)] - Get taxonomy Type Failed for Tools
2015-01-27 06:34:31,051 [26] ERROR www.Status404 [(null)] - ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20
    UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36
    HostAddress: 192.168.10.2
    RequestUrl: /ErrorPages/404.aspx
    MachineName: QA01
    Raw Url:/undefined/
    Referrer: http://qa1.www.something.com/toolset.aspx

2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled

Now I am using xm_multiline to capture each log entries.

----------------

<Extension multiline>
    Module        xm_multiline
    HeaderLine    /^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/
    EndLine        /\r?\n\r?\n^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/
</Extension>

I use a regex to capture the timestamp as the header then I use a regex to capture twice newline then the next timestamp as endline. However it still treat the second and last entry as ONE log entry.

Here's the output:

----------------

{  
   "EventReceivedTime":"2015-01-27 01:06:35",
   "SourceModuleName":"log4net",
   "SourceModuleType":"im_file",
   "time":"2015-01-27 01:06:18,859",
   "thread":"7",
   "level":"ERROR",
   "logger":"Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider",
   "ndc":"(null)",
   "message":"Get taxonomy Type Failed for Tools"
}{  
   "EventReceivedTime":"2015-01-27 06:34:35",
   "SourceModuleName":"log4net",
   "SourceModuleType":"im_file",
   "time":"2015-01-27 06:34:31,051",
   "thread":"26",
   "level":"ERROR",
   "logger":"www.Status404",
   "ndc":"(null)",
   "message":"  ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20\r\n  UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 
Safari/537.36\r\n  HostAddress: 192.168.10.2\r\n  RequestUrl: /ErrorPages/404.aspx\r\n  MachineName: QA01\r\n  
Raw Url:/undefined/\r\n  Referrer: http://qa1.www.something.com/toolset.aspx\r\n\r\n2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled"
}

I used this to produce that output:

----------------

Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- (.*)/s \
                { \
                    $time = $1; \
                    $thread = $2; \
                    $level = $3; \
                    $logger = $4; \
                    $ndc = $5; \
                    $message = $6; \
                    to_json(); \
                } \
                else \
                { \
                    drop(); \
                }

I've also tried to tweak it by using this to avoid the combining the last two entries as one. However I am not able to get the last entry anymore.

----------------

Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- ([\s\S]*?)(\r?\n\r?\n|$)/ \
                { \
                    $time = $1; \
                    $thread = $2; \
                    $level = $3; \
                    $logger = $4; \
                    $ndc = $5; \
                    $message = $6; \
                    to_json(); \
                } \
                else \
                { \
                    drop(); \
                }​

Will there be a Nxlog for Centos 7.

If so are there an expected date.

BTW has anybody succeded in compiling the tar file. I simply cannot figure out how to build the RPM files.

Regards

Jesper

I am having an issue where I am outputing a DNS log from Windows Server and having NXLOG read it and ship it to a Logstash environment. My issue however is that after a period of time the log file disappears and is not created. I am assuming it is an issue with NXLog having a lock on the file and when windows rotates it, it fails and doesnt get created.

The DNS Event View Log shows:

The DNS server was unable to open file E:\Log Files\DNS\dns.log for write.  Most likely the file is a zone file that is already open.  Close the zone file and re-initiate zone write.

Stopping and restarting the service does not help unless I completely stop NXLOG first. How can I change it so NXLog does not completely lock the log file so that windows can't rotate it?

This seems to occur on various versions of windows from 2008 to 2012 R2.

Appreciate any help you can provide regarding this issue.

Just installed nxlog to begin forwarding events to AlienVault, everything seems to be working so far with reading and forwarding events from the windows log using the im_msvistalog module.  One thing we would like to change to set the Syslog Facility before forwarding it AlienVault.  Have been unable to locate how to do so.

Hi, i used your nxlog-CE but only my Windows 7 hosts don't exhibit this issue.

"Nome dell'applicazione che ha generato l'errore: nxlog.exe, versione: 0.0.0.0, timestamp: 0x53ca79be
Nome del modulo che ha generato l'errore: im_msvistalog.dll, versione: 0.0.0.0, timestamp: 0x53ca79bd
Codice eccezione: 0xc0000005
Offset errore 0x00001cca
ID processo che ha generato l'errore: 0xbfc
Ora di avvio dell'applicazione che ha generato l'errore: 0x01d02f68896fae85
Percorso dell'applicazione che ha generato l'errore: C:\Program Files (x86)\nxlog\nxlog.exe
Percorso del modulo che ha generato l'errore: C:\Program Files (x86)\nxlog\modules\input\im_msvistalog.dll"

nxlog community edition version 2.8.1248

Many thanks.

Hi,

I noticed the NXlog service is not started at the end of the install process. I would the service to start automatically at the end of the install of the MSI.

Our goal is:

- modify the conf file in the MSI file to have it ready out of the box.

- Deploy with GPO on all servers.

For this second step, we need to ensure it starts automatically during the install process. The best would be to edit the install file to make a "net start nxlog" at the end.

Thanks

PM