Hello,

Has anyone ever set up NxLog to forward windows events to any log aggregator or SIEM that accetps LEEF format? I see the enterprise edition has a LEEF module but wanted to see if this had been done or if there are any issues in doing so. 

I have installed nxlog on our 2012R2 DC's.  I go into the file and uncomment out the path to the software.  I then replace the IP address of syslog server with ours and then save the file.  I then go and try and start the nxlog service and immediatly get an error 1053:  The service did not respond to a control request in a timely manner.

I look in the nxlog log file and see the following error message --->  nxlog failed to start: Couldn't change to SpoolDir '%ROOT%\data'
The system cannot find the path specified. 

I know this error message is incorrect because the same path is used for CacheDir, Pidfile, and LogFile and those seem to be working. 

Upon further experimentation if I comment out the Logfile path as well as the Logfile path I can get the service to start but no logs are sent over to my syslog server.  

I find it funny that even thought the error is for the SpoolDir and the Logfile seems to be working I have to comment out both items to get the service to start otherwise I continue to get the Error 1053.

I'm hoping someone can help with this.   Thanks.

Need some help, I want the fields "$srcip, $srcport, $dstip, $dstport" to be put together in another field, called "$netinfo", how do I do it ??

My logs

#fields    ts    uid    id.orig_h    id.orig_p    id.resp_h    id.resp_p    proto    trans_id    query    qclass    qclass_name    qtype    qtype_name    rcode    rcode_name    AA    TC    RD    RA    Z    answers    TTLs    rejected
#types    time    string    addr    port    addr    port    enum    count    string    count    string    count    string    count    string    bool    bool    bool    bool    count    vector[string]    vector[interval]    bool
1482865188.959602    CMyjvLxxxxxxx0MJjb    xxx.xx.192.250    3xxx    xxx.xxx.162.xxx    53    udp    19626    -    -    -    -    -    0    NOERROR    F    F    F    F    0    -    -    F
1482865189.162798    CW1kwxxxxxxxC3Ug0j    xxx.xx.192.250    xxxx5    xxx.xxx.xxx.xxx    53    udp    250    r4.sn-a5m7znes.googlevideo.com    -    -    -    -    0    NOERROR    T    F    F    F    0    xxx.194.xxx.233    1800.000000    F
1482865189.182565    Cir6Sz3xxxxxO60PD6    fe80::xxx:f35c:xxxx:61ad    65535    ff02::1:3    5355    udp    1772    host    1    C_INTERNET    1    A    -    -    F    F    F    F    0    -    -    F

Nxlog .conf
<Extension csv.dns.log>
 Module xm_csv
 Fields $timestamp,$uid,$srcip,$srcport,$dstip,$dstport,$service,$transid,$qresponse,$query,$qclass,$qclassname,$qtype,$qtypename,$rcode,$rcodename,$aa,$tc,$rd,$ra,$z,$answ
 FieldsType string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string
 Delimiter \t
</Extension>
<Input i.dns.log>
 Module im_file
 File "/*PATH*/dns.log"
 ReadFromLast TRUE
 Exec csv.dns.log->parse_csv();
</Input>
<Output o.dns.log>
 Module om_ssl
 Host 192.XXX.X.XXX
 Port ZZZZ
 OutputType GELF_TCP
 CAFile /data/conf/ca.crt
 AllowUntrusted TRUE
</Output>
<Route r.dns.log>
 Path i.dns.log => o.dns.log
</Route>

Thank you

Hello,

I am currently using NXLog (nxlog-ce-2.9.1716) and I noticed that the snare output format has missing fields on the date :

Aug 23 15:03:59 HOSTNAME.DOMAIN MSWinEventLog    1    Security    121659     Aug 23 15:03:59 2017    4624    Microsoft-Windows-Security-Auditing    N/A    N/A    Success Audit    HOSTNAME.DOMAIN    Logon        An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Impersonation    New Logon:   Security ID:  S-1-5-21-2980885132-2242275795-2054596362-15959   Account Name:  Account_name   Account Domain:  DOMAIN   Logon ID:  0x1F5DF4F   Logon GUID:  {E2944EC9-BBE0-21A7-50EF-C6A58DBD6A72}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: -   Source Network Address: X.X.X.X   Source Port:  50240    Detailed Authentication Information:   Logon Process:  Kerberos   Authentication Package: Kerberos   Transited Services: -   Package Name (NTLM only): -   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.    89247264
Aug 23 15:04:01 HOSTNAME.DOMAIN MSWinEventLog    1    Security    121661     Aug 23 15:04:01 2017    4624    Microsoft-Windows-Security-Auditing    N/A    N/A    Success Audit    HOSTNAME.DOMAIN    Logon        An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Delegation    New Logon:   Security ID:  S-1-5-21-2980885132-2242275795-2054596362-6841   Account Name:  Account_name   Account Domain:  DOMAIN   Logon ID:  0x1F5E7AD   Logon GUID:  {14B095C8-B17D-04D1-37E4-0A09F32E47BA}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: -   Source Network Address: X.X.X.X   Source Port:  63574    Detailed Authentication Information:   Logon Process:  Kerberos   Authentication Package: Kerberos   Transited Services: -   Package Name (NTLM only): -   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.    89247266

There should be the day like Wed Aug 23 15:04:01 2017, and I don't know if I can modify the configuration to add this field...

Moreover I also noticed that the space between the ID and the date (  121661     Aug 23 15:04:01 2017) is not always the same. Sometime there is a TAB and sometime just a space between the fields.

Is there any configuration to make it the same for each and every logs ? This cause my SIEM to not parse every logs correctly...

Thank you.

Regards,

Alexis H.

Tried on different machines, on different ports. Localy, remotely, syslog generators, real devices. Absolutely nothing. If I just change in conf to im-tcp  - it works. im_udp none.

This is a very interesting issue and I was wondering if anyone has encountered it before. I have ~200 development systems that I wish to gather windows event information from however, there are toolsets within this environment that are interrupted by NXLog.

NXLog is connecting to eventlog API and polls (pause/resume) the API every second (configurable) for new events within the various subscribed to channels. My issue is with reference information loaded by the event log API due to the active connection from NXLog. The reference dll is loaded into the eventlog service but is not released because NXLog does not release the call, it pauses and resumes.  This in turn causes our development automation to fail since the automation tools are looking to replace this dll once a dev pass is complete.

Is there an exec example to reconnect so that the eventlog service will release these dlls?  Something like:

<Input eventlog>

   Module im_msvistalog
   Query\
    <QueryList>\
     <Query Id="0">\
      <Select Path="Application">*</Select>\
     </Query>\
   </QueryList>
   Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;
   Exec $raw_event='{"jsonEvent":"Windows","Event":' + to_json() + '}';
    <Schedule>
         Every 30 sec
         Exec  eventlog->reopen();
     </Schedule>
</Input>

How can nxlog for windows be configured to send timestamp in high-precision forwarding format?

Similar to the RSYSLOG_ForwardFormat high-precision timestamps including year with timezone UTC offset information.

For example:

2017-08-22T18:36:28.568230+00:00

It is currently just being sent with ONLY Mon day hour:min:sec 

<14>Aug 22 15:16:46 Win7Prox64 MSWinEventLog 1 System 2 Tue Aug 22 15:16:46 2017 7036 Service Control Manager N/A N/A Information Win7Prox64 N/A The nxlog service entered the running state. 170362

Thanks!

Hello,

We are using nxlog to write all our event logs to syslog, but have a need for them to be formatted as CSV instead of the tab delimited it appears to be currently. Is anyone doing this currently and mind sharing their config, or know if a way to process this correctly?

Thanks!

Hello.

I like to collect Windows Event ID and send it to Linux Box. I installed "Syslog-NG" on Linux and it collect Windows Event Log very good but I have a problem about "Exec convert_fields("AUTO", "utf-8");". When I enable it in my config file then my "nxlog" can't send log!!!

My "nxlog" config is:

## This is a sample configuration file. See the nxlog reference manual about the

## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_syslog
    AutodetectCharsets utf-8
</Extension>

<Input in>
    Module      im_msvistalog
# For windows 2003 and earlier use the following:
#   Module      im_mseventlog
    Exec if $EventID NOT IN (4660, 4663) drop(); 
    Exec convert_fields("AUTO", "utf-8");
</Input>

<Output out>
    Module      om_udp
    Host        172.30.9.20
    Port        514
    Exec        to_syslog_snare();
 </Output>

<Route 1>
    Path        in => out
</Route>

Any idea?

Thank you.

Has anyone gotten nxlog running on linux to receive windows logs through Windows Log Collector initiated by a GPO? In simple terms, using nxlog to receive Windows logs without a nxlog agent running on Windows? If so, any documentation on how to make that happen?

Thank you in advance, Bruce M. Wink

Hello

I have noticed the on the download page of NXLog Community Edition the versions of the Windows and Linux packages are 2.9 while the source code (.tar.gz) is of the lower version 2.8.

Could the source code package please be updated?

We are missing the GELF_TCP feature which appeared in 2.9.1347.

Thank you
Michael

Hi all,

I can...

 - rotate log files based on size.

 - rotate log files based on size using event fields- such as $Hostname from Syslog

 - rotate log files based on size and schedule.

I CANNOT rotate log files based on size and schedule using event fields !  It seems that the Schedule component does not like references to fields.

At end of tether, please help.

ERROR invalid om_file keyword: $newfilepath

Thanks in advance. Paul

Back to conversation about current workaround... Windows Server 2016 has more than 256 channels. Is it possible to create a second thread/instance to subscribe for the remaining channels? I can try to guess and create XML filter to exclude some unneeded for now but tomorrow MS can create more channels with some update and would be nice if it handled automatically.

I am trying to use the om_http module to POST events to an AWS API Gateway (which won't allow SSLv3 connections). Looking at this forum and the documentation, it seems like the latest version of the community edition, nxlog-ce-2.9.1716, should support TLSv1.2, but I keep getting the following error when it attempts to connect to my API:

ERROR SSL error, SSL_ERROR_SSL: retval -1, sslv3 alert handshake failure

Below is my config for the output:

Is the Output module to ElasticSearch available/will be available in the community edition?

When I last checked it was a feature of the commerical edition only. 

I have been a nxlog champion for years now and have been forwarding to logstash. However with Elasticsearch ingest nodes, there is one extra redundent step now.

This will tip the scale to moving off into native beats/rsyslog if there are no plans to make this available.

Thanks

Ash Kumar

Hi there,

this might be a strange question but I'm new nxlog and was wondering what the difference between an input module and a processor module is.
the background to my question is:
I want to collect logs centrally on an nxlog server, and have configured all my clients to send their logs in a nxlog-binary format. at the moment I'm testing this with Windows IIS webserver logs.

on the nxlog server i want to read the binary logs and process them. Based on the processor I'd like to forward them to two separate destinations, outputting the logs in GELF and CSV format.

Now my question:

While this codeblock in the Input module works fine:

<Input in_syslog_tcp>

  Module  im_tcp

  Host  0.0.0.0

  Port  80

 

  <Exec>

        $Hostname = hostname();

        w3c->parse_csv();

        $EventTime = parsedate($date + " " + $time);

        $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;

        $SourceName = "IIS";

         w3c->to_csv();

  </Exec>

  InputType Binary

</Input>

<Processor transform_iis>
  Module      pm_null
   <Exec>

        $Hostname = hostname();
        w3c->parse_csv();
        $EventTime = parsedate($date + " " + $time);
        $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;
        $SourceName = "IIS";
        w3c->to_csv();
  </Exec>
</Processor>

Hi,

My scenario is:  (Windows server + nxlog configured for Windows events) => Logstash => Elasticsearch

I am wondering where nxlog stores current informations about sent Windows Events (for every category).
If i need to resend a few Windows events from past, how i can do it?

Is there any way to select last X hours (example: last 48 hours from Security category, from the starting nxlog service moment).
SavePos and ReadFromLast are helpful, but if both are false, event logs are sent from the first one stored on Windows server.

Also, where nxlog save all events in case of temporary lost tcp connection? This is probably SavePos location.

Thanks

Hi,

I've to use the same exec block multiple times in different paths. I've already tried it with a processor module, but unfortunately a processor module could only be used once. A possible solution is to implement them in the input modules, but I've more than one input module (syslog tcp/udp alone are already two) which ends in multiple code in different modules. But I want them once and want to use them multiple times.

Have someone an idea how to implement this?

Thanks in advance.

br

Hi,

My Server's TimeZone is EST(UTC-5) but the application running in the server has EET(UTC+2). The log file generated is names as <Application Date> ie todays log file is named as 20170717.log. Today's log will be truncated at 00:00 (UTC+2) and new log file 20170718.log will be created and old log file will remain in the same directory.

How can I configure NxLog to read the current log only