Hello,   I am currently using NXLog (nxlog-ce-2.9.1716) and I noticed that the snare output format has missing fields on the date :   Aug 23 15:03:59 HOSTNAME.DOMAIN MSWinEventLog    1    Security    121659     Aug 23 15:03:59 2017    4624    Microsoft-Windows-Security-Auditing    N/A    N/A    Success Audit    HOSTNAME.DOMAIN    Logon        An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Impersonation    New Logon:   Security ID:  S-1-5-21-2980885132-2242275795-2054596362-15959   Account Name:  Account_name   Account Domain:  DOMAIN   Logon ID:  0x1F5DF4F   Logon GUID:  {E2944EC9-BBE0-21A7-50EF-C6A58DBD6A72}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: -   Source Network Address: X.X.X.X   Source Port:  50240    Detailed Authentication Information:   Logon Process:  Kerberos   Authentication Package: Kerberos   Transited Services: -   Package Name (NTLM only): -   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.    89247264 Aug 23 15:04:01 HOSTNAME.DOMAIN MSWinEventLog    1    Security    121661     Aug 23 15:04:01 2017    4624    Microsoft-Windows-Security-Auditing    N/A    N/A    Success Audit    HOSTNAME.DOMAIN    Logon        An account was successfully logged on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Impersonation Level:  Delegation    New Logon:   Security ID:  S-1-5-21-2980885132-2242275795-2054596362-6841   Account Name:  Account_name   Account Domain:  DOMAIN   Logon ID:  0x1F5E7AD   Logon GUID:  {14B095C8-B17D-04D1-37E4-0A09F32E47BA}    Process Information:   Process ID:  0x0   Process Name:  -    Network Information:   Workstation Name: -   Source Network Address: X.X.X.X   Source Port:  63574    Detailed Authentication Information:   Logon Process:  Kerberos   Authentication Package: Kerberos   Transited Services: -   Package Name (NTLM only): -   Key Length:  0    This event is generated when a logon session is created. It is generated on the computer that was accessed.    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.    The impersonation level field indicates the extent to which a process in the logon session can impersonate.    The authentication information fields provide detailed information about this specific logon request.   - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.   - Transited services indicate which intermediate services have participated in this logon request.   - Package name indicates which sub-protocol was used among the NTLM protocols.   - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.    89247266   There should be the day like Wed Aug 23 15:04:01 2017, and I don't know if I can modify the configuration to add this field...   Moreover I also noticed that the space between the ID and the date (  121661     Aug 23 15:04:01 2017) is not always the same. Sometime there is a TAB and sometime just a space between the fields.   Is there any configuration to make it the same for each and every logs ? This cause my SIEM to not parse every logs correctly...   Thank you.   Regards, Alexis H.

Tried on different machines, on different ports. Localy, remotely, syslog generators, real devices. Absolutely nothing. If I just change in conf to im-tcp  - it works. im_udp none.

This is a very interesting issue and I was wondering if anyone has encountered it before. I have ~200 development systems that I wish to gather windows event information from however, there are toolsets within this environment that are interrupted by NXLog. NXLog is connecting to eventlog API and polls (pause/resume) the API every second (configurable) for new events within the various subscribed to channels. My issue is with reference information loaded by the event log API due to the active connection from NXLog. The reference dll is loaded into the eventlog service but is not released because NXLog does not release the call, it pauses and resumes.  This in turn causes our development automation to fail since the automation tools are looking to replace this dll once a dev pass is complete. Is there an exec example to reconnect so that the eventlog service will release these dlls?  Something like: <Input eventlog>    Module im_msvistalog    Query\     <QueryList>\      <Query Id="0">\       <Select Path="Application">*</Select>\      </Query>\    </QueryList>    Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;    Exec $raw_event='{"jsonEvent":"Windows","Event":' + to_json() + '}';     <Schedule>          Every 30 sec          Exec  eventlog->reopen();      </Schedule> </Input>  

How can nxlog for windows be configured to send timestamp in high-precision forwarding format? Similar to the RSYSLOG_ForwardFormat high-precision timestamps including year with timezone UTC offset information. For example: 2017-08-22T18:36:28.568230+00:00   It is currently just being sent with ONLY Mon day hour:min:sec  <14>Aug 22 15:16:46 Win7Prox64 MSWinEventLog 1 System 2 Tue Aug 22 15:16:46 2017 7036 Service Control Manager N/A N/A Information Win7Prox64 N/A The nxlog service entered the running state. 170362 Thanks! p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; background-color: #fef49c} span.s1 {font-variant-ligatures: no-common-ligatures}

Hello, We are using nxlog to write all our event logs to syslog, but have a need for them to be formatted as CSV instead of the tab delimited it appears to be currently. Is anyone doing this currently and mind sharing their config, or know if a way to process this correctly? Thanks!

Hello. I like to collect Windows Event ID and send it to Linux Box. I installed "Syslog-NG" on Linux and it collect Windows Event Log very good but I have a problem about "Exec convert_fields("AUTO", "utf-8");". When I enable it in my config file then my "nxlog" can't send log!!! My "nxlog" config is:   ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>     Module      xm_syslog     AutodetectCharsets utf-8 </Extension> <Input in>     Module      im_msvistalog # For windows 2003 and earlier use the following: #   Module      im_mseventlog     Exec if $EventID NOT IN (4660, 4663) drop();      Exec convert_fields("AUTO", "utf-8"); </Input> <Output out>     Module      om_udp     Host        172.30.9.20     Port        514     Exec        to_syslog_snare();  </Output> <Route 1>     Path        in => out </Route>   Any idea?   Thank you.

I have an input for some log files, however each restart of the machine the log file is rotated to a new file. I'm trying to create an input that is able to track the rotated log file. I presume a wildcard can be used in the File: string and that the save position and read from last wouldnt cause NXlog to re read old files? Below you can see the example file path with the 'X' representing a number that changes each time the active log is rotated. C:\ProgramData\VMware\hostd\hostd-X.log   Regards, G

Has anyone gotten nxlog running on linux to receive windows logs through Windows Log Collector initiated by a GPO? In simple terms, using nxlog to receive Windows logs without a nxlog agent running on Windows? If so, any documentation on how to make that happen? Thank you in advance, Bruce M. Wink

Hello I have noticed the on the download page of NXLog Community Edition the versions of the Windows and Linux packages are 2.9 while the source code (.tar.gz) is of the lower version 2.8. Could the source code package please be updated? We are missing the GELF_TCP feature which appeared in 2.9.1347. Thank you Michael

Hi all, I can...  - rotate log files based on size.  - rotate log files based on size using event fields- such as $Hostname from Syslog  - rotate log files based on size and schedule. I CANNOT rotate log files based on size and schedule using event fields !  It seems that the Schedule component does not like references to fields. At end of tether, please help.   ERROR invalid om_file keyword: $newfilepath   <Extension syslog>     Module xm_syslog </Extension>   <Input in>     Module im_udp     Host 0.0.0.0     Port 514     Exec parse_syslog(); </Input>   <Output out>     Module om_file $newfilepath = "D:/Test/" + $Hostname + "/" + $SourceName + "/" $currfilename = strftime(now(), "%Y%m%d") + ".current" File $newfilepath + $currfilename   # Check the size of our log file every minute and rotate if it is larger than 1Mb <Schedule> Every 60 sec Exec if (out->file_size() >= 1M) \ { \ $newfilename = $newfilepath + strftime(now(), "%Y%m%dT%H%M%S") + ".s"); \ out->rotate_to($newfilename); \ } </Schedule>   CreateDir   TRUE </Output>   <Route 1>     Path in => out </Route>   Thanks in advance. Paul  

Back to conversation about current workaround... Windows Server 2016 has more than 256 channels. Is it possible to create a second thread/instance to subscribe for the remaining channels? I can try to guess and create XML filter to exclude some unneeded for now but tomorrow MS can create more channels with some update and would be nice if it handled automatically.

I am trying to use the om_http module to POST events to an AWS API Gateway (which won't allow SSLv3 connections). Looking at this forum and the documentation, it seems like the latest version of the community edition, nxlog-ce-2.9.1716, should support TLSv1.2, but I keep getting the following error when it attempts to connect to my API: ERROR SSL error, SSL_ERROR_SSL: retval -1, sslv3 alert handshake failure Below is my config for the output: <Output out>     Module       om_http     URL         https://<my api endpoint>     ContentType "application/json" </Output>   I have also tried using HTTPSAllowUntrusted TRUE, which doesn't change anything. Also, I tested the API endpoint with curl and it works fine.   Thanks in advance for any help!   UPDATE: It looks like nxlog is negotiating using TLSv1.2 (discovered with wireshark). I also discovered that AWS API Gateway requires the SNI extension to TLS. This is likely the problem if nxlog-ce doesn't support SNI. Any idea if/when that will be supported in the community edition?

Is the Output module to ElasticSearch available/will be available in the community edition? When I last checked it was a feature of the commerical edition only.  I have been a nxlog champion for years now and have been forwarding to logstash. However with Elasticsearch ingest nodes, there is one extra redundent step now. This will tip the scale to moving off into native beats/rsyslog if there are no plans to make this available.   Thanks   Ash Kumar  

Hi there, this might be a strange question but I'm new nxlog and was wondering what the difference between an input module and a processor module is. the background to my question is: I want to collect logs centrally on an nxlog server, and have configured all my clients to send their logs in a nxlog-binary format. at the moment I'm testing this with Windows IIS webserver logs. on the nxlog server i want to read the binary logs and process them. Based on the processor I'd like to forward them to two separate destinations, outputting the logs in GELF and CSV format. Now my question: While this codeblock in the Input module works fine: <Input in_syslog_tcp>   Module  im_tcp   Host  0.0.0.0   Port  80     <Exec>         $Hostname = hostname();         w3c->parse_csv();         $EventTime = parsedate($date + " " + $time);         $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;         $SourceName = "IIS";          w3c->to_csv();   </Exec>   InputType Binary </Input> This codeblock doesn't work: <Processor transform_iis>   Module      pm_null    <Exec>         $Hostname = hostname();         w3c->parse_csv();         $EventTime = parsedate($date + " " + $time);         $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;         $SourceName = "IIS";         w3c->to_csv();   </Exec> </Processor> Is there any reason why this has to go into the input module? The disadvantage would be that I would have to have several input modules for all kinds of different data sources....currently I have only one input module that receives all the logs in binary format from lots of different hosts..   Is there maybe a better solution for this?   cheers, micsnare

Hi, My scenario is:  (Windows server + nxlog configured for Windows events) => Logstash => Elasticsearch I am wondering where nxlog stores current informations about sent Windows Events (for every category). If i need to resend a few Windows events from past, how i can do it? Is there any way to select last X hours (example: last 48 hours from Security category, from the starting nxlog service moment). SavePos and ReadFromLast are helpful, but if both are false, event logs are sent from the first one stored on Windows server. Also, where nxlog save all events in case of temporary lost tcp connection? This is probably SavePos location.   Thanks  

Hi,   I've to use the same exec block multiple times in different paths. I've already tried it with a processor module, but unfortunately a processor module could only be used once. A possible solution is to implement them in the input modules, but I've more than one input module (syslog tcp/udp alone are already two) which ends in multiple code in different modules. But I want them once and want to use them multiple times. Have someone an idea how to implement this? Thanks in advance.   br    

Hi, My Server's TimeZone is EST(UTC-5) but the application running in the server has EET(UTC+2). The log file generated is names as <Application Date> ie todays log file is named as 20170717.log. Today's log will be truncated at 00:00 (UTC+2) and new log file 20170718.log will be created and old log file will remain in the same directory. How can I configure NxLog to read the current log only

hi all! im first time meet nxlog. it looks very simple and powerfull, but i need help! i have cisco which can send syslog to my nxlog service. logs looks like 2017-06-21T07:19:49.214Z,"vpn.domain.com","Jun 21 2017 10:19:55: %ASA-4-722051: Group <GroupPolicy_domain> User <123456@domain.com> IP <213.87.163.123> IPv4 Address <10.217.130.23> IPv6 address <fc00::59> assigned to session" please, help me write processor, which parse this string and get date (Jun 21 2017 10:19:55)? login (123456@domain.com) and external ip adress (213.87.163.123) and put them to text log file. thanks!

Hello, I need to parse logs mounted on file system NFS on centos. Is NXLog reliable with NFS ? Thanks, Regards

I hava some data  like this  -------------------------- a         9650     mktd     07:30     [OK    ] a    b         9650     mktd     07:30     [OK    ] b    c         9650     mktd      07:30     [OK    ] c     d         9650     mktd     07:30     [OK    ] d if data have change to this a        9650     mktd     07:30    [OK    ] a b        9650     mktd     07:30    [OK    ] b c        0                                     [FAILED] c d        0                                     [FAILED] d   how could I reload the file to nxlog I have been to tried to use PollInterval ,but it seem to use for new file and new log entries I also try to use file_touch,but both of them doesn't useful for me.   my scripts below   #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json>     Module      xm_json </Extension> <Extension fileop>     Module      xm_fileop </Extension> <Extension exec>     Module      xm_exec </Extension> <Extension kvp>     Module       xm_kvp     KVPDelimiter ,     KVDelimiter  =     EscapeChar   \\ </Extension> <Input test>        Module       im_file      File         'D:\winapp\log\chk2.txt'      SavePos      FALSE      ReadFromLast FALSE </Input>     <Processor pro>     Module       pm_null          Exec         delete($SourceModuleName);                            \                  delete($SourceModuleType);                            \                  delete($EventReceivedTime);                                                                    <Exec>                          if ($raw_event =~ /^(\w+)\s+(\d+)(.*)/)                   {                                                                           $app_name      = $1;                                                    $pid           = integer($2);                                           $msg           = $3;                       to_json();                                                          }                                                                       else                                                                    {                                                                         drop();                                                               }                                                                       if ($pid == 0)                                                          {                                                                          if($msg =~ / \s+\[(\w+)\] (.*)/)                                        {                                                                          $status      = $1;                                                      $event_time    = integer(now())/1000;                                   delete($msg);                                                           to_json();                                                           }                  }                  if($pid != 0)                                                           {                                                                         if($msg =~ / (\w+)\s+(\d\d:\d\d)\s+\[(\w+)\s+\] (.*)/)                      {                                                                           $account     = $1;                        $start_time  = $2;                        $status      = $3;                        $event_time    = integer(now())/1000;                        delete($msg);                                                           to_json();                                                          }                                                                     }                                                           </Exec> </Processor> <Output out>     Module om_file                                                          File 'D:\winapp\log\pchk_test.txt'        </Output> <Route 1>     Path        test => pro => out </Route>     Thank you for help