This is a very interesting issue and I was wondering if anyone has encountered it before. I have ~200 development systems that I wish to gather windows event information from however, there are toolsets within this environment that are interrupted by NXLog. NXLog is connecting to eventlog API and polls (pause/resume) the API every second (configurable) for new events within the various subscribed to channels. My issue is with reference information loaded by the event log API due to the active connection from NXLog. The reference dll is loaded into the eventlog service but is not released because NXLog does not release the call, it pauses and resumes.  This in turn causes our development automation to fail since the automation tools are looking to replace this dll once a dev pass is complete. Is there an exec example to reconnect so that the eventlog service will release these dlls?  Something like: <Input eventlog>    Module im_msvistalog    Query\     <QueryList>\      <Query Id="0">\       <Select Path="Application">*</Select>\      </Query>\    </QueryList>    Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;    Exec $raw_event='{"jsonEvent":"Windows","Event":' + to_json() + '}';     <Schedule>          Every 30 sec          Exec  eventlog->reopen();      </Schedule> </Input>  

How can nxlog for windows be configured to send timestamp in high-precision forwarding format? Similar to the RSYSLOG_ForwardFormat high-precision timestamps including year with timezone UTC offset information. For example: 2017-08-22T18:36:28.568230+00:00   It is currently just being sent with ONLY Mon day hour:min:sec  <14>Aug 22 15:16:46 Win7Prox64 MSWinEventLog 1 System 2 Tue Aug 22 15:16:46 2017 7036 Service Control Manager N/A N/A Information Win7Prox64 N/A The nxlog service entered the running state. 170362 Thanks! p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; background-color: #fef49c} span.s1 {font-variant-ligatures: no-common-ligatures}

Hello, We are using nxlog to write all our event logs to syslog, but have a need for them to be formatted as CSV instead of the tab delimited it appears to be currently. Is anyone doing this currently and mind sharing their config, or know if a way to process this correctly? Thanks!

Hello. I like to collect Windows Event ID and send it to Linux Box. I installed "Syslog-NG" on Linux and it collect Windows Event Log very good but I have a problem about "Exec convert_fields("AUTO", "utf-8");". When I enable it in my config file then my "nxlog" can't send log!!! My "nxlog" config is:   ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>     Module      xm_syslog     AutodetectCharsets utf-8 </Extension> <Input in>     Module      im_msvistalog # For windows 2003 and earlier use the following: #   Module      im_mseventlog     Exec if $EventID NOT IN (4660, 4663) drop();      Exec convert_fields("AUTO", "utf-8"); </Input> <Output out>     Module      om_udp     Host        172.30.9.20     Port        514     Exec        to_syslog_snare();  </Output> <Route 1>     Path        in => out </Route>   Any idea?   Thank you.

I have an input for some log files, however each restart of the machine the log file is rotated to a new file. I'm trying to create an input that is able to track the rotated log file. I presume a wildcard can be used in the File: string and that the save position and read from last wouldnt cause NXlog to re read old files? Below you can see the example file path with the 'X' representing a number that changes each time the active log is rotated. C:\ProgramData\VMware\hostd\hostd-X.log   Regards, G

Has anyone gotten nxlog running on linux to receive windows logs through Windows Log Collector initiated by a GPO? In simple terms, using nxlog to receive Windows logs without a nxlog agent running on Windows? If so, any documentation on how to make that happen? Thank you in advance, Bruce M. Wink

Hello I have noticed the on the download page of NXLog Community Edition the versions of the Windows and Linux packages are 2.9 while the source code (.tar.gz) is of the lower version 2.8. Could the source code package please be updated? We are missing the GELF_TCP feature which appeared in 2.9.1347. Thank you Michael

Hi all, I can...  - rotate log files based on size.  - rotate log files based on size using event fields- such as $Hostname from Syslog  - rotate log files based on size and schedule. I CANNOT rotate log files based on size and schedule using event fields !  It seems that the Schedule component does not like references to fields. At end of tether, please help.   ERROR invalid om_file keyword: $newfilepath   <Extension syslog>     Module xm_syslog </Extension>   <Input in>     Module im_udp     Host 0.0.0.0     Port 514     Exec parse_syslog(); </Input>   <Output out>     Module om_file $newfilepath = "D:/Test/" + $Hostname + "/" + $SourceName + "/" $currfilename = strftime(now(), "%Y%m%d") + ".current" File $newfilepath + $currfilename   # Check the size of our log file every minute and rotate if it is larger than 1Mb <Schedule> Every 60 sec Exec if (out->file_size() >= 1M) \ { \ $newfilename = $newfilepath + strftime(now(), "%Y%m%dT%H%M%S") + ".s"); \ out->rotate_to($newfilename); \ } </Schedule>   CreateDir   TRUE </Output>   <Route 1>     Path in => out </Route>   Thanks in advance. Paul  

Back to conversation about current workaround... Windows Server 2016 has more than 256 channels. Is it possible to create a second thread/instance to subscribe for the remaining channels? I can try to guess and create XML filter to exclude some unneeded for now but tomorrow MS can create more channels with some update and would be nice if it handled automatically.

I am trying to use the om_http module to POST events to an AWS API Gateway (which won't allow SSLv3 connections). Looking at this forum and the documentation, it seems like the latest version of the community edition, nxlog-ce-2.9.1716, should support TLSv1.2, but I keep getting the following error when it attempts to connect to my API: ERROR SSL error, SSL_ERROR_SSL: retval -1, sslv3 alert handshake failure Below is my config for the output: <Output out>     Module       om_http     URL         https://<my api endpoint>     ContentType "application/json" </Output>   I have also tried using HTTPSAllowUntrusted TRUE, which doesn't change anything. Also, I tested the API endpoint with curl and it works fine.   Thanks in advance for any help!   UPDATE: It looks like nxlog is negotiating using TLSv1.2 (discovered with wireshark). I also discovered that AWS API Gateway requires the SNI extension to TLS. This is likely the problem if nxlog-ce doesn't support SNI. Any idea if/when that will be supported in the community edition?

Is the Output module to ElasticSearch available/will be available in the community edition? When I last checked it was a feature of the commerical edition only.  I have been a nxlog champion for years now and have been forwarding to logstash. However with Elasticsearch ingest nodes, there is one extra redundent step now. This will tip the scale to moving off into native beats/rsyslog if there are no plans to make this available.   Thanks   Ash Kumar  

Hi there, this might be a strange question but I'm new nxlog and was wondering what the difference between an input module and a processor module is. the background to my question is: I want to collect logs centrally on an nxlog server, and have configured all my clients to send their logs in a nxlog-binary format. at the moment I'm testing this with Windows IIS webserver logs. on the nxlog server i want to read the binary logs and process them. Based on the processor I'd like to forward them to two separate destinations, outputting the logs in GELF and CSV format. Now my question: While this codeblock in the Input module works fine: <Input in_syslog_tcp>   Module  im_tcp   Host  0.0.0.0   Port  80     <Exec>         $Hostname = hostname();         w3c->parse_csv();         $EventTime = parsedate($date + " " + $time);         $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;         $SourceName = "IIS";          w3c->to_csv();   </Exec>   InputType Binary </Input> This codeblock doesn't work: <Processor transform_iis>   Module      pm_null    <Exec>         $Hostname = hostname();         w3c->parse_csv();         $EventTime = parsedate($date + " " + $time);         $raw_event = $Hostname + ' IIS-NXLOG  '  + $raw_event;         $SourceName = "IIS";         w3c->to_csv();   </Exec> </Processor> Is there any reason why this has to go into the input module? The disadvantage would be that I would have to have several input modules for all kinds of different data sources....currently I have only one input module that receives all the logs in binary format from lots of different hosts..   Is there maybe a better solution for this?   cheers, micsnare

Hi, My scenario is:  (Windows server + nxlog configured for Windows events) => Logstash => Elasticsearch I am wondering where nxlog stores current informations about sent Windows Events (for every category). If i need to resend a few Windows events from past, how i can do it? Is there any way to select last X hours (example: last 48 hours from Security category, from the starting nxlog service moment). SavePos and ReadFromLast are helpful, but if both are false, event logs are sent from the first one stored on Windows server. Also, where nxlog save all events in case of temporary lost tcp connection? This is probably SavePos location.   Thanks  

Hi,   I've to use the same exec block multiple times in different paths. I've already tried it with a processor module, but unfortunately a processor module could only be used once. A possible solution is to implement them in the input modules, but I've more than one input module (syslog tcp/udp alone are already two) which ends in multiple code in different modules. But I want them once and want to use them multiple times. Have someone an idea how to implement this? Thanks in advance.   br    

Hi, My Server's TimeZone is EST(UTC-5) but the application running in the server has EET(UTC+2). The log file generated is names as <Application Date> ie todays log file is named as 20170717.log. Today's log will be truncated at 00:00 (UTC+2) and new log file 20170718.log will be created and old log file will remain in the same directory. How can I configure NxLog to read the current log only

hi all! im first time meet nxlog. it looks very simple and powerfull, but i need help! i have cisco which can send syslog to my nxlog service. logs looks like 2017-06-21T07:19:49.214Z,"vpn.domain.com","Jun 21 2017 10:19:55: %ASA-4-722051: Group <GroupPolicy_domain> User <123456@domain.com> IP <213.87.163.123> IPv4 Address <10.217.130.23> IPv6 address <fc00::59> assigned to session" please, help me write processor, which parse this string and get date (Jun 21 2017 10:19:55)? login (123456@domain.com) and external ip adress (213.87.163.123) and put them to text log file. thanks!

Hello, I need to parse logs mounted on file system NFS on centos. Is NXLog reliable with NFS ? Thanks, Regards

I hava some data  like this  -------------------------- a         9650     mktd     07:30     [OK    ] a    b         9650     mktd     07:30     [OK    ] b    c         9650     mktd      07:30     [OK    ] c     d         9650     mktd     07:30     [OK    ] d if data have change to this a        9650     mktd     07:30    [OK    ] a b        9650     mktd     07:30    [OK    ] b c        0                                     [FAILED] c d        0                                     [FAILED] d   how could I reload the file to nxlog I have been to tried to use PollInterval ,but it seem to use for new file and new log entries I also try to use file_touch,but both of them doesn't useful for me.   my scripts below   #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json>     Module      xm_json </Extension> <Extension fileop>     Module      xm_fileop </Extension> <Extension exec>     Module      xm_exec </Extension> <Extension kvp>     Module       xm_kvp     KVPDelimiter ,     KVDelimiter  =     EscapeChar   \\ </Extension> <Input test>        Module       im_file      File         'D:\winapp\log\chk2.txt'      SavePos      FALSE      ReadFromLast FALSE </Input>     <Processor pro>     Module       pm_null          Exec         delete($SourceModuleName);                            \                  delete($SourceModuleType);                            \                  delete($EventReceivedTime);                                                                    <Exec>                          if ($raw_event =~ /^(\w+)\s+(\d+)(.*)/)                   {                                                                           $app_name      = $1;                                                    $pid           = integer($2);                                           $msg           = $3;                       to_json();                                                          }                                                                       else                                                                    {                                                                         drop();                                                               }                                                                       if ($pid == 0)                                                          {                                                                          if($msg =~ / \s+\[(\w+)\] (.*)/)                                        {                                                                          $status      = $1;                                                      $event_time    = integer(now())/1000;                                   delete($msg);                                                           to_json();                                                           }                  }                  if($pid != 0)                                                           {                                                                         if($msg =~ / (\w+)\s+(\d\d:\d\d)\s+\[(\w+)\s+\] (.*)/)                      {                                                                           $account     = $1;                        $start_time  = $2;                        $status      = $3;                        $event_time    = integer(now())/1000;                        delete($msg);                                                           to_json();                                                          }                                                                     }                                                           </Exec> </Processor> <Output out>     Module om_file                                                          File 'D:\winapp\log\pchk_test.txt'        </Output> <Route 1>     Path        test => pro => out </Route>     Thank you for help

I have to retain the new lines in a syslog.  I'm using NXLog to send logs from my laptop to a test syslog server.  I'm currently using the following: <Extension _syslog> Module xm_syslog </Extension> <Input in> Module im_msvistalog Exec to_syslog_ietf(); </Input> <Processor rewrite> Module pm_null Exec $Message = $EventID + "|" + $EventType + "|" + $Hostname + "|" + $SourceName + "|" + $AccountName + "|" + $AccountType + "|" + $Domain + "|" + $UserID + "|" + $raw_event; </Processor> <Output out> Module om_udp Host 192.168.100.33 Port 514 #Exec to_syslog_bsd(); </Output> <Route 1> Path in => rewrite => out </Route> I can get the logs to send with the \r\n intact is to remove the Exec to_syslog_snare(), then I loose all the other details about the log such as event id etc.  So I thought ok I'll construct my own by using Exec $Message = all the data fields I want....this doesn't work...so then I started playing with to_syslog_ieft and to_syslog_bsd() and they both strip out new lines. What am I doing wrong?

Hi, I need some help with nxlog.conf (Client side + Server side). I need to send JSON file over syslog udp connection from a client to server and rebild JSON in server side. I know that with nxlog it's posible to perform this but i don't know the correct configuration. can you help me with this issue ?