I am using nxlog-ce clients in windows machines, with om_ssl.  NXlog seems to keep the connection open, but does not send anything, except for the bursts for every Poll Interval.  The load balancer seems confused, and tries to send every second burst to a different server.  Is there any option to tell nxlog-ce that it should make a new connection for each burst, so that the load balancer can easily balance the load for different clients to different log servers?

Hello,   I have 2 infrastructures installed one on Amazon and another on premises. On Amazon every thing is OK, but when I send logs on premises I have a 5 to 7 minutes delayed and I loss almost 30% of the data, I activated the buffer option and I pass through it before sending logs to my on premise server but it doesn't work Somebody know how to correct that?   Thank you for your help and answer   Ana

We are pushing logs from file with the im_file module to logstash and then to elasticsearch. However, some of these logs only have second accuracy, and therefore not returned in order by elasticsearch when sorted by just the time. To get around this problem, we would like to add the position of the log event to a field that we store (for example beginning line number or byte offset within the source file). Is there a way I can add a separate field for this in the events pushed by nxlog, either in the input or output configuration?

I have the following regexp: ^\s*(\d{2}:\d{2}:\d{2}\.\d{4})\s+(\[\S+\])\s+([\s\S]*)$ and a line from the log that I am trying to parse (there are some spaces at the beginning of the line):  13:33:00.1205 [-] Persisting VDOM path / trying to process a file using pm_pattern module with LogLevel DEBUG this is what I get: 2016-11-16 13:38:00 DEBUG regexp '^\s*(\d{2}:\d{2}:\d{2}\.\d{4})\s+(\[S+\])s+([\s\S]*)$' did not match against '  13:38:00.1857 [-] Persisting VDOM path /' Part of the xml file used to process the input: <pattern>     <id>1</id>     <name>log_line</name>     <description>Regular log line</description>     <matchfield>         <name>raw_event</name>         <type>regexp</type>         <value>^\s*(\d{2}:\d{2}:\d{2}\.\d{4})\s+(\[S+\])s+([\s\S]*)$</value>                                 <capturedfield>             <name>EventTime</name>             <type>string</type>         </capturedfield>         <capturedfield>             <name>SourceName</name>             <type>string</type>         </capturedfield>         <capturedfield>             <name>Message</name>             <type>string</type>         </capturedfield>     </matchfield> </pattern> What is wrong with my expression?   

I am wanting to see if this is possiable to put in a line for Input event log.     <Input eventlog>    Module im_msvistalog    SavePos FALSE    ReadFromLast TRUE    Query  <QueryList>\             <Query Id="0">\              <Select Path="System">*[System[(EventID=22 or EventID=1076 or EventID=6005 or EventID=6006)]] and *[System/Level=2]</Select>\              </Query>\ </QueryList> </Input>     Can i have this line in the select path along with just detecting event logs? I also want to monitor all Error messages.  *[System/Level=2]

Hello.  The case is: I am creating Stream with rules using API. Question is: Is there a way to set the input for a stream using GrayLog API?   Thanks.

Hi, I want to store my logs in .evtx file in windows. I tried following configuration. <Output out2>         Module      om_file   File     '%ROOT%\tmp\test.evtx' </Output> This created evtx file but it was also opening with notepad, wordpad,etc. For security purpose, I want to make it open with MS EventViewer API only. Is this possible using nxlog om_file module? Is there any plugin for nxlog to store data in .evtx files?

Hi, I am a new at this and would like to know the command line argument that will let me specify what config file to use. I need to have this done during installation and avoid changing the file post installation. Any help would be greatly appreciated.   Thanks.

Hi,   I am totally new to nxlog and trying to set up a config file to watch a file on my local hard drive and send it to Loggly whenever it changes.  My conf file is below.  I copied this file from the loggly documentation and tried modifiying it as per their instructions so there may be some unnecessary lines in it. The file I want to watch and send is in the C:\\Program Files\\Autodesk\\Vault Professional 2016\\Explorer\\ folder.  The file name is JobProcessor.log.  When I uncomment the "#Exec file_write("C:\\Program Files\\Autodesk\\Vault Professional 2016\\Explorer\\JobProcessor.log",  $raw_event);" line, it does send it out so I know the connection is working.  But I have not been able to 'trigger' the file to process whenever there is a change to it.  Any advice is appreciated.  Thanks.   #define ROOT_STRING C:\\Program Files\\nxlog define ROOT C:\\Program Files (x86)\\nxlog define ROOT_STRING C:\\Program Files (x86)\\nxlog define CERTDIR %ROOT%\\cert   Moduledir %ROOT%\\modules CacheDir %ROOT%\\data Pidfile %ROOT%\\data\\nxlog.pid SpoolDir %ROOT%\\data LogFile %ROOT%\\data\\nxlog.log   # Include fileop while debugging, also enable in the output module below <Extension fileop>     Module      xm_fileop </Extension>   <Extension json>     Module      xm_json </Extension>   <Extension syslog>     Module xm_syslog </Extension>   <Input internal>     Module im_internal     Exec  $Message = to_json();  </Input> ##Module to watch a file <Input file_watch_1>     Module im_file     File "C:\\Program Files\\Autodesk\\Vault Professional 2016\\Explorer\\"     Exec   $SourceName = 'JobProcessor.log';      SavePos TRUE     ##include the message and add meta data     Exec $Message = $raw_event;     Exec to_syslog_ietf(); </Input>   <Output out>     Module om_tcp     Host logs-01.loggly.com     Port 514       Exec to_syslog_ietf();     Exec $raw_event; #Use the following line for debugging (uncomment the fileop extension above as well)     #Exec file_write("C:\\Program Files\\Autodesk\\Vault Professional 2016\\Explorer\\JobProcessor.log",  $raw_event); </Output>   <Route 1>     #Path internal, eventlog => out     Path    in => out </Route>

Hi! I have a problem using module im_odbc. I use sql query to receive some data from sql server. For example, query looks like "SELECT [Name] From [MyTable] WHERE [Id] > ?". In some moment, the "?" has stored incorrect value (I don't know how, but it's bigger then MAX(Id) value in my table) and sinse that moment, query has not return any data.  Please, help me reset value of "?" (nxlog service restart not help). And, maybe, you can give me some recommendations, how to avoid this problem in future?   UPD: I reset value of "?" by renaming input name in config file. The first query after input renaming was executed with "0" as value of "?" and return all rows from table. But next time query executed with "?"-value "161027120023", wich bigger than max value of "Id" column in my table (max value was 161027115813 at that time). What am i doing wrong? <Input odbc1>     Module                im_odbc     ConnectionString    DSN=im_odbc_test;database=MyDB;     SQL                    SELECT [Name] From [MyTable] WHERE [Id] > ?;     PollInterval        120     SavePos                TRUE     IdIsTimestamp        FALSE </Input>

Hello, This is jsut a quick question on the following lines of a config for Nxlog. Basically is the code suppressing all security events with the exception of those event IDs with in the brackets or are the event IDs within the brackets being suppress? # Windows Event Log <Input eventlog> # Uncomment im_msvistalog for Windows Vista/2008 and later    Module im_msvistalog Query    <QueryList>\                    <Query Id="0">\                      <Select Path="Security">*</Select>\                      <Suppress Path="Security">*[System[(EventID=4624 or EventID=4776 or EventID=4634 or EventID=4672 or EventID=4688 or EventID=4662 or EventID=5159) or EventID=5136 or EventID=4932 or EventID=4933]]</Suppress>\                    </Query>\         </QueryList>    Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Input>

Does it work to send IIS logs in snare format ??? when i use the "Exec to_syslog_snare();" option in output, it sends logs like they are MSWinEvents...   <13>Oct 21 06:26:36 SRV-00-20-21 MSWinEventLog    1    N/A    17    Fri Oct 21 06:26:36 2016    N/A    N/A    N/A    N/A    N/A    N/A    N/A        N/A    N/A

I was reading the Enterprise Edition documentation on Check Point OPSEC LEA. Can this be done on a Windows server or do I need to use a Linux server?

I have    <Extension w3c>     Module            xm_csv     Fields            $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $time-taken     FieldTypes        string, string, string, string, string, string, integer, string, string, string, string, string, integer, integer, integer, integer     Delimiter        ' '     QuoteChar        '"'     EscapeControl    FALSE     UndefValue        - </Extension>   And i have    <Input iis>     Module            im_file     File            "C:\\inetpub\\logs\\LogFiles\\u_ex*.log"     SavePos            TRUE     Exec \                     if $raw_event =~ /^#/ drop(); \                     else \                         { \                             w3c->parse_csv(); \                             $EventTime = parsedate($date + " " + $time); \                             $EventTime = strftime($EventTime, "%Y-%m-%dT%H:%M:%SZ"); \                             $SourceName = "IIS"; \                         } </Input>   When im trying to add something like if $raw_event =~ /^#/ or $sc-status == 101 drop(); \ I am getting error in log like that:  couldn't parse statement at line 91, character 46 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; syntax error, unexpected -, expecting ( I have tried to use similar options, but it doesnt works.  What im doing wrong, help me please. Thanks.  

I'm attempting to use nxlog to pull in audit files from ModSecurity the root of the log files reside at /var/log/modsec_audit/. The actual log files are two folders lower, as modsecurity creates the folders that are timestamped by day and then minute. Inside my nxlog input config, I have my logfile path as /var/log/modsec_audit/*, and I have enabled recursive file lookup, but to this point, no modsecurity logs are being picked up. The same sidecar instance is grabbing apache logs perfectly fine, however. I feel like the issue is related to the directory structure, but I'm not seeing an issue with my config. Any thoughts?   <Input 579d7bd625b5bc05cc7f1614>         Module im_file         File "/var/log/modsec_audit/*"         PollInterval 5         SavePos True         ReadFromLast True         Recursive True         RenameCheck False         Exec $FileName = file_name(); # Send file name with each message </Input>  

All, I'm somewhat new to Netwrix, and I need some help understanding how I can send ONLY specific logs up to our SEIM.  I've dug through the documentation a bit, and will continue to do so.  I'm sure someone on here can whip up a quick response... Here is my config file, please help me with mocking up a config that will only send up events 4648 and 4624... ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog>     Module xm_syslog </Extension> <Input internal>     Module im_internal </Input> <Input eventlog>     Module im_msvistalog </Input> <Output out>     Module      om_udp     Host        xx.yy.zz.xyz     Port        514     Exec        to_syslog_snare(); </Output> <Route 1>     Path        eventlog, internal => out </Route>  

What is the best way to do a log rotation? I'm using fileop and I wanted log rotation at midnight, but it hasn't been working. Would it be better to move the Schedule to the Output section? <Extension fileop>     Module      xm_fileop     # Check the size of our log file every hour and rotate if it is larger than 1Mb     <Schedule>         Every   1 hour         Exec    if (file_size('%LOGFILE%') >= 1M) file_cycle('%LOGFILE%', 500);     </Schedule>     # Rotate our log file every week on sunday at midnight     <Schedule>         When    @midnight         Exec    file_cycle('%LOGFILE%', 500);     </Schedule> </Extension>

Two related threads I found on this https://nxlog.co/question/2029/logon-vs-logoff-events-seim https://nxlog.co/question/1711/configuration-send-windows-security-logs-only Posted a question the Windows forums as well https://social.technet.microsoft.com/Forums/windowsserver/en-US/f70aa0fd-9f59-470a-bafe-47c82f4a7151/broken-xml-for-eventid-4624?forum=winserver8gen There are domain controllers that a logging server connects to using Windows Eventlog Forwarding (poll). In the eventlog eventid 4624 comes in but in the message field its all % placeholders but the XML data is correct. Before some patching the xml data was broken. The Data fields looked like this <Data Name="AuthenticationPackageName">Kerberos</Data> <Data Name="WorkstationName"> <Data Name="LogonGuid">{B3A61084-7036-1568-AFB2-3290B7F943F9}</Data> </Data> <Data Name="TransmittedServices">-</Data> <Data Name="LmPackageName">-</Data> But now are more correct and look like this <Data Name="AuthenticationPackageName">Kerberos</Data> <Data Name="WorkstationName" /> <Data Name="LogonGuid">{B3A61084-7036-1568-AFB2-3290B7F943F9}</Data> <Data Name="TransmittedServices">-</Data> <Data Name="LmPackageName">-</Data> The issue is only when nothing is in the WorktationName field. NXlog reads this and converts to JSON and thats where NXlog is obviosly having problems The JSON looks like this "AuthenticationPackageName":"Kerberos","WorkstationName' /> <Data Name='LogonGuid":"{B3A6..... So at the workstation name it fails to parse the XML, possibly because the XML field is closed with /> instead of a full </Data> Another issue is that the Message field does not come through NXlog at all. Would love to get some help debugging this.

All, I'm struggling with NXLOG forwarding events to our SEIM.  I'm able to see Logoff (4647) events, but not Logon (4624) events.  Under the WIndows Event Log, we see both events occuring as expected, but our SEIM is only getting Logoffs... Our config file is standard, but here it is below... ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/docs/ ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog>     Module xm_syslog </Extension> <Input internal>     Module im_internal </Input> <Input eventlog>     Module im_msvistalog </Input> <Output out>     Module      om_udp     Host        xx.yy.zz.xyz     Port        514     Exec        to_syslog_snare(); </Output> <Route 1>     Path        eventlog, internal => out </Route>

I am unable to replace \r\n and \t when using to_json. When using to_syslog_bsd I can replace them, but not with JSON. Do I have an error in my config or is this a bug? Please set the ROOT to the folder your nxlog was installed into, otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog> Module xm_syslog </Extension> <Extension json> Module xm_json </Extension> <Input internal> Module im_internal </Input> <Input eventlog> Module im_msvistalog For windows 2003 and earlier use the following: Module im_mseventlog </Input> <Input mark> Module im_mark MarkInterval 60 Mark 226d61792074686520666f726365206265207769746820796f7522 </Input> <Output out> Module om_udp Host logspot.server.org Port 514 Exec $raw_event = replace($raw_event, "\r\n", " "); Exec $raw_event = replace($raw_event, "\t", " "); Exec to_json(); </Output> <Route 1> Path eventlog, internal, mark => out </Route>