Where can I find the documenation for this om_redis module? Only resource I can find is: https://nxlog.co/question/1593/redis-module-lpush  

Hi All, I have previously got smtp logs to go into Graylog using NXlog, it was worknig fine. I then had a disk sapce issue on the graylog host so had to redo some bits, including the nxlog.conf for our SMTP server.  The SMTP log header specifes the following #Software: Microsoft Internet Information Services 8.5 #Version: 1.0 #Date: 2017-03-26 23:00:10 #Fields: date time c-ip cs-username s-sitename s-computername s-port cs-method cs-uri-query sc-bytes time-taken  This is the error I am getting ERROR if-else failed at line 44, character 436 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 224 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 11, got 8 in input I ahve checked and rechecked and their should be 11 items as per the .conf. define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf>     Module         xm_gelf </Extension> <Extension fileop>     Module         xm_fileop </Extension>  <Extension json>     Module      xm_json </Extension> # Create the parse rule for IIS logs. You can copy these from the header of the IIS log file. <Extension w3c>     Module             xm_csv     Fields             $date, $time, $c-ip, $cs-username, $s-sitename, $s-computername, $s-port, $cs-method, $cs-uri-query, $sc-bytes, $time-taken     FieldTypes         string, string, string, string, string, string, integer, string, string, integer, integer     Delimiter         ' '     QuoteChar         '"'     EscapeControl     FALSE     UndefValue         '-' </Extension> <Input smtp>     Module        im_file     File        "C:\\Logs\\SMTPSVC1\\\ex*.log"     SavePos      TRUE     Exec        if $raw_event =~ /^#/ drop();                    \                 else                                             \                 {                                                \                     w3c->parse_csv();                            \                     $EventTime = parsedate($date + " " + $time); \                     $SourceName = "smtp";                         \                     $Message = to_json();                         \                 } </Input> <Input eventlog>     Module      im_msvistalog </Input> <Output graylog>     Module      om_udp     Host        graylog.mydomain.com     Port        12201     OutputType    GELF     #Use the following line for debugging (uncomment the fileop extension above as well)     #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event); </Output> <Route eventlog>    Path        eventlog => graylog </Route> <Route smtp-to-graylog>     Path        smtp => graylog </Route> Its so frustracting that I know this was working correctly. Any help would be great. Thanks

I'm struggling with rotating syslog files at midnight so that they are named "YYYY-MM-DD.log" (and contain log records for that date). I wonder if anyone has an example of the best way to achieve this.  The manual could use an example like this. Thanks Phil    

I'm attempting to use NXlog to perform a one time read of a CSV file which will then be passed to Fluent-D for processing and writing to a MySQL database. I've read through the documentation for setting up the nxlog.config file but I'm at a loss to how to configure for my CSV file. Has anyone has a config file that reads a CSV I'd apreciate the help. Thanks. 

Hello. While reading log from windows, I got string with two encodings WIN-1251 and UTF-8: like 2017-02-21 16:40:24 IT-73.domain.name INFO 44 NT AUTHORITY\҈Ғƌ�Центр обновления Windows начал скачивать обновление. where all message in utf-8, and part with AccountName that (usually NT AUTHORITY\SYSTEM in english version of Windows 7) but in russian version its NT AUTHORITY\СИСТЕМА and in logs it looks like NT AUTHORITY\҈Ғƌ� If I use  convert_fields("AUTO", "utf-8") I got proper AccountName, but all other parts got wrong encoding. Is where any way to replace that part with correct encoding?  

Hello, nxlog CE v2.9.1504, Windows Server 2008 Enterprise relevant part of config file: <Input eventlog> # Uncomment im_msvistalog for Windows Vista/2008 and later Module im_msvistalog Exec if ($Severity == 'INFO') drop(); <QueryXML> <QueryList> <Query Id="0"> <Select Path="System">*</Select> <Select Path="Security">*</Select> <Select Path="Application">*</Select> <!-- EventID 2137 - Shrepoint Health Analyzer - ignore --> <Suppress Path="Application">*[System[(EventID=2137)]]</Suppress> <!-- EventID 2138 - Shrepoint Health Analyzer - ignore --> <Suppress Path="Application">*[System[(EventID=2138)]]</Suppress> <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select> </Query> </QueryList> </QueryXML> # Uncomment im_mseventlog for Windows XP/2000/2003 # Module im_mseventlog </Input> One event (EventID 1309 from Application channel) always returns an empty message field (message:null)  as you can see in debug output. {"EventTime":"2017-03-21 09:54:13","Hostname":"HOST.DOMAIN.TLD","Keywords":36028797018963968, "EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1309,"SourceName":"ASP.NET 2.0.50727.0", "Task":3,"RecordNumber":1013344,"ProcessID":0,"ThreadID":0,"Channel":"Application","ERROR_EVT_UNRESOLVED":true, "Category":"Web Event","EventReceivedTime":"2017-03-21 09:54:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog", "message":null} How can I get more informations, why those events has a null message field? What means "ERROR_EVT_UNRESOLVED":true? Thank you.

hello: I find some problem under AIX6.1 system.   the log: 2017-03-21 14:02:41 WARNING additional group memberships couldn't be set because getgrouplist()and setgroups() are not available on this platform 2017-03-21 14:02:41 INFO nxlog-ce-2.8.1248 started 2017-03-21 14:02:41 ERROR failed to open /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied 2017-03-21 14:02:43 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied 2017-03-21 14:02:49 ERROR last message repeated 2 times 2017-03-21 14:02:57 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied 2017-03-21 14:03:13 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied 2017-03-21 14:03:45 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied 2017-03-21 14:04:50 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied and i sccan the source code, find: static void nxlog_set_groups(nxlog_t *nxlog, apr_uid_t uid) {     gid_t grplist[100];     int ngroups = sizeof(grplist);     char *user;     apr_status_t rv;     if ( (rv = apr_uid_name_get(&user, uid, nxlog->pool)) != APR_SUCCESS )     {     log_aprerror(rv, "couldn't resolve uid %d to name", uid);     return;     } #ifdef HAVE_GETGROUPLIST  # ifdef HAVE_SETGROUPS     if ( getgrouplist(user, getgid(), grplist, &ngroups) == -1 )     {     log_error("couldn't get group membership for user %s (uid: %d), too many groups?", user, uid);     return;     }     if ( setgroups((size_t) ngroups, grplist) != 0 )     {     log_errno("couldn't get group membership for user %s (uid: %d), setgroups() failed",           user, uid);     return;     } # else     log_warn("additional group memberships couldn't be set because getgrouplist()"          "and setgroups() are not available on this platform"); # endif #else     log_warn("additional group memberships couldn't be set because getgrouplist()"          "and setgroups() are not available on this platform"); #endif } I don't what' wrong with this problem, thanks.  

Hi All, Checking to see if anyone has run into this.  I have a windows eventlog collector, with a subscription setup to move specific security audit events to the "Forwarded Events" log.  From there, I am looking to push those logs to Sumologic.  Unfortunately Sumo's collector does not handle this well due to the out of sequence EventRecordID of the various events coming from multiple desktops/servers we're collecting from. Question: I'm trying to take advantage of Sumo's native Windows eventlog parser, however the options for sending the eventlog data using NXlog send in the specific formats, syslog_snare, xml, json, etc.  Is there a configuration i can use which send the messages as windows eventlog format?  You'll see from my config below, I've tried several formats, to no avail.  Any suggestions would be greatly appreciated. <Input eventlog>     Module      im_msvistalog <QueryXML>    <QueryList>                          <Query Id="0">          <Select Path="ForwardedEvents">*</Select>      </Query>    </QueryList> </QueryXML> </Input> <Output out>     Module      om_tcp     Host        10.x.x.x     Port        514 #    Exec       to_xml();        Exec to_syslog_snare(); #    Exec $raw_event = replace($raw_event, "\r\n"," "); #    Exec $raw_event = replace($raw_event, "\t", " "); #    Exec    $raw_event(); </Output> <Route 1> # Path in => out  Path eventlog, internal => out </Route> Thanks in advance, -A

Hello All,   I am new to nxlog or rather logics in terms of regex and all.  I am looking to drop any message which has *.*.*.255 in message field. I tried below , however it does not seem to be working for me:   Exec if $Message =~ /^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . ^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . ^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . 255/  drop();   Please suggest

config file: ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally under ## /usr/share/doc/nxlog-ce/ and is also available online at ## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ######################################## # Global directives                    # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules                              # ######################################## <Extension _syslog>     Module      xm_syslog </Extension> <Extension multiline>  Module xm_multiline  HeaderLine /^\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4}/ </Extension> <Input oerrorin1>  Module    im_file  File     '/u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log'  InputType multiline  SavePos TRUE  ReadFromLast  FALSE </Input> <Output udpout>  Module    om_udp  Host   10.1.227.45  Port   514 </Output>s ######################################## # Routes                               # ######################################## <Route 1>     Path        oerrorin1 =>udpout </Route>   then i get the log: 2017-03-17 16:16:39 INFO nxlog-ce-2.8.1248 started 2017-03-17 16:16:39 ERROR failed to open /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:16:41 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:16:47 ERROR last message repeated 2 times 2017-03-17 16:16:55 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:17:11 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:17:43 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:18:47 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:20:55 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:25:11 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied 2017-03-17 16:34:16 DEBUG reading config cache from /var/spool/nxlog/configcache.dat 2017-03-17 16:34:16 DEBUG Setting up module '_syslog' using xm_syslog 2017-03-17 16:34:16 DEBUG module _syslog has 4 exported functions 2017-03-17 16:34:16 DEBUG registering function syslog_facility_value 2017-03-17 16:34:16 DEBUG function 'syslog_facility_value' registered 2017-03-17 16:34:16 DEBUG registering function syslog_facility_string 2017-03-17 16:34:16 DEBUG function 'syslog_facility_string' registered 2017-03-17 16:34:16 DEBUG registering function syslog_severity_value 2017-03-17 16:34:16 DEBUG function 'syslog_severity_value' registered 2017-03-17 16:34:16 DEBUG registering function syslog_severity_string 2017-03-17 16:34:16 DEBUG function 'syslog_severity_string' registered 2017-03-17 16:34:16 DEBUG module _syslog has 9 exported procedures 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog' registered 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog' registered 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_bsd 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_bsd' registered 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_bsd 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_bsd' registered 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_ietf 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_ietf' registered 2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_ietf 2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_ietf' registered 2017-03-17 16:34:16 DEBUG registering procedure to_syslog_bsd 2017-03-17 16:34:16 DEBUG procedure 'to_syslog_bsd' registered 2017-03-17 16:34:16 DEBUG registering procedure to_syslog_ietf 2017-03-17 16:34:16 DEBUG procedure 'to_syslog_ietf' registered 2017-03-17 16:34:16 DEBUG registering procedure to_syslog_snare 2017-03-17 16:34:16 DEBUG procedure 'to_syslog_snare' registered 2017-03-17 16:34:16 DEBUG Setting up module 'multiline' using xm_multiline 2017-03-17 16:34:16 DEBUG Setting up module 'oerrorin1' using im_file 2017-03-17 16:34:16 DEBUG module oerrorin1 has 1 exported functions 2017-03-17 16:34:16 DEBUG registering function file_name 2017-03-17 16:34:16 DEBUG function 'file_name' registered 2017-03-17 16:34:16 DEBUG module oerrorin1 has 0 exported procedures 2017-03-17 16:34:16 DEBUG FlowControl enabled for oerrorin1 2017-03-17 16:34:16 DEBUG Setting up module 'udpout' using om_udp 2017-03-17 16:34:16 DEBUG Setting up module 'fileout2' using om_file 2017-03-17 16:34:16 DEBUG module fileout2 has 2 exported functions 2017-03-17 16:34:16 DEBUG registering function file_name 2017-03-17 16:34:16 DEBUG function 'file_name' registered 2017-03-17 16:34:16 DEBUG registering function file_size 2017-03-17 16:34:16 DEBUG function 'file_size' registered 2017-03-17 16:34:16 DEBUG module fileout2 has 2 exported procedures 2017-03-17 16:34:16 DEBUG registering procedure rotate_to 2017-03-17 16:34:16 DEBUG procedure 'rotate_to' registered 2017-03-17 16:34:16 DEBUG registering procedure reopen 2017-03-17 16:34:16 DEBUG procedure 'reopen' registered 2017-03-17 16:34:16 DEBUG CONFIG: _syslog 2017-03-17 16:34:16 DEBUG inputreader 'Syslog_TLS' registered 2017-03-17 16:34:16 DEBUG Inputreader 'Syslog_TLS' registered 2017-03-17 16:34:16 DEBUG outputwriter 'Syslog_TLS' registered 2017-03-17 16:34:16 DEBUG Outputwriter 'Syslog_TLS' registered 2017-03-17 16:34:16 DEBUG CONFIG: multiline 2017-03-17 16:34:16 DEBUG inputreader 'multiline' registered 2017-03-17 16:34:16 DEBUG Inputreader 'multiline' registered 2017-03-17 16:34:16 DEBUG CONFIG: oerrorin1 2017-03-17 16:34:16 DEBUG adding string [/u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log] 2017-03-17 16:34:16 DEBUG string literal declared at line 29, character 86 in /etc/nxlog.conf 2017-03-17 16:34:16 DEBUG literal 2017-03-17 16:34:16 DEBUG parsed expression 2017-03-17 16:34:16 DEBUG CONFIG: udpout 2017-03-17 16:34:16 DEBUG CONFIG: fileout2 2017-03-17 16:34:16 DEBUG adding string [/var/log/logmsg2.txt] 2017-03-17 16:34:16 DEBUG string literal declared at line 43, character 27 in /etc/nxlog.conf 2017-03-17 16:34:16 DEBUG literal 2017-03-17 16:34:16 DEBUG parsed expression 2017-03-17 16:34:16 DEBUG pidfile /var/run/nxlog/nxlog.pid removed 2017-03-17 16:34:16 DEBUG daemonizing... 2017-03-17 16:34:16 DEBUG INIT: _syslog 2017-03-17 16:34:16 DEBUG INIT: multiline 2017-03-17 16:34:16 DEBUG INIT: oerrorin1 2017-03-17 16:34:16 DEBUG INIT: udpout 2017-03-17 16:34:16 DEBUG Pollset initialized for module udpout (method: poll) 2017-03-17 16:34:16 DEBUG INIT: fileout2 2017-03-17 16:34:16 DEBUG Pollset initialized for module fileout2 (method: poll) 2017-03-17 16:34:16 DEBUG now running as group nxlog 2017-03-17 16:34:16 WARNING additional group memberships couldn't be set because getgrouplist()and setgroups() are not available on this platform 2017-03-17 16:34:16 DEBUG now running as user nxlog 2017-03-17 16:34:16 DEBUG running as uid: 203, euid: 203, gid: 204, egid: 204 2017-03-17 16:34:16 DEBUG pidfile /var/run/nxlog/nxlog.pid created 2017-03-17 16:34:16 DEBUG parsing path: oerrorin1 =>udpout 2017-03-17 16:34:16 DEBUG adding module oerrorin1 to route 1 2017-03-17 16:34:16 DEBUG adding module udpout to route 1 2017-03-17 16:34:16 DEBUG parsing path: in2 => fileout2 2017-03-17 16:34:16 ERROR [router.c:68/nx_route_add_module()] module 'in2' is not declared at /etc/nxlog.conf:54 2017-03-17 16:34:16 DEBUG adding module fileout2 to route tcproute 2017-03-17 16:34:16 ERROR [router.c:347/nx_add_route()] route tcproute is not functional without input modules, ignored at /etc/nxlog.conf:54 2017-03-17 16:34:16 DEBUG jobgroup created with priority 99 2017-03-17 16:34:16 DEBUG jobgroup created with priority 10 2017-03-17 16:34:16 DEBUG spawning 3 worker threads 2017-03-17 16:34:16 DEBUG worker thread 0 started 2017-03-17 16:34:16 DEBUG worker thread 1 started 2017-03-17 16:34:16 DEBUG worker thread 2 started   I have no idea about this problem.    /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log   --> user and group is oracle:oinstall and i add the nxlog user in oinstall group， but why continue report Permission denied?  

Hi there! I am a college student...i am working on windows event logger to collect & analyse windows logs(event & syslog). I don't know how to configure nxlog.conf file such that ... i can get all windows generated logs locally on host machine. Waiting for reply. Thank You

have recently been trying the Community Edition of nxlog which does exactly what I need (and more) but have an issue with the msi. In order to be something I could deploy it has to pass uninstall and re-install tests. Unfortunatley the uninstall via Windows installer leaves a lot of file and registry entries behind that subsequently causes issues with a re-installation. Does anyone know if this is likely to be addressed within CE? Does anyone know if Enterprise edition has the same msi issue? Thanks. 

Hello All,   I am using nxlog-ce-2.9.1716 client to collect logs from my device and then forward it to logstash. I want to make sure nxlog client accept syslogs from particular source instead any. However when using specific IP in im_udp module, I am getting error as below:   "2017-03-07 16:04:51 ERROR failed to start im_udp; couldn't bind udp socket to 172.20.20.20:514; The requested address is not valid in its context. "    my input driver is configured as below:   <Input in>     Module    im_udp     Host    172.20.20.20     Port    514 </Input>   I tried by changing host as below as well:   Host 172.20.20.20/32 - No luck Host 172.20.20.20/255.255.255.0 - No luck Host 172.20.20.20/255.255.255.255 - No luck Host 172.20.20.20 255.255.255.255 - no luck Host 172.20.20.20 255.255.255.0 - No luck   However if I make Host 0.0.0.0 - it works , but this is not what I am looking for.    Please help if I am missing anything.        Regards, Gaurang 

Hi! I'm new to NXLOG so sorry if this is a dumb question. So I have a Windows 2012 server that I'm attempting to set up to accept Syslog messages from an outsourced proxy system. I've been able to get NXLOG accept the logs and dump them to a flat file which our SIEM tool can pick up. However it ends up being a gigantic file...  I'm trying to refine this now. Instead of a huge file that we purge out every day, I'd like to have NXLOG limit the log dump to something like 200MB, copy the now older 200MB file to a different directory appending a time stamp to it, then start collecting in a new file in the original folder. The catch is, the live log that is in the original folder needs to keep the same file name, otherwise our SIEM tool won't know what to look for. I've tried to have a PowerShell script do this and it works, however, if the NXLOG service is running, the log dump file is locked. We could get around this by disabling the service while copying the old log file to a new location, however we'd lose 10-30 minutes of logs in the file copy (not ideal). Going through the reference manual I see a few ways to limit file size, starting a new file, and copying files over directly in the NXLOG config file. However it's turning out to be a problem condencing all of that into one configuration file that works on Windows. I've only been able to get a 1 of 3 pieces working at one time.   I appreciate any help the community can provide! Let me know if there are any questions or if something isn't clear.  

​ What is the most efficient way to parse Microsoft DNS Server debug logs into something more tidy, say into a CSV or KVP format on the nxlog agent? Consider the following log message: "24/02/2017 16:37:22 09B0 PACKET  0000009657E7BA40 UDP Rcv 10.0.100.15   a490   Q [0001   D   NOERROR] A      (7)example(3)com(0)" First of all, what would be the most efficient way performance-wise to convert this into a CSV or KVP format? And also, is there some other way besides using Exec and replacing parenthesis and numbers in a sed-like manner to get the clean query name? We have tried to use the Exec method before, but we were hitting some serious performance issues on busy DNS servers. I have currently switched on to using the pm_pattern module to drop invalid log lines (the beginning of the log file and empty lines) and I was wondering if there would be some easy way to perform both of the tasks (the formatting and the cleaning) using the pm_pattern module? An example output could look something like the following: datetime=24/02/2017 16:37:22,thread_id=09B0,context=PACKET,packet_id=0000009657E7BA40,protocol=UDP,action=Rcv,remote_ip=10.0.100.15, xid=a490,event_type=-,opcode=Q,flags_hex=0001,is_authorative=-,is_truncated=-,recursion_desired=D,recursion_available=-, response_code=NOERROR,question_type=A,question_name=example.com The empty or "-" values result from fields specified in the DNS debug log format that are not present in the above message (e.g. all possible flags would be "ATDR", and event_type is "-" because "R" marks a response but an empty value (whitespace) marks a query. And of course, if the above even is possible, would it be too resource consuming?

I'm not sure how to characterize what's going on, but here goes... My route path has two inputs, an itermittently high-volume input, and a low-volume input. The high-volume input can be thousands in a couple minutes, or it can be practically nothing. The low-volume input is, at most, one or two entries per ~3-4 minutes. There are also three outputs, two HTTP and a rotated file. They are disconnecting a fair bit, presumably due to timeouts or lack of pipelining or something. In my current configuration, I consistently find that my log data for the high-volume log gets dropped after a couple minutes. I'm not sure if the timing correlates with the HTTP disconnects, but it might. Sometimes I get just over a thousand log lines through, sometimes I get a couple hundred log lines, sometimes I get a couple thousand. Interestingly, the low-volume log is unaffected. I do have flow control enabled, and putting a buffer on the inputs did not seem to help. I didn't try disabling flow control because I don't understand it very well. I have to have both inputs going into the same route because how the messages interleave is important to the meaning of the entries. Here's the path that I use:   Path       vg_tsw_client, vg_tsw_combat => vg_tsw_pattern => vg_tsw_unparse_finder => vg_tsw_testfile, vg_tsw_es, vg_tsw_cdb _client is the low-volume, _combat is the high volume. _pattern is a pm_pattern with ~25-30 regexes and a small script on every pattern (0-10 lines), _unparse_finder is a pm_null uses add_to_route to copy unmatched patterns to a new route (0 hits lately) and does some light data enrichment via Exec. _testfile is the rotated file output, _es and _cdb are the HTTP outputs. Thanks in advance.

Hello, I have question. My variable $EventTime contains DateTime in only this format "2017-12-30 01:30:00" How me with use NxLog, convert to UNIX TIME format or convert to this format  Dec 30 01:30:00. Thank

Hello, I need to read logfiles from oracle, which are structured in xml with attributes It's nxlog still unable to access the attributes? I've read here that the entrerprise edition does it  https://nxlog.co/announcing-nxlog-enterprise-edition-v30 Handling structured data formats better The xm_xml extension has been enhanced so that it can now parse nested XML and data stored in XML attributes. Parsing of nested JSON has been also implemented in xm_json and UTF-8 validation can be enforced in order to avoid parser failures caused by invalid UTF-8 in other tools. This is a feature of the enterprise edition only or it will be ported to community edition too?

I'm trying to get a PatternDB working correctly, and it looks like I'm getting some fields but not all of them. There's only one pattern that's actually generating extra fields, and even it is dropping the first field (ParsedDate). Not sure what's going on here... Config file (via file inclusion): <Extension json>     Module      xm_json </Extension> <Extension syslog>     Module      xm_syslog </Extension> <Input vg_tsw_client>   Module     im_file   File       "C:\Program Files (x86)\Steam\steamapps\common\The Secret World\ClientLog.txt"   Exec       if not ($raw_event =~ /Scaleform\.TSWACT/) drop();   Exec         parse_syslog(); </Input> <Input vg_tsw_combat>   Module     im_file   File       "C:\Program Files (x86)\Steam\steamapps\common\The Secret World\CombatLog-*.txt"   Exec       if ($raw_event =~ /Sprinting [VI]+/) drop();   Exec         parse_syslog(); </Input> <Processor vg_tsw_pattern>     Module    pm_pattern     PatternFile %ROOT%\conf\SecretWorld\patterndb.xml </Processor> <Output vg_tsw_testfile>   Module     om_file   File       "C:\\ProgramData\\nxlogs\\vg-tsw-logs.log"   Exec       to_json(); </Output> <Route vg_tsw_route>   Path       vg_tsw_client, vg_tsw_combat => vg_tsw_pattern => vg_tsw_testfile </Route> Pattern DB: <?xml version='1.0' encoding='UTF-8'?> <patterndb>  <created>2010-01-01 01:02:03</created>  <version>42</version>   <group>   <name>tswCombat</name>   <id>50284624</id>   <matchfield>     <name>SourceModuleName</name>     <type>exact</type>     <value>vg_tsw_combat</value>   </matchfield>   <pattern>     <id>1000</id>     <name>basic combat swing</name>     <matchfield>      <name>Message</name>      <type>regexp</type>         <!-- [00:00:28] (Critical) Solomon County Cop's Spray and Pray hits (Normal) Ravenous Horde for 522 physical damage. (Normal) -->      <value>^\[([^\]]+)\] ((?:\(Critical\) |\(Normal\) )?)(.+?'s|Your) (.+?) hits \((Normal|Glancing)\) (.*?) for (\d+) (physical|magical) damage. \((Normal|Penetrated|Blocked)\)</value>      <capturedfield>       <name>ParsedTime</name>       <type>datetime</type>      </capturedfield>      <capturedfield>       <name>CriticalHit</name>       <type>string</type>      </capturedfield>      <capturedfield>         <name>AttackerName</name>         <type>string</type>      </capturedfield>      <capturedfield>         <name>AttackName</name>         <type>string</type>      </capturedfield>      <capturedfield>         <name>Glancing</name>         <type>string</type>      </capturedfield>      <capturedfield>         <name>VictimName</name>         <type>string</type>      </capturedfield>      <capturedfield>         <name>Damage</name>         <type>integer</type>      </capturedfield>      <capturedfield>         <name>DamageType</name>         <type>string</type>      </capturedfield>      <capturedfield>         <name>BlockOrPen</name>         <type>string</type>      </capturedfield>     </matchfield>     <set>      <field>        <name>type</name>        <value>Swing</value>        <type>string</type>      </field>     </set>   </pattern> </group> <group>   <name>tswClient</name>   <id>50284625</id>   <matchfield>     <name>SourceModuleName</name>     <type>exact</type>     <value>vg_tsw_client</value>   </matchfield>     <pattern>     <id>2000</id>     <name>tswact load plugin</name>     <matchfield>      <name>Message</name>      <type>regexp</type>         <!-- [2017-02-10 05:47:07Z #3886] [ID:0] ERROR: Scaleform.TSWACT - TSWACT Loaded for |Sheriban| -->      <value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - TSWACT Loaded for - \|(\w+)\|</value>      <capturedfield>       <name>ParsedTime</name>       <type>string</type>      </capturedfield>      <capturedfield>       <name>PlayerName</name>       <type>string</type>      </capturedfield>     </matchfield>     <set>      <field>        <name>type</name>        <value>TswactLoaded</value>        <type>string</type>      </field>     </set>   </pattern>     <pattern>     <id>2001</id>     <name>tswact load playfield</name>     <matchfield>      <name>Message</name>      <type>regexp</type>         <!-- [2017-02-10 05:47:07Z #3886] [ID:0] ERROR: Scaleform.TSWACT - Playfield - |Kingsmouth Town| -->      <value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - Playfield - \|(\w+)\|</value>      <capturedfield>       <name>ParsedTime</name>       <type>datetime</type>      </capturedfield>      <capturedfield>       <name>ZoneName</name>       <type>string</type>      </capturedfield>     </matchfield>     <set>      <field>        <name>type</name>        <value>SetZoneName</value>        <type>string</type>      </field>     </set>   </pattern>   <pattern>     <id>2002</id>     <name>tswact enter combat</name>     <matchfield>      <name>Message</name>      <type>regexp</type>         <!-- [2017-02-10 05:00:22Z #10910] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Sprinting VI:Elemental Force:Third Degree :World Domination| -->      <value>^\[([0-9-:]+)Z #\d+\] \[ID:\d+\] ERROR: Scaleform.TSWACT - Enter combat - \|(\w+)\|</value>      <capturedfield>       <name>ParsedTime</name>       <type>datetime</type>      </capturedfield>      <capturedfield>       <name>PlayerName</name>       <type>string</type>      </capturedfield>     </matchfield>     <set>      <field>        <name>type</name>        <value>EnterCombat</value>        <type>string</type>      </field>     </set>     <exec>       $TestField = 'testValue';     </exec>   </pattern> </group> </patterndb> Some of the output I'm getting: {"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] Your Pop Shot hits (Normal) Undead Islander for 1437 physical damage. (Normal)","CriticalHit":"","AttackerName":"Your","AttackName":"Pop Shot","Glancing":"Normal","VictimName":"Undead Islander","Damage":1437,"DamageType":"physical","BlockOrPen":"Normal","PatternID":1000,"PatternName":"basic combat swing","type":"Swing"} {"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] (Critical) Your Pop Shot hits (Normal) Undead Islander for 2965 physical damage. (Penetrated)","CriticalHit":"(Critical) ","AttackerName":"Your","AttackName":"Pop Shot","Glancing":"Normal","VictimName":"Undead Islander","Damage":2965,"DamageType":"physical","BlockOrPen":"Penetrated","PatternID":1000,"PatternName":"basic combat swing","type":"Swing"} {"EventReceivedTime":"2017-02-10 11:45:00","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:00","Hostname":"shepard","Message":"[11:45:00] You gain buff Live Wire"} {"EventReceivedTime":"2017-02-10 11:45:01","SourceModuleName":"vg_tsw_client","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:01","Hostname":"shepard","Message":"[2017-02-10 16:45:01Z #18498] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban|"} {"EventReceivedTime":"2017-02-10 11:45:10","SourceModuleName":"vg_tsw_combat","SourceModuleType":"im_file","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2017-02-10 11:45:10","Hostname":"shepard","Message":"[11:45:10] Buff Live Wire terminated."} Some of the vg_tsw_combat input file: [11:45:00] Your One in the Chamber hits (Normal) Undead Islander for 231 physical damage. (Normal) [11:45:00] Buff Sudden Return terminated on Undead Islander. [11:45:00] Buff One in the Chamber terminated on Undead Islander. [11:45:00] You gained 146 XP. [11:45:00] Undead Islander died. [11:45:00] Your Sudden Return hits (Normal) Undead Islander for 259 physical damage. (Normal) [11:45:00] Your Pop Shot hits (Normal) Undead Islander for 2045 physical damage. (Penetrated) [11:45:00] Your Pop Shot hits (Normal) Undead Islander for 2175 physical damage. (Penetrated) [11:45:00] Your Pop Shot hits (Normal) Undead Islander for 1437 physical damage. (Normal) [11:45:00] (Critical) Your Pop Shot hits (Normal) Undead Islander for 2965 physical damage. (Penetrated) [11:45:00] You gain buff Live Wire [11:45:02] You start using Sprinting VI. [11:45:03] You gain buff Sprinting VI [11:45:03] You successfully used Sprinting VI. [11:45:10] Buff Live Wire terminated. Some of the vg_tsw_client input: [2017-02-10 16:33:43Z #6790] [ID:0] ERROR: Scaleform.TSWACT - TSWACT Loaded for |Sheriban| [2017-02-10 16:33:43Z #6790] [ID:0] ERROR: Scaleform.TSWACT - Playfield - |The Savage Coast| [2017-02-10 16:34:12Z #7313] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:World Domination| [2017-02-10 16:34:14Z #7373] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban| [2017-02-10 16:39:06Z #10609] [ID:0] ERROR: MagicCommand - Trying to prepone the execute timeline to the pass. Spell:7760057 [2017-02-10 16:39:06Z #10624] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Elemental Force:World Domination| [2017-02-10 16:39:08Z #10655] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban| [2017-02-10 16:44:58Z #18330] [ID:0] ERROR: MagicCommand - Trying to prepone the execute timeline to the pass. Spell:7760057 [2017-02-10 16:44:59Z #18388] [ID:0] ERROR: Scaleform.TSWACT - Enter combat - |Sheriban|Buffs:Elemental Force:World Domination| [2017-02-10 16:45:01Z #18498] [ID:0] ERROR: Scaleform.TSWACT - Out of combat - |Sheriban| Any ideas?

Hello There, Please help me to get answers of below questions. 1. What all logs can be captured using NXLog client on Windows, Unix, AIX and Linux platforms? 2. What is the system prerequsites for installing NXLog client on Windows, UNIX, AIX and Linux platforms? Thank you.