Hello I was wondering if it is possible to have multiple instances of im_dbi at one time. It is important for access to my global variables. If it is possibe, then  my question is: does NXLog make nx_im_dbi_conf_t for each module instance? Where I should store variables for module if I want to have per load visibility?

Not sure it can be achieved but wanted to check as to what was the best way to use UNC names in the File path.  I want to stage files locally from a remote location, digest and then delete the files. I have tried "//FileServer/directory/file as well as "\\FileServer/directory/file" without success. Is it possible to use a variation of im_exec such as  <Input mapdrive>   Module  im_file command  net  arg           use arg           z: arg          "\\fileserver\directory"   ....etc?   Thanks    Ash  

I believe the latest version of the windows ce msi is not available to download. In the following forum post it was mentioned that version nxlog-ce-2.9.1362 is out: http://nxlog.org/support-tickets/nxlog-crashing-windows-2012r2   I can't find that version for download via http://nxlog.org/products/nxlog-community-edition/download ? Please advised.    

2015-04-29 20:32:46 INFO nxlog-ce-2.9.1347 started 2015-04-29 21:08:14 WARNING stopping nxlog service 2015-04-29 21:08:14 WARNING nxlog-ce received a termination request signal, exiting... 2015-04-29 21:08:15 ERROR invalid keyword: else at C:\Program Files (x86)\nxlog\conf\nxlog.conf:60 2015-04-29 21:08:15 ERROR module 'in' has configuration errors, not adding to route '2' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:89 2015-04-29 21:08:15 WARNING not starting unused module in 2015-04-29 21:08:15 INFO connecting to 172.18.1.11:5142 2015-04-29 21:08:15 INFO nxlog-ce-2.9.1347 started 2015-04-29 21:08:15 ERROR if-else failed at line 77, character 234 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 77, character 80 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 17, got 1 in input ' #Software: Microsoft Exchange Server' ​   Here is my conf file #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define IGNORE_COMMENT if $raw_event =~ /^#/ drop(); define BASEDIR C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\MessageTracking define AgentLog_dir C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\AgentLog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log #LogLevel DEBUG <Extension _syslog>       Module      xm_syslog </Extension> <Extension _json>      Module        xm_json </Extension> <Extension ExAgentLog>     Module        xm_csv     Fields        $Timestamp,$SessionId,$LocalEndpoint,$RemoteEndpoint,$EnteredOrgFromIP,$MessageId,$P1FromAddress,$P2FromAddresses,$Recipient,$NumRecipients,$Agent,$Event,$Action,$SmtpResponse,$Reason,$ReasonData,$Diagnostics     FieldTypes    String,String,String,String,String,String,String,String,String,Integer,String,String,String,String,String,String,String     Delimiter    , </Extension> <Extension ExMSGTRK>     Module        xm_csv     Fields        $date-time,$client-ip,$client-hostname,$server-ip,$server-hostname,$source-context,$connector-id,$source,$event-id,$internal-message-id,$message-id,$recipient-address,$recipient-status,$total-bytes,$recipient-count,$related-recipient-address,$reference,$message-subject,$sender-address,$return-path,$message-info,$directionality,$tenant-id,$original-client-ip,$original-server-ip,$custom-data     FieldTypes    String,String,String,String,String,String,String,String,String,String,String,String,String,Integer,Integer,String,String,String,String,String,String,String,String,String,String,String     Delimiter    , </Extension> <Extension charconv>     Module xm_charconv     AutodetectCharsets utf-8,UNICODE,utf-16, utf-32, iso8859-2 </Extension> <Input in>     Module      im_file     File        '%BASEDIR%\MSGTRK????????*-*.LOG'     ReadFromLast FALSE     Exec        if $raw_event =~ /^#/ drop();\                                 else \                 {    \                     ExMSGTRK->parse_csv();\                     delete($SourceModuleName);\                     delete($SourceModuleType);\                     delete($EventReceivedTime);\                     $SourceName="Message Tracking Log";    \                     to_json();\                 } </Input> <Input in2>     Module        im_file     ReadFromLast FALSE     File        '%AgentLog_dir%\AgentLog*.LOG'          EXEC         if $raw_event =~ /^#/ drop();\                 else \                 {\                     ExAgentLog->parse_csv();    \                     delete($SourceModuleName);    \                     delete($SourceModuleType);    \                     delete($EventReceivedTime); \                     $SourceName="Agent Log";    \                     to_json();            \                 } </Input> <Output out2>     Module      om_tcp     Host        172.18.1.11     Port        5142 </Output> <Route 2>     Path        in,in2 => out2 </Route>

2015-04-29 10:12:10 ERROR procedure 'parse_csv' failed at line 50, character 24 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Couldn't parse datetime value: '锘?Software: Microsoft Exchange Server'   see http://stackoverflow.com/questions/2223882/whats-different-between-utf-8-and-utf-8-without-bom   

Hi, I was wondering if it is possible to ship MS Event logs from nxlog to Logstash directly without writing to disk first.

Hello, I'm trying to use file_cycle to clean up old NXLog files.  When I start NXLog I see my log file "Demo.log" created and being written to.  When my schedule executes I see the log file getting renamed to "Demo.log.1" but no new Demo.log file is created and NXLog still continues to write to the "Demo.log.1" file. I'm not sure if I have something set incorrectly or if there is a bug.  Here's the necessary bits from nxlog.conf: define NXLOG_DEMO c:\NXlog\Demo.log <Extension fileop> Module xm_fileop <Schedule> #Cycle the NXLog files daily and only keep 14 days When @daily Exec file_cycle('%NXLOG_DEMO%', 14); </Schedule> </Extension> <Output Demo_out> Module om_file file '%NXLOG_DEMO%' CreateDir TRUE </Output> I'm not sure if I have something set incorrectly or if there's a better way to do what I'm trying to accomplish. Thanks! Jeff  

Hi there! I was wondering if NXlog community edition has a failover in dbi module. I couldn't find reconnect in im_dbi. So if SQL read fail, then module will be stopped? Is there some plan for do it in future versions?

This is a lengthy description but pelase bear with me, I'm really starting to loose hope here... So I have tried to catch this "oversized string" and avoid it braking my logging but am not able to, even writing debug log failed. Here is the nxlog.log where you can see that it broke at 5:30, then source log changed and then it broke again and after that it wrote no more to debug nor to syslog anything: 2015-04-17 05:30:45 ERROR procedure 'file_write' failed at line 95, character 100 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 95, character 99 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:30:45 ERROR procedure 'to_syslog_ietf' failed at line 58, character 24 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:30:45 ERROR Syslog_TLS output is over the limit of 65000, will be truncated 2015-04-17 05:31:18 WARNING inode changed for 'C:\Program Files (x86)\Agfa\Sec\Audit\log\audit_9702ad06-126b-4dfd-8b38-ad007eecc9c1.log': reopening possibly rotated file 2015-04-17 05:31:18 ERROR procedure 'file_write' failed at line 95, character 100 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 95, character 99 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:31:18 ERROR procedure 'to_syslog_ietf' failed at line 58, character 24 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:31:18 ERROR Syslog_TLS output is over the limit of 65000, will be truncated 2015-04-17 05:31:18 ERROR procedure 'file_write' failed at line 95, character 100 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 95, character 99 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:31:18 ERROR procedure 'to_syslog_ietf' failed at line 58, character 24 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; oversized string, limit is 1048576 bytes 2015-04-17 05:31:18 ERROR Syslog_TLS output is over the limit of 65000, will be truncated 2015-04-17 05:31:18 ERROR string limit (1048576 bytes) reached 2015-04-17 05:31:22 ERROR last message repeated 2 times 2015-04-17 05:31:24 ERROR string limit (1048576 bytes) reached 2015-04-17 05:31:26 ERROR string limit (1048576 bytes) reached 2015-04-17 05:31:30 ERROR last message repeated 2 times 2015-04-17 05:31:33 ERROR string limit (1048576 bytes) reached ....and so on just this one message every few second (plus some debug.log rotantion messages) but strangely enough one old log entry from 05:31 popped up later: 2015-04-17 06:00:17 ERROR string limit (1048576 bytes) reached 2015-04-17 05:31:18 ERROR string limit (1048576 bytes) reached 2015-04-17 06:00:21 INFO removing file F:\\temp\debug.log.2 .. and also debug.log failed but this is not the concerne right now: 2015-04-17 07:00:21 ERROR failed to determine file size of 'F:\\temp\debug.log': The system cannot find the file specified.  The last events in debug.log.1 are: EventTime: , raw_event: !SYS 2015-04-17 05:31:18,754 - apr 08 11:33:31 193.40.48.28 <?xml version="1.0" encoding="UTF-8" ?> EventTime: , raw_event: <IHEYr4><DicomQuery><Keys></Keys><Requestor><IP></IP></Requestor><CUID></CUID><SyntaxUID>LittleIndianImplicit</SyntaxUID></DicomQuery><Host>193.40.48.28</Host><TimeStamp>2015-04-08T11:33:31+03:00</TimeStamp></IHEYr4> EventTime: , raw_event: !SYS 2015-04-17 05:31:18,832 - apr 08 11:33:31 193.40.48.28 <?xml version="1.0" encoding="UTF-8" ?>   The last NORMALISED event (the whole configuration depens on dropping the CUID's - there can be up to 20000 CUIDs in one event - and wiritin this instead: <UIDs>dropped by nxlog</UIDs> and taking the TimeStamp from the end of the raw event and making it the real EventTime) in syslog server is:   2015-04-08T11:33:31.000000+03:00 <IHEYr4><DICOMInstancesUsed><ObjectAction>Access</ObjectAction><AccessionNumber>83_13532</AccessionNumber><SUID>1.2.124.113532.192.168.100.131.20050117.92248.281238</SUID><Patient><PatientID>50411232772</PatientID><PatientName>PATIENT^1</PatientName></Patient><User><LocalUser>user1@Agfa Healthcare</LocalUser></User><UIDs>dropped by nxlog</UIDs><NumberOfInstances>91</NumberOfInstances><MPPSUID></MPPSUID></DICOMInstancesUsed><Host>193.40.48.28</Host><TimeStamp>2015-04-08T11:33:31+03:00</TimeStamp></IHEYr4>   So to come back to nxlog.log events in the beginning here is the output to syslog at the moment when nxliog broke at 5:30 and 5:31 (here you can see that some events start normally (without the tag) and with <IHEYr4> in the beginning but as they are here it means that the event time has not been replaced and the CUID's have not been cut out and these events then get broken up to serveral garbaged messages: @timestamp                                 tag                                                               severity      host                    facility      message 2015-04-17T05:30:45.776+03:00        debug    server1    invld    <IHEYr4><DICOMInstancesUsed><Object... 2015-04-17T05:30:45.777+03:00    UID><CUID>1.3.46.670589.11.24125.5.0.576....    notice    server1    user    UID>1.3.46.670589.11.24125.5.0.576.201404240852... 2015-04-17T05:30:45.777+03:00    531903787</CUID><CUID>1.3.46.670589.11...    notice    server1    user    08531907795</CUID><CUID>1.3.46.670589.... 2015-04-17T05:30:45.777+03:00    589.11.24125.5.0.3364.2014042408530195660</CUID...    notice    server1    user    70589.11.24125.5.0.3364.2014042408530207668</CU... 2015-04-17T05:30:45.778+03:00    ><CUID>1.3.46.670589.11.24125.5.0.576.201...    notice    server1    user    >1.3.46.670589.11.24125.5.0.576.201404240903450... 2015-04-17T05:30:45.778+03:00    .0.576.2014042409034275361</CUID><CUID>...    notice    server1    user    .2014042409034289370</CUID><CUID>1.3.4... 2015-04-17T05:30:45.778+03:00    D><CUID>1.3.46.670589.11.24125.5.0.576.20...    notice    server1    user    D>1.3.46.670589.11.24125.5.0.576.20140424090340... 2015-04-17T05:30:45.779+03:00    0.576.2014042409034712619</CUID><CUID>...    notice    server1    user    2014042409034726627</CUID><CUID>1.3.46... 2015-04-17T05:30:45.779+03:00                                                                                debug    server1    invld    <CUID>1.3.46.670589.11.24125.5.0.576.2014042... 2015-04-17T05:31:18.831+03:00                                                                               debug    server1    invld    <IHEYr4><DICOMInstancesUsed><Object... 2015-04-17T05:31:18.832+03:00    D><CUID>1.3.46.670589.11.24125.5.0.576.20...    notice    server1    user    D>1.3.46.670589.11.24125.5.0.576.20140424090340... 2015-04-17T05:31:18.832+03:00    UID><CUID>1.3.46.670589.11.24125.5.0.576....    notice    server1    user    UID>1.3.46.670589.11.24125.5.0.576.201404240852... 2015-04-17T05:31:18.832+03:00    531903787</CUID><CUID>1.3.46.670589.11...    notice    server1    user    08531907795</CUID><CUID>1.3.46.670589.... 2015-04-17T05:31:18.832+03:00    589.11.24125.5.0.3364.2014042408530195660</CUID...    notice    server1    user    70589.11.24125.5.0.3364.2014042408530207668</CU... 2015-04-17T05:31:18.833+03:00    .0.576.2014042409034275361</CUID><CUID>...    notice    server1    user    .2014042409034289370</CUID><CUID>1.3.4... 2015-04-17T05:31:18.833+03:00    ><CUID>1.3.46.670589.11.24125.5.0.576.201...    notice    server1    user    >1.3.46.670589.11.24125.5.0.576.201404240903450... 2015-04-17T05:31:18.836+03:00                                                                                debug    server1    invld    <CUID>1.3.46.670589.11.24125.5.0.576.2014042... 2015-04-17T05:31:18.836+03:00    0.576.2014042409034712619</CUID><CUID>...    notice    server1    user    2014042409034726627</CUID><CUID>1.3.46... 2015-04-17T05:31:18.841+03:00    UID><CUID>1.3.46.670589.11.24125.5.0.576....    notice    server1    user    UID>1.3.46.670589.11.24125.5.0.576.201404240852... 2015-04-17T05:31:18.841+03:00                                                                            debug    server1    invld    <IHEYr4><DICOMInstancesUsed><Object... 2015-04-17T05:31:18.841+03:00    589.11.24125.5.0.3364.2014042408530195660</CUID...    notice    server1    user    70589.11.24125.5.0.3364.2014042408530207668</CU... 2015-04-17T05:31:18.841+03:00    531903787</CUID><CUID>1.3.46.670589.11...    notice    server1    user    08531907795</CUID><CUID>1.3.46.670589.... 2015-04-17T05:31:18.842+03:00                                                                                debug    server1    invld    <CUID>1.3.46.670589.11.24125.5.0.576.2014042... 2015-04-17T05:31:18.842+03:00    D><CUID>1.3.46.670589.11.24125.5.0.576.20...    notice    server1    user    D>1.3.46.670589.11.24125.5.0.576.20140424090340... 2015-04-17T05:31:18.842+03:00    .0.576.2014042409034275361</CUID><CUID>...    notice    server1    user    .2014042409034289370</CUID><CUID>1.3.4... 2015-04-17T05:31:18.842+03:00    ><CUID>1.3.46.670589.11.24125.5.0.576.201...    notice    server1    user    >1.3.46.670589.11.24125.5.0.576.201404240903450... 2015-04-17T05:31:18.842+03:00    0.576.2014042409034712619</CUID><CUID>...    notice    server1    user    2014042409034726627</CUID><CUID>1.3.46...   so... I could live with the broken events scattered around (if I see it I know it broke there) but the main thing is that nxlog stops working - it seems it was able to overcome the problem at 5:30 but at 5:31:18 it stopped sending events from this log (other logs were not affected). So what should I do to make it not break ot at leas always recover and pick up at next message? Here are the important bits of my config: <Extension charconv> Module xm_charconv AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-1, iso8859-4 </Extension> <Output sslout> Module om_ssl Host 192.168.1.2 Port 10514 Exec to_syslog_ietf(); CAFile %CERTDIR%/cacert.pem CertFile %CERTDIR%/cert.pem CertKeyFile %CERTDIR%/key.pem AllowUntrusted TRUE OutputType Syslog_TLS Exec if $Message =~ /DEBUG/ drop(); Exec convert_fields("AUTO", "utf-8"); </Output> <Extension fileop> Module xm_fileop <Schedule> Every 1 hour Exec if (file_size('%ROOT3%\debug.log') >= 1M) file_cycle('%ROOT3%\debug.log', 2); </Schedule> </Extension> <Input agfaauditlog> Module im_file File 'C:\Program Files (x86)\Agfa\Sec\Audit\log\audit_*.log' SavePos TRUE ReadFromLast TRUE Exec file_write("%ROOT3%\debug.log", "EventTime: " + $EventTime + ", raw_event: " + $raw_event); Exec if $raw_event =~ /!SYS/ drop(); Exec if $raw_event =~ /\<TimeStamp\>(.+)\</ {$EventTime = parsedate($1);} Exec if $raw_event =~ /^(.+?)(CUID.+CUID)(.+)/ $raw_event = $1 + 'UIDs>dropped by nxlog</UIDs' + $3; </Input> <Route 1> Path internal, nxlog, mseventlog, agfaauditlog => sslout </Route>     P.S. and then I get events like this also from the same log, why is that? 2015-04-17T05:29:40.406+03:00 㰱㌾ㄠ㈰ㄵⴰ㐭〸吱ㄺ㌲㨰㘮〰〰〰⬰㌺〰⁡灳㑴汮‭‭‭⁛乘䱏䝀ㄴ㔰㘠䕶敮瑒散敩癥摔業攽∲〱㔭〴ⴱ㜠〵㨲... notice server1     user 䅣捥獳楯湎畭扥爾䡐剈䵒ㄱ㔰㐰㠰〲㰯䅣捥獳楯湎畭扥爾㱓啉䐾ㄮ㈮㈵〮ㄮ㔹⸴㜰⸱㌮㐵㈮㈰ㄵ〴〸〹㔲㐸⸶㈳⸱...

I am trying to append a pre-defined GUID to some application log files. The log files are written in JSON and I would like to append these logs with a GUID.  I have defined the GUID in my config file with the variable CUSTOMER_TOKEN. I am not sure how to accomplish this using the raw_event + function within my config settings. Sorry for the basic question. I read the community reference manual a few times, but couldn't quite figure out how to use the raw_event + to append the CUSTOMER_TOKEN to the logs.  Thank you in advance. Here is my config file: #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CUSTOMER_TOKEN 10401ffc-42c2-49a6-9292-7eb31c9df605   Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log     # Include fileop while debugging, also enable in the output module below #<Extension fileop> #    Module      xm_fileop #</Extension> <Extension json>     Module      xm_json </Extension> <Input MonitoringAgent>    Module   im_file    File     "C:\Users\logman\AppData\Local\Temp\MonitoringAgent.log"    #SavePos  TRUE </Input>    <Output out>    Module om_file    File "C:\Users\logman\Desktop\App_Logs\Logs.txt" </Output>   <Route 1>    Path MonitoringAgent => out </Route>  

Hello- I'm looking to use the xm_multiline extension to try to concatinate log messages that all fall under the same headerline, but to this point have had little luck.  All messages begin with either -E, -A, -W, -I, and all proceeding lines with "at" are part of the same message. Any assistance would be appreciated. Using nxlog.conf: ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json>     Module      xm_json </Extension>   <Extension syslog>    Module xm_syslog </Extension> <Extension charconv>     Module    xm_charconv     AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2 </Extension> <Extension multiline>     Module xm_multiline     Headerline /^-./ </Extension>   <Input internal>    Module im_internal    Exec  $Message = to_json();  </Input>   # Watch any file you'd like <Input bsi_watch>    Module   im_file    File     "D:\\ose\\log\\S.SI.*_*_*_*.log"    SavePos  TRUE    InputType LineBased    Exec $source_server = 'Servername'; $source_file = file_name(); $message = $raw_event; to_json(); </Input> <Output out>     Module      om_tcp     Host        6.x.x.x     Port        5514 </Output> <Route 1>    Path internal, si_watch => out </Route> example log: -E 03-25 04:37:16.477 10992 30 (ISE02E_50013) () GTS_ORA Exception while initializing ReferenceData. OSE.Library.ITF.ITFMessaging.MessageRequestTimeout: Message request timed out: sessId=I.B.ORA_13.3D3B01ECB, reqId=1 at OSE.Library.RefData.Client.Singleton`1.Get(Originator orig) in D:\OSE_WD_I\OSE\library\ReferenceDataService\RefDataClient\Singleton.cs:line 93 at OSE.Applications.Options.OrderRoutingSystem.ORS.ResourceManager..ctor() in D:\OSE_WD_IORS\ISE\Applications\Options\ORS\S-ORA\ResourceManager.cs:line 150 at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init() in D:\OSE_WD_IORS\OSE\Applications\Options\ORS\S-ORA\S-ORA.cs:line 9160 at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init() at OSE.Library.SIFramework.AdapterBase.AdapterBase.SetTraceAndInit()

Hi, We are looking for a way to only send certain windows application log types to Loggly, could use some help in getting this setup. Sample (sanitzed) windows application log: Log Name:      Application Source:        PlatformService Date:          4/15/2015 5:59:58 PM Event ID:      0 Task Category: None Level:         Information Keywords:      Classic User:          N/A Computer:     XXXXXX.domain.com Description: AccountId: 6239745 Email: f3a61cd60de521d6d2c4598713b6e0600aae4e17 Client: PlatformService EventType: Stats LoginMethod: Setup Event Xml: <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">   <System>     <Provider Name="PlatformService" />     <EventID Qualifiers="0">0</EventID>     <Level>4</Level>     <Task>0</Task>     <Keywords>0x80000000000000</Keywords>     <TimeCreated SystemTime="2015-04-15T17:59:58.000000000Z" />     <EventRecordID>XXXXXX</EventRecordID>     <Channel>Application</Channel>     <Computer>XXXXXX.domain.com</Computer>     <Security />   </System>   <EventData>     <Data>AccountId: 123456 Email: 123456 Client: Harmony Platform Service EventType: Stats LoginMethod: Setup </Data>   </EventData> </Event> We want to be able to search in Loggly using source: source = "PlatformService"

Hello, Has anyone observed any memory leaks with the community edition of nxlog v2.1.2148 on Windows (2008R2, 2012, and 2012R2)? On our busier servers, we periodically will see a burst of errors like the following in the nxlog.log file: 2015-04-12 12:22:09 ERROR EvtNext failed with error 14: Not enough storage is available to complete this operation. 2015-04-12 12:22:10 ERROR EvtUpdateBookmark failed: The handle is invalid. 2015-04-12 12:22:11 ERROR EvtNext failed with error 14: Not enough storage is available to complete this operation. 2015-04-12 12:22:11 ERROR EvtUpdateBookmark failed: The handle is invalid. (These two errors can take up megabytes of space in the logfile.) Once I see these errors, nxlog is effectively "mute" until I restart it. I currently have a system where this has happened, and the nxlog process is taking over 700MB of RAM. I do have nxlog configured with pm_buffer (memory), with a buffer size of 100MB. If it's helpful, I've included my config below (flattened and comments removed -- it was spread across two files with one including the other). For troubleshooting memory leaks on Linux, I've seen comments about using Valgrind. Is there something comparable for Windows? Thanks, - Daniel ############################################################################### define ROOT C:\Program Files (x86)\nxlog define EVLOGHOST ip_address_of_my_loghost Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension xml> Module xm_xml </Extension> <Processor membuffer> Module pm_buffer MaxSize 102400 Type Mem WarnLimit 76800 </Processor> <Input internal> Module im_internal Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Input> <Input eventlog> Module im_msvistalog Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; Exec $EventType = lc(string($EventType)); Exec $FileName = lc(string($FileName)); Exec $Hostname = lc(string($Hostname)); Exec $Severity = lc(string($Severity)); Exec delete($SourceModuleType); Exec delete($EventTimeWritten); Exec delete($EventTime); Exec rename_field("Message", "full_message"); Exec if ($IpAddress =~ /::ffff:(.*)/) $IpAddress = $1; Exec to_json(); </Input> <Output EventLogOut> Module om_tcp Host %EVLOGHOST% Port 3515 </Output> <Route EventLogRoute> Path internal, eventlog => membuffer => EventLogOut </Route> ###############################################################################

I have looked at debugging options but they all seem to write out any message that arrives but is there an option to write the raw message to debug log only if there is a problem with parsing of this message?

TL;DR: what's the recommended way of converting logs to a common (e.g. GELF) format? I'm using NXLog together with Logstash and EalsticSearch. I'm collecting logs from Windows, NXLogs (internal) and my app logs using line based JSON. Windows logs and NXLogs seem to share a lot of field names. I can write my app so that it uses the same fields. This greatly facilitates viewing data in elasticsearch. I could stick with windows fields or convert them all to GELF. AFAIK, the convertion from Windows Logs to GELF seems seems to require a lot of per-field convertion work. There is a good chance I won't get it right until enough data is produced. I was looking for a convert_to_gelf() function which would take care of converting Windows Logs, Internal logs, IIS, etc to GELF. Is there such thing? Is manual conversion my only option?

Dear NxLog Community, I've heard some debate lately as to the question of 64-bit Windows support with NxLog.  One camp claims that the NxLog "Community Edition" cannot export Event Logs from 64-bit Windows systems, a.k.a Server 2008 R2 and Server 12.  These folks argue that to get 64-bit Windows support the NxLog Enterprise Edition is required. Is this correct?  If so, does anyone have a link to an NxLog document that explains this?  Is there an official document highlighting the differences between the Community and Enterprise Editions?     Thanks, groundLoop

Hello! I had install NXLog agent on domain controllers with WUS.   Installation ik, doomain is ok. But I see in NXlog.log this:   INFO reconnecting in 1 seconds ERROR om_tcp send failed; An established connection was aborted by the software in your host machine.   WUS service stop working: it do not dowsnload updates from central WUS-server. WUS logs looks fine, but in WUS console it show looks like update was not downloaded To make  work WUS and NXLog I need to reinstall WUS - after that it work fine

Can I insert a custom tag (right now it is not present "-" ) somehow like this maybe: <Input hl7out> Module         im_file File         'C:\Connectivity\mcf\log\hl7out.log' Exec        $tag = hl7out </Input>

If wildcards are not enough, can I specify multiple files like   File "/var/log/messages;/var/log/otherlog;/var/log/something" Or are regular expressions allowed here?

I am a bit confused by the documentation on nxlogce.  Below is my sample config.  It collects some of the logs just fine but does not collect the logs that I want.  Security comes through just fine, application not so much, and system is spotty. I would like to explictyly define Microsoft-Windows-TerminalServices-LocalSessionManager/Operational.  If that is not possible I would like to get the forwarded logs from a logcollector and index them.  I have tried the documented query statements       Query   <QueryList>\         <Query Id="0">\ #            <Select Path="Security">*</Select>\             <Select Path="System">*</Select>\             <Select Path="Application">*</Select>\             <Select Path="Setup">*</Select>\             <Select Path='Microsoft-Windows-TerminalServices-LocalSessionManager/Operational'>*</Select>\         </Query>\     </QueryList> and this does not seem to work. If I just leave the sataement as below I do get some messages back but not all, Do I need to buffer my messages to get everything flowing through?        ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf>     Module      xm_gelf </Extension>     <Input Eventlog>     # Use 'im_mseventlog' for Windows XP and 2003     Module      im_msvistalog </Input>   <Output outevt>     Module      om_tcp     Host       myhost.mycomany.local     Port        1338     OutputType  GELF </Output> <Route Eventlog>     Path        Eventlog => outevt </Route>