Nxlog-ce memory leak?
Hello,
Has anyone observed any memory leaks with the community edition of nxlog v2.1.2148 on Windows (2008R2, 2012, and 2012R2)?
On our busier servers, we periodically will see a burst of errors like the following in the nxlog.log file:
2015-04-12 12:22:09 ERROR EvtNext failed with error 14: Not enough storage is available to complete this operation.
2015-04-12 12:22:10 ERROR EvtUpdateBookmark failed: The handle is invalid.
2015-04-12 12:22:11 ERROR EvtNext failed with error 14: Not enough storage is available to complete this operation.
2015-04-12 12:22:11 ERROR EvtUpdateBookmark failed: The handle is invalid.
(These two errors can take up megabytes of space in the logfile.)
Once I see these errors, nxlog is effectively "mute" until I restart it.
I currently have a system where this has happened, and the nxlog process is taking over 700MB of RAM. I do have nxlog configured with pm_buffer (memory), with a buffer size of 100MB. If it's helpful, I've included my config below (flattened and comments removed -- it was spread across two files with one including the other).
For troubleshooting memory leaks on Linux, I've seen comments about using Valgrind. Is there something comparable for Windows?
Thanks,
- Daniel
###############################################################################
define ROOT C:\Program Files (x86)\nxlog
define EVLOGHOST ip_address_of_my_loghost
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension json>
Module xm_json
</Extension>
<Extension xml>
Module xm_xml
</Extension>
<Processor membuffer>
Module pm_buffer
MaxSize 102400
Type Mem
WarnLimit 76800
</Processor>
<Input internal>
Module im_internal
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>
<Input eventlog>
Module im_msvistalog
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;
Exec $EventType = lc(string($EventType));
Exec $FileName = lc(string($FileName));
Exec $Hostname = lc(string($Hostname));
Exec $Severity = lc(string($Severity));
Exec delete($SourceModuleType);
Exec delete($EventTimeWritten);
Exec delete($EventTime);
Exec rename_field("Message", "full_message");
Exec if ($IpAddress =~ /::ffff:(.*)/) $IpAddress = $1;
Exec to_json();
</Input>
<Output EventLogOut>
Module om_tcp
Host %EVLOGHOST%
Port 3515
</Output>
<Route EventLogRoute>
Path internal, eventlog => membuffer => EventLogOut
</Route>
###############################################################################
- Upgrade to 2.9.1347
- Get rid of pm_buffer in your conf. It's not useful with flow-control and om_tcp