NXLog unable to keep up with logs being produced
We currently have 1 Nagios Log server to record logs and 1 windows server with NXlog installed, which has 2 types of logs, TLIB and SIP, from one folder. There are 16 TLIB logs and only 1 SIP log with around 25 increments of each. Both generate a 51,201kb file with 429780 lines and have a total of 483 files in the log folder.
When less logs are produced, the 483 logs are overwritten less often and are recorded to Nagios Log successfully. Both log types are recorded to Nagios Log within 1 second of the time stamp of the log entry.
When more logs are produced, the 483 logs are overwritten every few minutes. TLIB logs are recorded to Nagios Log successfully within 1 second of the time stamp of the log entry. However the SIP logs starts to fall behind. Entries recorded to Nagios Log can be upto 2 hours different from the time stamp of the log entry.
Does anyone know why one log location would fall behind when the other remains unaffected? Is there anyway to improve the reliability of the SIP logs that fall behind?
nxlog.conf
See the nxlog reference manual at
http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
#define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define CERT %ROOT%\cert
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log
<Extension json> Module xm_json </Extension>
<Extension syslog> Module xm_syslog </Extension>
<Output out1> Module om_tcp Host xx.xx.xx.xx Port 3515
Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();
# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
<Output out2> Module om_tcp Host xx.xx.xx.xx Port 3515
Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();
# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
<Route 1> Path SIP => out1 </Route>
<Route 2> Path TLIB => out2 </Route>
<Extension multiline_SIPTLIB> Module xm_multiline HeaderLine /^@?\d\d:\d\d:\d\d./ </Extension>
<Input TLIB> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001_TLIB-0*' SavePos TRUE Exec $Message = $raw_event; </Input>
<Input SIP> Module im_file InputType multiline_SIPTLIB File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001.*' SavePos TRUE Exec $Message = $raw_event; </Input>
Hi Trevor,
A sample of your input log types could help to understand if the SIP logs have higher complexity/size that could be causing the delay. Another possibility is that your destination can't keep up during peak times and the NXLog agent detects that via the FlowControl feature to prevent dataloss. Finally there are some directives in im_file such as RenameCheck and ReadOrder that could be worth trying in case the delay is caused by the high number of files being monitored in the directory (even though most of them have been rotated).
Kind regards,
Konstantinos