1
response

We currently have 1 Nagios Log server to record logs and 1 windows server with NXlog installed, which has 2 types of logs, TLIB and SIP, from one folder. There are 16 TLIB logs and only 1 SIP log with around 25 increments of each. Both generate a 51,201kb file with 429780 lines and have a total of 483 files in the log folder.

When less logs are produced, the 483 logs are overwritten less often and are recorded to Nagios Log successfully. Both log types are recorded to Nagios Log within 1 second of the time stamp of the log entry.

When more logs are produced, the 483 logs are overwritten every few minutes. TLIB logs are recorded to Nagios Log successfully within 1 second of the time stamp of the log entry. However the SIP logs starts to fall behind. Entries recorded to Nagios Log can be upto 2 hours different from the time stamp of the log entry.

Does anyone know why one log location would fall behind when the other remains unaffected? Is there anyway to improve the reliability of the SIP logs that fall behind?

nxlog.conf

## See the nxlog reference manual at
## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension json>
Module xm_json
</Extension>

<Extension syslog>
Module xm_syslog
</Extension>

<Output out1>
Module om_tcp
Host xx.xx.xx.xx
Port 3515

Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();

# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>

<Output out2>
Module om_tcp
Host xx.xx.xx.xx
Port 3515

Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();

# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>

<Route 1>
Path SIP => out1
</Route>

<Route 2>
Path TLIB => out2
</Route>

<Extension multiline_SIPTLIB>
Module xm_multiline
HeaderLine /^@?\d\d:\d\d:\d\d./
</Extension>

<Input TLIB>
Module im_file
InputType multiline_SIPTLIB
File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001_TLIB-0*'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

<Input SIP>
Module im_file
InputType multiline_SIPTLIB
File 'E:\GenesysLogs\H_SIPS_01_SIP001\H_SIPS_01_SIP001.*'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

AskedJuly 6, 2022 - 10:56am

Comments (1)

  • konstantinos's picture
    (NXLog)

    Hi Trevor,

    A sample of your input log types could help to understand if the SIP logs have higher complexity/size that could be causing the delay. Another possibility is that your destination can't keep up during peak times and the NXLog agent detects that via the FlowControl feature to prevent dataloss. Finally there are some directives in im_file such as RenameCheck and ReadOrder that could be worth trying in case the delay is caused by the high number of files being monitored in the directory (even though most of them have been rotated).

    Kind regards,

    Konstantinos

Answers (0)