missing log when log file rotate | Log collection solutions

#1 SamRui 6 years ago

<Input Result_Log> Module im_file File "C:\xxx\Result*.log" SavePos TRUE </Input>

above is my nxlog input config.
Result-3156.log is my log file name and 3156 is pid. The log file will rotate when it reaches the max size and generate Result-3156.log.1 file. I find some logs loss at the bottom of Result-123.log.1 And nxlog's log shows "2018-03-01 20:48:02 INFO inode changed for 'C:\xxx\Result-3156.log' (25746->25799): reopening possibly rotated file" at the same time point.

I suspect this is caused by the monitor interval 1 second.

Is it a bug ? or nxlog can not guarantee this scenario ? or something can do to avoid it ?

Permalink

#4 b0ti Nxlog ✓ 6 years ago

#1 SamRui

<Input Result_Log> Module im_file File "C:\xxx\Result*.log" SavePos TRUE </Input> above is my nxlog input config. Result-3156.log is my log file name and 3156 is pid. The log file will rotate when it reaches the max size and generate Result-3156.log.1 file. I find some logs loss at the bottom of Result-123.log.1 And nxlog's log shows "2018-03-01 20:48:02 INFO inode changed for 'C:\xxx\Result-3156.log' (25746->25799): reopening possibly rotated file" at the same time point. I suspect this is caused by the monitor interval 1 second. Is it a bug ? or nxlog can not guarantee this scenario ? or something can do to avoid it ?

Log rotation (i.e. rename) should not cause im_file to miss any data while the file is kept open as it also keeps track of the file size but there are a number of parameters such as ActiveFiles, RenameCheck that are also relevant. See the docs for more info.

If you can reproduce this manually (e.g. via a script that fills the file and rotates it) then feel free to provide more information and we will investigate.