Messsage Log was truncated, restarting from beginning

#1 jmlps 6 years ago

Hi,

I'm using nxlog ce latest version with centos 7 with latest patches to read files from a directory that is a cifs mount to a windows server. Logs are in w3c format.

However from times to times, I'm getting the information in nxlog log file "INFO input file '' was truncated, restarting from the beginning", but the file isn't truncated.

Using strace with options -tfp (since debug option of nxlog doesn't give me much information) i get this:

pid 7675] 12:52:58 close(16 <unfinished ...> [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=144080774, ...}) = 0 [pid 7676] 12:52:58 stat("", <unfinished ...> [pid 7675] 12:52:58 <... close resumed> ) = 0 [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=54218426, ...}) = 0 [pid 7675] 12:52:58 stat("", <unfinished ...> [pid 7676] 12:52:58 stat("", <unfinished ...> [pid 7675] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7675] 12:52:58 fstat(26, {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=141529135, ...}) = 0 [pid 7676] 12:52:58 stat("", {st_mode=S_IFREG|0400, st_size=148346198, ...}) = 0

My nxlog configuration is this:

User nxlog Group nxlog

LogFile /var/log/nxlog/nxlog.log LogLevel INFO

define DIR

Module xm_syslog Module im_file File '%DIR%/_*_.w3c' SavePos TRUE ReadFromLast TRUE Recursive TRUE RenameCheck TRUE CloseWhenIdle TRUE PollInterval 5 DirCheckInterval 10 ActiveFiles 4

    <Exec>
                  to_syslog_bsd();
    </Exec>

Module om_tcp Host Port 514 Path in1 => out_syslog-ng

Can you understand why nxlog says that file is truncated when it's not?

Permalink

#2 b0ti Nxlog ✓ 6 years ago

#1 jmlps

Hi, I'm using nxlog ce latest version with centos 7 with latest patches to read files from a directory that is a cifs mount to a windows server. Logs are in w3c format. However from times to times, I'm getting the information in nxlog log file "INFO input file '' was truncated, restarting from the beginning", but the file isn't truncated. Using strace with options -tfp (since debug option of nxlog doesn't give me much information) i get this: pid 7675] 12:52:58 close(16 <unfinished ...> [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=144080774, ...}) = 0 [pid 7676] 12:52:58 stat("", <unfinished ...> [pid 7675] 12:52:58 <... close resumed> ) = 0 [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=54218426, ...}) = 0 [pid 7675] 12:52:58 stat("", <unfinished ...> [pid 7676] 12:52:58 stat("", <unfinished ...> [pid 7675] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7675] 12:52:58 fstat(26, {st_mode=S_IFREG|0400, st_size=277116066, ...}) = 0 [pid 7676] 12:52:58 <... stat resumed> {st_mode=S_IFREG|0400, st_size=141529135, ...}) = 0 [pid 7676] 12:52:58 stat("", {st_mode=S_IFREG|0400, st_size=148346198, ...}) = 0 My nxlog configuration is this: User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO define DIR Module xm_syslog Module im_file File '%DIR%/_*_.w3c' SavePos TRUE ReadFromLast TRUE Recursive TRUE RenameCheck TRUE CloseWhenIdle TRUE PollInterval 5 DirCheckInterval 10 ActiveFiles 4 <Exec> to_syslog_bsd(); </Exec> Module om_tcp Host Port 514 Path in1 => out_syslog-ng Can you understand why nxlog says that file is truncated when it's not?

I think it may be related to the cifs mount and the way it reports inode number, file size and file modification date that makes the NXLog CE think the file was truncated. I remember there were some fixes to im_file wrt network shares by I'm not sure what the status of this is in the CE. You may want to test the NXLog EE as it likely handles this better.