NxLog not even attempting to sync first few lines


#1 javid90khan

I have a im_file to om_tcp route on the community edition.
My applications starts generating logs at 5am and runs every 15 minutes.
I have observed that it essentially skips the first X lines and syncs things only after the x lines starting 5:15 and works fine throughout the day.
There is nothing in logs at 5am.
I have tried routing it to another om_file on the same machine, it works ok.

Can you please help me fix it?

Here is the config:

########################################
# Global directives #
########################################
User nxlog
Group nxlog

LogFile /var/log/nxlog/nxlog.log
LogLevel INFO

########################################
# Modules #
########################################
<Extension _syslog>
Module xm_syslog
</Extension>

<Extension _fileop>
Module xm_fileop
</Extension>

########################################
# Input #
########################################
<Input in-my-important-app>
Module im_file
File "/var/log/tomcat/my-important-app-*.log"
ReadFromLast True
SavePos True
Exec $Filename = "myApp/" +file_basename(file_name());
Exec $Hostname = hostname_fqdn();
</Input>

########################################
# Output #
########################################
# Output to destination
<Output out-destination>
Module om_tcp
Host [destination host]
Port [destination port]
OutputType Binary
</Output>

########################################
# Routes #
########################################
<Route to_destination>
Path in-my-important-app => out-destination
</Route>

#2 ArkadiyDeactivated Nxlog ✓
#1 javid90khan
I have a im_file to om_tcp route on the community edition. My applications starts generating logs at 5am and runs every 15 minutes. I have observed that it essentially skips the first X lines and syncs things only after the x lines starting 5:15 and works fine throughout the day. There is nothing in logs at 5am. I have tried routing it to another om_file on the same machine, it works ok. Can you please help me fix it? Here is the config: ######################################## # Global directives # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules # ######################################## <Extension _syslog> Module xm_syslog </Extension> <Extension _fileop> Module xm_fileop </Extension> ######################################## # Input # ######################################## <Input in-my-important-app> Module im_file File "/var/log/tomcat/my-important-app-*.log" ReadFromLast True SavePos True Exec $Filename = "myApp/" +file_basename(file_name()); Exec $Hostname = hostname_fqdn(); </Input> ######################################## # Output # ######################################## # Output to destination <Output out-destination> Module om_tcp Host [destination host] Port [destination port] OutputType Binary </Output> ######################################## # Routes # ######################################## <Route to_destination> Path in-my-important-app => out-destination </Route>

Cheers,

Most likely the reason is because of SavePos TRUE and ReadFromLast TRUE.
Those directives are making nxlog to remember, where did it stopped last time, to start from this exact point.
I suggest you to change those directives to FALSE and look what would happen.
Take a note that this will enable nxlog to read all provided files from start to finish every time you will start or restart it so be careful.

Regards, Arch