How to delete input files after reading ?
Hi Everyone,
New forum user, so sorry if I do not respect some uses rules…and I’m French by the way, so sorry about my bad English level too…;-)
I use Nxlog in the following context :
A machine A send periodically a file on a machine B (same file name)
Machine B is used like a first level of log centralization. Nxlog is installed on it
Then, Nxlog send log to Machine C who host Logstash
I’m looking for a process who remove input files after end of reading and processing, but I’m not sure about how process “in-out” operate, is-it after each line or after a n entire file reading ?
How can I test a “end of reading” condition ?
Right now, I’m going to test something like that :
<Extension fileop>
Module xm_fileop
</Extension>
<Input in>
Module im_file
SavePos TRUE
ReadFromLast TRUE
PollInterval 900 # 15mn
File "/var/log/inpufile*.log"
Exec $InputFileName1 = file_name()
</Input>
<Output logstash>
Module om_tcp
Port 6002
Host 10.x.y.z
Exec file_remove($InputFileName1);
</Output>
<Route 1>
Path in => logstash
</Route>
Is it correct ?
Thanks in advance for your help
NXLog reads the data from files one line per record by default. EOF might not always mean that the file is complete as there are situations when there is more data appended later.
Currently there is no support in im_file to delete the input on EOF, adding this feature is on the roadmap though. For now you should use external log rotation tools or <Schedule>.