11
responses
responses
demetri
I have the following NXLog config below that reads a file from disk and forwards it to another host (syslog). By default, it appears NXLog only forwards the data in the log file if it has changed.
How can you configure 'nxlog.conf' so that it always forwards the file, even if the data hasn't changed?
<Input in>
Module im_file
File "C:\\myfile.txt"
</Input>
<Output out>
Module om_tcp
Host 127.0.0.1
Port 12345
</Output>
<Route 1>
Path in => out
</Route>
Comments (10)
I've looked at setting both of those to FALSE. It didn't help with my objective of always sending the file while NXLog is running, even if the contents haven't changed. I have a a schedule running every 60 seconds that regenerates this file - I was hoping to send this file every 60 seconds regardless of whether or not it has changed.
<Extension _exec>
Module xm_exec
<Schedule>
Every 60 sec
Exec exec_async("C:\\create_myfile.txt.exe");
</Schedule>
</Extension>
What do you mean by
if it has changed?im_filewas designed to read append-only files and to only read the appended records. It also handles truncation and detection of rotated log files when it assumes the full content is new.That design makes sense b0ti.
But for me, I'd always like to send the file periodically, whether or not any changes have been detected. So for instance, consider this scenario:
1. myfile has contents "abc" @ 9:00
2. myfile has contents "abc" @ 9:10
3. myfile has contents "def" @ 9:20
I'd like the file to be sent in all 3 cases above via some config settings. Is this possible?
I see what you are trying to accomplish.
Is this script that makes the log something that you have written? If so, it may be easier and more direct to simply write to the file in append mode instead.
Hi,
Yes I have written this script/exe (code). I thought about appending to the end of the file, but eventually it will get huge.
I believe I can delete the file before the next scheduled run, which should then 'trick' NXLog to forward the data. But I am a little worried that there might be something 'in memory' that could prevent this.
You could also use the built in features in NXLog to rotate the log file so that it doesn't get so big.
https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_fileop
https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#xm_fileop_config_examples
Truncating the file and rewriting it is an anti-pattern that I would strongly advise against if you don't want data to be missed. As mentioned you should set up log rotation so that the file does not grow unbounded, this can be done by your script or by nxlog. See Log Rotation and Retention for more information.
Thanks for the suggestion. I'm attempting to perform the rotation on the file, but am having issues. Using my config at the beginning of the post, can you give me an example of how to do this?
Here is what I'm using now, but 'myfile.txt' is locked by NXLog's log rotation mechanism, so I can't write into myfile.txt at times
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _exec>
Module xm_exec
<Schedule>
Every 10 sec
Exec exec_async(<path-to-file-that-generates-C:\\myfile.txt>);
</Schedule>
</Extension>
<Input in>
Module im_file
File "C:\\myfile.txt"
</Input>
<Output out>
Module om_tcp
Host 127.0.0.1
Port 12345
</Output>
<Route 1>
Path in => out
</Route>
<Extension _fileop>
Module xm_fileop
</Extension>
<Output out2>
Module om_file
File "C:\\myfile.txt"
<Schedule>
Every 6 sec
<Exec>
file_cycle(file_name(), 2);
reopen();
</Exec>
</Schedule>
</Output>
<Route 2>
Path in => out2
</Route>
thanks.
A modification of the second example in the link suggested by @b0ti or the second link I sent should work for your uses.
Note that you shouldn't need both of the below schedules. First one will check every hour and rotate when the file gets to 1M in size, keeping 7 copies. Second one will rotate daily at midnight and keep 7 copies.
That works, thanks!