3
responses

Hello,

I have nxlog installed on a server where we collect multiple logging streams, nxlog writes those logs to a flat file that is tailed and sent out to our other solutions. This system was not built for log retention so I need nxlog to clear these log files every hour or potentially based on file size. I setup the xm_fileop module on them and defined each log file path and setup the rotation as pasted below, however the logs grow and are not being cleared. Please take a look and let me know where I may have gone wrong. Thank you.

define OUTPUTFILE0 /opt/nxlog/data/fortifirewall/forti.log
define OUTPUTFILE1 /opt/nxlog/data/ciscovpn/ciscovpn.log
define OUTPUTFILE2 /opt/nxlog/data/cylance/cylance.log
define OUTPUTFILE3 /opt/nxlog/data/gpcvpcflow/gpcvpcflow.log
define OUTPUTFILE4 /opt/nxlog/data/infobloxdhcp/infobloxdhcp.log
define OUTPUTFILE5 /opt/nxlog/data/juniperips/juniperips.log
define OUTPUTFILE6 /opt/nxlog/data/pulsevpn/pulsevpn.log
define OUTPUTFILE7 /opt/nxlog/data/tanium/tanium.log
define OUTPUTFILE8 /opt/nxlog/data/windhcp/windhcp.log
define OUTPUTFILE9 /opt/nxlog/data/windns/windns.log
define OUTPUTFILE10 /opt/nxlog/data/winevents/winevents.log

<Extension fileop>
Module xm_fileop
# Truncate the file every hour
<Schedule>
Every 1 hour
Exec file_truncate('%OUTPUTFILE0%');
Exec file_truncate('%OUTPUTFILE1%');
Exec file_truncate('%OUTPUTFILE2%');
Exec file_truncate('%OUTPUTFILE3%');
Exec file_truncate('%OUTPUTFILE4%');
Exec file_truncate('%OUTPUTFILE5%');
Exec file_truncate('%OUTPUTFILE6%');
Exec file_truncate('%OUTPUTFILE7%');
Exec file_truncate('%OUTPUTFILE8%');
Exec file_truncate('%OUTPUTFILE9%');
Exec file_truncate('%OUTPUTFILE10%');
</Schedule>
</Extension>

AskedMay 19, 2020 - 7:17pm

Comments (2)

  • seth.stenzel's picture
    (NXLog)

    Hey Anthony,

    I double-checked, and you don't have to have an input/output block after all. Also, I tested your config and it is working fine for me. Is this the entire config? Also, are there any errors in the agent log? I tested this on my system which is admittedly windows, but that shouldn't matter. I didn't see any issues until I reached writing to the log files at over 80 eps which created an occasional lockout condition. Is it possible that you have that much volume that quickly?

    I set up a lab test using your configuration but using Every 5 sec so I could test the truncation more frequently. I was able to write events to all 10 files and truncate them every 5 seconds with your conf listed above. In the cases that I did lockout the files due to an attempt in truncation when the file was being written to at over 80 eps to the files NXLog agent logged the access error. In those cases, truncation would fail and would not attempt again until the next scheduled time.

    Are there any other sections of the conf that could be causing issues, or is this the full conf?

    Thanks.

    ~Seth S.

  • anthonyweller's picture

    Hi Seth,

    I appreciate you taking the time to look into my config and testing it. I looked over the entire config file and I did notice extra <extension> tags and module lines. I pasted the config from the verbatim config section under expert from NXLOG manager. I thought that when I entered verbatim config I would need to add everything as I do when modifying the file directly on that agent but I realize now that the manager modules must add the relevant lines. I got it working now and I will make sure to keep an eye on how the config is written when adding a module with verbatim config from the manager. Thank you again for the attention on this I don't think I would have caught the extra erroneous lines had you not let me know that the config should have worked as written. Have a great day!

Answer (1)

Hello,

You need to put <Schedule> block somewhere into <Input> or <Output> block, it should work from there.
More info here, please take a look: https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#config_module_schedule

Regards, Arch