File Rotate Directory | Log collection solutions

#1 tgdesrochers 5 years ago

I am attempting to use xm_ifileop to rotate some logfiles I am collecting with nxlog. I can see that rotation works as expected if I specify the file path but can I use the same logic to rotate all files in directory.

Example: /var/log/osquery/ on linux/mac and C:\ProgramData\osquery\log on windows has 3 files in it that start with osdqueryd. and I want to watch those and rotate them if they get over 3M. I have tried on windows and Mac to use a * in the file path to specify the directory: define OSQLOGFILE C:\\ProgramData\\osquery\\log\\osqueryd.\*

but that doesn't rotate the log. If I specify each file by name then it works as expected but then I need 3 xm_fileop sections.

Is there an easy way to tell nxlog to rotate all files matching a pattern?

Here is my logic so far:

<Extension osquery_fileop>
    Module      xm_fileop
    # Check the log file size every hour and rotate if larger than 3 MB
    <Schedule>
        Every   1 hour
        Exec    if (file_exists('%OSQLOGFILE%') and (file_size('%OSQLOGFILE%') >= 1M)) file_cycle('%OSQLOGFILE%', 4);
    </Schedule>
</Extension>

Permalink

#2 b0ti Nxlog ✓ 5 years ago

#1 tgdesrochers

I am attempting to use xm_ifileop to rotate some logfiles I am collecting with nxlog. I can see that rotation works as expected if I specify the file path but can I use the same logic to rotate all files in directory. Example: /var/log/osquery/ on linux/mac and C:\ProgramData\osquery\log on windows has 3 files in it that start with osdqueryd. and I want to watch those and rotate them if they get over 3M. I have tried on windows and Mac to use a * in the file path to specify the directory: define OSQLOGFILE C:\\ProgramData\\osquery\\log\\osqueryd.\* but that doesn't rotate the log. If I specify each file by name then it works as expected but then I need 3 xm_fileop sections. Is there an easy way to tell nxlog to rotate all files matching a pattern? Here is my logic so far: <Extension osquery_fileop> Module xm_fileop # Check the log file size every hour and rotate if larger than 3 MB <Schedule> Every 1 hour Exec if (file_exists('%OSQLOGFILE%') and (file_size('%OSQLOGFILE%') >= 1M)) file_cycle('%OSQLOGFILE%', 4); </Schedule> </Extension>

The problem here is that file_exists() and file_size() cannot accept a wildcard since this really doesn't make sense so the condition will evaluate to FALSE and file_cycle() is not invoked.
There is no easy way to do what you need other than to declare the statement for each file, or to write an external script and run it via exec_async().