NXLog search
Let's talk Start free
Let's talk Start free

Schedule log collection.

Tags: nxlog scheduler
#1 fbrollo

Hi everyone,

Do you know if it is possible to schedule a module execution ?

Or does the Schedule function only takes Exec commands ?

Here is the solution I've come up with to schedule log sending:

<Input im_file_test>
        Module  im_file
        File    '/mnt/test/scheduler/*log'
        InputType       LineBased
        SavePos TRUE
        <Schedule>
                  When 0 12 * * *
                  Exec fileop->file_copy("/mnt/test/test.log", "/mnt/scheduler/test.log");
        </Schedule>
</Input>

<Output om_tcp_siem>
Module  om_tcp
Host    192.168.0.10
Port    514
OutputType      LineBased
</Output>


<Route RouteTestB>
​    Path        im_file_test => om_tcp_siem
</Route>

Here is what I would like to do (getting rid of xm_fileop for access privilege reasons:)

<Input im_file_test>
        <Schedule>
          When 0 12 * * *
          Module  im_file
          File    '/mnt/test/test.log'
          InputType       LineBased
          SavePos TRUE
       </Schedule>
</Input>

Thank you for your time :)

Permalink
#2 b0ti Nxlog ✓ (Last updated )
#1 fbrollo
Hi everyone, Do you know if it is possible to schedule a module execution ? Or does the Schedule function only takes Exec commands ? Here is the solution I've come up with to schedule log sending: <Input im_file_test> Module im_file File '/mnt/test/scheduler/*log' InputType LineBased SavePos TRUE <Schedule> When 0 12 * * * Exec fileop->file_copy("/mnt/test/test.log", "/mnt/scheduler/test.log"); </Schedule> </Input> <Output om_tcp_siem> Module om_tcp Host 192.168.0.10 Port 514 OutputType LineBased </Output> <Route RouteTestB> ​ Path im_file_test => om_tcp_siem </Route> Here is what I would like to do (getting rid of xm_fileop for access privilege reasons:) <Input im_file_test> <Schedule> When 0 12 * * * Module im_file File '/mnt/test/test.log' InputType LineBased SavePos TRUE </Schedule> </Input> Thank you for your time :)

Your first option is obviously not very nice since it involves copying the file and the second is not supported. Basically NXLog runs as a daemon/service trying to collect events in real-time but there is also a separate binary called nxlog-processor that exits when it finishes processing the data.

With the above in mind there are 3 options:

  1. Use nxlog-processor and run it from cron with the same crontab specification (0 12 * * *) that you tried to use within the <Schedule> block.
  2. Use nxlog with im_exec that invokes a script reading the required files when you want it. This would be a bit tricky, especially if it needs to follow multiple files.
  3. NXLog has pm_blocker. This can be used to disallow sending data. See this example.

I think the first option would be the most suitable if the requirement is to ship all new data once a day.
Login to see more