Let's Talk
Let's Talk

Schedule log collection.

Tags: nxlog scheduler
#1 fbrollo

Hi everyone,

Do you know if it is possible to schedule a module execution ?

Or does the Schedule function only takes Exec commands ?

Here is the solution I've come up with to schedule log sending:

<Input im_file_test>
        Module  im_file
        File    '/mnt/test/scheduler/*log'
        InputType       LineBased
        SavePos TRUE
        <Schedule>
                  When 0 12 * * *
                  Exec fileop->file_copy("/mnt/test/test.log", "/mnt/scheduler/test.log");
        </Schedule>
</Input>

<Output om_tcp_siem>
        Module  om_tcp
        Host    192.168.0.10
        Port    514
        OutputType      LineBased
</Output>

<Route RouteTestB>
​    Path        im_file_test => om_tcp_siem
</Route>

Here is what I would like to do (getting rid of xm_fileop for access privilege reasons:)

<Input im_file_test>
        <Schedule>
          When 0 12 * * *
          Module  im_file
          File    '/mnt/test/test.log'
          InputType       LineBased
          SavePos TRUE
       </Schedule>
</Input>

Thank you for your time :)

Permalink
#2 b0ti Nxlog ✓ (Last updated )
#1 fbrollo

Hi everyone,

Do you know if it is possible to schedule a module execution ?

Or does the Schedule function only takes Exec commands ?

Here is the solution I've come up with to schedule log sending:

<Input im_file_test>
        Module  im_file
        File    '/mnt/test/scheduler/*log'
        InputType       LineBased
        SavePos TRUE
        <Schedule>
                  When 0 12 * * *
                  Exec fileop->file_copy("/mnt/test/test.log", "/mnt/scheduler/test.log");
        </Schedule>
</Input>

<Output om_tcp_siem>
        Module  om_tcp
        Host    192.168.0.10
        Port    514
        OutputType      LineBased
</Output>

<Route RouteTestB>
​    Path        im_file_test => om_tcp_siem
</Route>

Here is what I would like to do (getting rid of xm_fileop for access privilege reasons:)

<Input im_file_test>
        <Schedule>
          When 0 12 * * *
          Module  im_file
          File    '/mnt/test/test.log'
          InputType       LineBased
          SavePos TRUE
       </Schedule>
</Input>

Thank you for your time :)

Your first option is obviously not very nice since it involves copying the file and the second is not supported. Basically NXLog runs as a daemon/service trying to collect events in real-time but there is also a separate binary called nxlog-processor that exits when it finishes processing the data.

With the above in mind there are 3 options:

  1. Use nxlog-processor and run it from cron with the same crontab specification (0 12 * * *) that you tried to use within the <Schedule> block.
  2. Use nxlog with im_exec that invokes a script reading the required files when you want it. This would be a bit tricky, especially if it needs to follow multiple files.
  3. NXLog has pm_blocker. This can be used to disallow sending data. See this example.

I think the first option would be the most suitable if the requirement is to ship all new data once a day.
Login to see more