4
responses

Hello,
We are using nxlog-ce 3.0.2284 on Windows 2016 to collect event logs and forward them to our new SIEM.

We experience an increasing delay in the reception of these events by the SIEM. It starts with a few seconds delay and after a moment, the delay has progressively increased to hours. We collect all the event logs, without any filter and the observed throughput is around 150eps, with peaks at 700eps. The output is om_tcp with TLS. Last, we disabled the buffer processor to see if it helps, but with no improvement.

The server does not seem to be very loaded in terms of RAM, CPU and disk.

Are we reaching nxlog maximum performance ? Looking at the docs, it seems and the default values for BatchSize is 31 and PollInterval is 1 second. Does it mean that the maximum input thoughput is 31eps ? Lowering the PollInterval to 0.1 seconds seems to help, but is it the recommended strategy to improve performance ?

How can we configure nxlog to improve the performance ?

Thank you for your help !

David

AskedAugust 29, 2022 - 2:21pm

Comments (4)

  • Klevin's picture
    (NXLog)

    Hello Sir,

    Please can you share the config you are using? Please remember to redact sensitive data since this forum is public.

    Sincerely
    Klevin

  • dgpv's picture

    Hello,

    Here is the configuration :
    ## Please set the ROOT to the folder your nxlog was installed into,
    ## otherwise it will not start.
    LogLevel INFO

    define ROOT C:\Program Files\nxlog
    define CERTDIR %ROOT%\cert

    Moduledir %ROOT%\modules
    CacheDir %ROOT%\data
    Pidfile %ROOT%\data\nxlog.pid
    SpoolDir %ROOT%\data
    LogFile %ROOT%\data\nxlog.log

    <Extension _syslog>
    Module xm_syslog
    </Extension>

    <Extension _json>
    Module xm_json
    </Extension>

    <Input eventlog>
    # Use 'im_mseventlog' for Windows XP, 2000 and 2003
    Module im_msvistalog
    Exec $Message = to_json();
    </Input>

    <Output rsyslog>
    Module om_tcp
    Host ******
    Port 10524
    OutputType Syslog_TLS

    Exec to_syslog_ietf();
    </Output>

    #<Route eventlog_to_rsyslog>
    # Path eventlog => rsyslog
    #</Route>

    <Route eventlog_to_rsyslog>
    Path eventlog => buffer =>rsyslog
    </Route>

    <Processor buffer>
    Module pm_buffer
    # 20 MB buffer
    MaxSize 20480
    Type Disk
    # warn at 10 MB
    WarnLimit 10240
    </Processor>

    We added "PollInterval 0.1" to the im_msvistalog input. But is it the recommended configuration to improve the performance ?

    Thank you,

  • NenadM's picture
    (NXLog)

    It might be a good idea to 'locate' the delay - in other words: does it start on the output side of the NXLog or on the input side of your SIEM.
    In order to check it, you can add a simple test output module on the NXLog side and try writing some logs to a file:

    <Output debug_file_output>
    Module om_file
    File '/some_dir/some_file'
    </Output>

    In case the delay is present even when writing logs to a local file, my best bet for the cause would be the pm_buffer module. Using pm_buffer is only recommended when there is a chance of message loss. The built-in flow control in NXLog ensures that messages will not be read by the input module until the output side can send, store, or forward.

    When reading from the Windows EventLog with im_msvistalog it is rarely necessary to use the pm_buffer module unless log rotation is used.
    In this case, you use Type Disk - disk based buffering. Have you tried checking the size of the file used for buffering? Or disabling the pm_buffer module (at least temporary)?

Answers (0)