3
responses

Hi,

im new to NXlog, would like to understand few things in Nxlog.
1.what if a windows server which has nxlog running on it and lost connectivity to network? will the logs stored in buffer, if so what is the size of it and where is the location of it.

AskedAugust 8, 2018 - 2:03am

Answer (1)

By default, NXLog uses a method called FlowControl to handle this type of scenario. Each module has a memory-based 100 event buffer by default.
This built-in flow control mechanism ensures that the input modules will pause until the output modules can write. More specifically, the buffer will fill and then pause the input modules.
NXLog will continue to read logs where it left off when it is able to ship logs again.

For cases where a larger or disk based queue is needed, we have the pm_buffer module.

Please see the following links for more information as needed.

https://nxlog.co/documentation/nxlog-user-guide#processing_buffering
https://nxlog.co/documentation/nxlog-user-guide#processing-stops-if-an-output-fails
https://nxlog.co/documentation/nxlog-user-guide#config_module_flowcontrol
https://nxlog.co/documentation/nxlog-user-guide#pm_buffer
https://nxlog.co/documentation/nxlog-user-guide#reliability

Comments (2)

  • vivek's picture

    Thanks this was very useful. one final question. you have mentioned that "NXLog will continue to read logs where it left off when it is able to ship logs again." how its keep tracking of where it left off, is that based off line number or something else, if its based upon line number what if line number of the event got changed.

    the reason behind the question, my security log of the domain controller overrides in approx three to five hours.so looking for best solution not to loose the log

  • Zhengshi's picture
    (NXLog)

    Most logs write in append-only mode so where the pointer is shouldn't change, the file should just have additional lines. We cache the line/pointer position so that when NXLog starts up again, it will start where it left off. In the case where your logs are rotated, that shouldn't be an issue due to how the files are read and how often we poll. If you do notice an issue with dropping events, I would suggest opening a ticket or a new forum post with relevant info.

    If there is some reason that read/write performance is affected, you may want to look into pm_buffer and disk-based buffer.