I am using om_tcp for forwarding Windows logs to a SIEM system. What will be the expected behavior of nxlog if e.g. a firewall blocks the TCP connections from the nxlog agent to the SIEM? Is there any potential danger in nxlog buffering outgoing logs so that large amounts of memory or disk space would be consumed on the sending host while the connections get blocked?
It should not consume disk space or large amounts of memory unless you configure it that way.
NXLog has proper flow-control. If the om_tcp module receives a connection error, it will print a message in nxlog.log. All modules in the route which are feeding your om_tcp instance will be paused until om_tcp can send again. There is only a small internal memory queue which stores 100 messages for each module instance by default. If you want more buffering you can use pm_buffer or adjust the built-in QueueSize.
Comments (2)
So, om_tcp instance will "remember" last position in log file, and resume reading at this position after network connection to SIEM/syslog server is restored again ?
Yes, but im_file does that.