Agentless vs. Agent based log collection

The NXLog Enterprise Edition offers both agentless and agent based log collection so it's possible to use it in both ways or even in mixed mode.

In most enterprise environments the common data sources are mainly Syslog, local log files (IIS, DNS, custom app logs) and Windows Eventlog. Let's take a look at how this event data can be collected with the agentless or the agent based method.

Agentless log collection

The NXLog Enterprise Edition is capable of collecting log data from Syslog sources over the UDP, TCP or TLS/SSL. It supports both the RFC3164 and RFC5424 based Syslog format.

Windows Eventlog can be collected with the NXLog Enterprise Edition using the following options:

  • Using the im_wmi input module that can pull eventlog data remotely over the WMI protocol.
  • Using the im_msvistalog input module that can pull eventlog data remotely over MSRPC.
  • Using the im_wseventing input module to act as a Windows Event Forwading (WEF) collector. It works on all platforms so you can direct windows eventlogs from GPO to a machine on Linux running NXLog EE.
  • Set up Windows Event Forwarding and only install NXLog as the agent on the server that receives the forwarded events.

Pros:

  • Agentless log collection does not introduce an additional requirement to install agents on all the servers we wish to collect logs from avoiding potential issues caused by the agent software.

Cons:

  • Most syslog devices can only send data over UDP which lacks security and reliability. Data sent over UDP gets lost while the log server is restarted or becomes unreachable.
  • TLS/SSL is rarely supported by syslog devices whereas a reliable and encrypted transport might be a requirement that is mandated by various compliance standards such as PCI/DSS.
  • WMI operates with a much higher overhead than the agent based method. In our experience Windows Event Forwarding can also be problematic in high EPS scenarios.
  • WMI does not transfer ancillary event data stored with the Windows EventLog such as the extra fields in the Security log.
  • WEF can only forward data that is stored in the Windows Eventlog. Logs from ETW (i.e. Analytic and Debug channels) cannot be forwarded via WEF.
  • It's not always possible to properly filter data on the source system.
  • Agent health cannot be monitored.
  • Event data stored in local files or databases cannot be collected.

Agent based log collection

The NXLog Enterprise Edition can be installed on most Windows and Linux platforms to be used as a forwarder, i.e. agent.

Pros:

  • The NXLog Enterprise Edition can forward syslog data over UDP, TCP and TLS/SSL using RFC3164 or RFC5424 based syslog format. It can be installed on Linux systems without the need to replace the stock syslog daemon.
  • Windows Eventlog can be collected more efficiently than using WMI or Windows Event Forwarding. The im_msvistalog input module is capable of collecting all the information stored in the Windows Eventlog records such as the ancillary fields, including UserData and EventData.
  • Event data can be collected from local files in virtually any format.
  • The ability to send structured data (JSON, Binary, KVP, etc).
  • The NXLog Enterprise Edition offers extra delivery guarantees to avoid message loss such as flow-control and acknowledged message transfer  in addition to supporting encryption for security and compression for low-bandwidth environments.
  • Event data can be filtered, normalized and rewritten on the client side.
  • The NXLog Enterprise Edition can send to more than one destination in different formats.
  • Low resource overhead with a memory consumption typically around a few megabytes and negligable CPU load on even busier systems.
  • The agent health can be monitored.

Cons:

  • Some devices such as network or embedded devices do not support installing third-party tools.
  • Deployment and management of agents across a large number of systems poses additional challenges, though NXLog Manager will greatly simplify this process.

Feel free to download a trial of the NXLog Enterprise Edition and evaluate both options to see which can serve your requirements better.

Share this post