Centralized Windows log collection - NXLog Enterprise Edition vs. WEF

One of the challenges that security-conscious Windows administrators face is collecting and centralizing Windows event logs. One of the obvious solutions that come to mind is the native Windows Event Forwarding (WEF) feature available on all modern Windows operating systems.

WEF offers the convenience of forwarding Windows events to a central event collector without installing and managing agents. To objectively portray the role this valuable technology plays in the larger scope of enterprise log collection, we have written several articles that discuss it:

There is no doubt that WEF can be advantageous, especially in small or mixed environments where you only need to collect events from a handful of Windows machines. However, when it comes to centralizing logs from hundreds or thousands of sources, WEF can be a challenge to work with, and in the long term, you are likely to run into problems. This blog post looks at some of the drawbacks and presents how you can implement a more robust centralized log collection strategy with NXLog Enterprise Edition and agent-based log collection.

Resource Utilization & Performance

WEF has several limitations when it comes to performance. In fact, there is an entire section, WEC server’s limitations, in Microsoft’s Windows security documentation dedicated to known "factors that limit the scalability of WEC servers", namely:

  • Received events are saved to local EVTX files, so you must ensure ample disk space is available. The disk writing speed will also limit the speed of logging.

  • Remote log collection increases the network load. WEF doesn’t use compression, so you can expect the impact on your network to be pretty high. Additionally, the number of WEF clients that can connect to the server simultaneously is limited by the open ports available on the server.

  • As the number of WEF sources increases, you risk the registry and Event Viewer becoming unusable, leaving you to manage your subscriptions with a command-line tool.

Microsoft’s recommendation for a stable WEC server on commodity hardware is an average of 3,000 events per second. This relatively low EPS rate will be easily exceeded in medium to large environments.

Agent-based log collection with NXLog reduces these challenges considerably. NXLog is a lightweight agent that runs in the background. It can process log volumes above 100,000 events per second, and there is no need to save events to file if you don’t wish to. Whether you choose a centralized deployment method, where agents send events to a central NXLog agent, or agents send events directly to your SIEM, you can use compression to reduce the network load. Several compression methods are available:

  • SSL compression when using TLS/SSL

  • HTTP compression when using HTTP(s)

  • NXLog’s compression protocol

Stability & Reliability

In agent-less (remote) log collection, monitoring the health of your log sources is more complex, and WEF does not provide monitoring tools. You are also introducing a single point of failure. If a WEC server becomes overwhelmed, some events will be lost, which will negatively impact the entire logging infrastructure. How can you monitor your collector to ensure it doesn’t reach its maximum logging rate, and thus prevent it from dropping events? Or, even worse, how do you determine if it already has reached its maximum logging capacity?

NXLog implements default delivery guarantees and flow control systems that ensure logs reach their destination. Further mechanisms are available for reliable log delivery, including memory and disk-based caching, to help you avoid data loss. In addition, there are several ways you can monitor the health of NXLog agents. Since it runs as a service, you can easily monitor it using monitoring tools you may already have deployed in your environment. NXLog Manager is available for more advanced monitoring, but more on that below.

Deployment & Management

Once your WEC servers are up and running, you can easily configure WEF settings on your log sources via Group Policy. However, WEF’s most significant weaknesses are management, health monitoring, and capacity planning. Unfortunately, not only does Event Viewer fail to offer any viable management options, its performance also takes a noticeable hit when you’re dealing with multiple WEC servers, causing it to slow down and become unresponsive. Third-party options to manage a WEF setup are also minimal. What about large environments with several collector servers? How do you distribute the load from WEF clients, you may be asking? Well, that is something you’ll have to figure out yourself because WEF does not support load balancing of any kind, not even third-party load balancers.

With NXLog, you can implement a complete log collection solution that is easy to deploy, monitor, and manage. Besides deploying agents via Group Policy, there are other options you can choose from. NXLog Manager is a web-based management system that facilitates agent deployment and configuration and supports clustered setups. If your organization prefers an API-first agent management solution, there’s NXLog Minder (in closed beta testing). It’s an automation-friendly solution that can easily integrate with software deployment and management platforms.

Configuration & Functionality

It is common knowledge that WEF has a steep learning curve, and deploying it is not easy. Multiple server and client operating system components need to be preconfigured for WEF to work successfully, making it a daunting task to undertake. In fact, when browsing online support platforms, you often come across advice cautioning you to think twice about using the technology. Troubleshooting WEF is also not straightforward. When things are not working as expected, you are limited to a few Windows events with bogus error codes and messages.

Suppose you plan on forwarding logs from your WEC servers to a SIEM or a log analytics solution, which is standard practice for businesses that take security seriously; you will find that WEC servers have no capabilities to forward logs. In this case, you have no other option than to use another tool to collect the forwarded logs from WEC servers. Unfortunately, such a setup introduces a delay in getting your logs to their destination, which is undesirable for threat analysis and management.

In contrast, the NXLog configuration system is much easier to learn. The NXLog Documentation is well-maintained and covers the entire gamut: from getting you up and running to detailed integration guides, complete with examples and use cases for well over 100 popular, third-party solutions. In addition, NXLog’s rich feature-set allows you to achieve everything you need for a centralized log collection solution with ample benefits over WEF.

Feature Comparison

The following table compares WEF and how it stacks up against both editions of the NXLog agent. It looks at functional capabilities, supported operating systems, as well as some deployment and management options that one might expect from a log collection solution.

Table 1. Windows Event Forwarding vs. NXLog Manager/CE and EE
Feature WEF NXLog Community Edition NXLog Enterprise Edition

Operating System Support

Microsoft Windows

20

20

20

Debian

20

20

20

Ubuntu

20

20

20

RHEL

20

20

20

CentOS

20

20

20

SLES

20

20

20

FreeBSD

20

20

20

OpenBSD

20

20

20

macOS

20

20

20

IBM AIX

20

20

20

Oracle Solaris

20

20

20

Deployment and Management

Group Policy deployment

20

20

20

Log source health monitoring

20

20

20*

Web-based UI log source configuration and management

20

20

20*

API-based log source configuration and management

20

20

20*

Log Processing Features

Event filtering

20

20

20

Event parsing

20

20

20

Message rewrite

20

20

20

Data enrichment

20

20

20

Event log caching

20

20

20

Event correlation

20

20

20

UTC logging

20

20

20

Alerting

20

20

20

Log Transfer and Output Features

SSL/TLS encryption

20

20

20

Compression

20

20

20

Failover

20

20

20

Multiple output formats (syslog, JSON, XML, CSV, etc.)

20

20

20

Log forwarding to third-party platforms (SIEMs, log analytics solutions, etc.)

20

20

20

Event throttling

20

20

20

*Using NXLog Manager / NXLog Minder

Conclusion

While WEF is a valuable tool and can be a viable solution for collecting logs from remote Windows machines, implementing it for large-scale log collection and centralization can be a headache. As highlighted in this blog post, a WEF setup does not scale well and becomes difficult to manage. Apart from that, you still require a solution to forward logs from your WEC servers to third-party platforms.

NXLog Enterprise Edition gives you the ability to implement an agent-based or hybrid solution that you can tailor to handle all your logging needs. Its ease of deployment, configuration, and management will reduce your operational costs, which in turn can significantly reduce the Total Cost of Ownership (TCO) of this scalable, enterprise-ready logging solution when compared with other, less capable alternatives. If you are looking for a centralized Windows log collection solution, we invite you to take NXLog for a test drive and see for yourself.

GET STARTED TODAY:

NXLog Ltd. develops multi-platform log collection tools that support many different log sources, formats, transports, and integrations. The tools help administrators collect, parse, and forward logs so they can more easily respond to security issues, investigate operational problems, and analyze event data. NXLog distributes the free and open source NXLog Community Edition and offers additional features and support with the NXLog Enterprise Edition.

This document is provided for informational purposes only and is subject to change without notice. Trademarks are the properties of their respective owners.

Download a fully functional trial of the Enterprise Edition for free