Centralized Windows log collection - NXLog Enterprise Edition vs. WEF

WEF has several limitations when it comes to performance. In fact, there is an entire section, WEC server’s limitations, in Microsoft’s Windows security documentation dedicated to known "factors that limit the scalability of WEC servers", namely:

Microsoft’s recommendation for a stable WEC server on commodity hardware is an average of 3,000 events per second. This relatively low EPS rate will be easily exceeded in medium to large environments.

Agent-based log collection with NXLog reduces these challenges considerably. NXLog is a lightweight agent that runs in the background. It can process log volumes above 100,000 events per second, and there is no need to save events to file if you don’t wish to. Whether you choose a centralized deployment method, where agents send events to a central NXLog agent, or agents send events directly to your SIEM, you can use compression to reduce the network load. Several compression methods are available:

In agent-less (remote) log collection, monitoring the health of your log sources is more complex, and WEF does not provide monitoring tools. You are also introducing a single point of failure. If a WEC server becomes overwhelmed, some events will be lost, which will negatively impact the entire logging infrastructure. How can you monitor your collector to ensure it doesn’t reach its maximum logging rate, and thus prevent it from dropping events? Or, even worse, how do you determine if it already has reached its maximum logging capacity?

NXLog implements default delivery guarantees and flow control systems that ensure logs reach their destination. Further mechanisms are available for reliable log delivery, including memory and disk-based caching, to help you avoid data loss. In addition, there are several ways you can monitor the health of NXLog agents. Since it runs as a service, you can easily monitor it using monitoring tools you may already have deployed in your environment. NXLog Manager is available for more advanced monitoring, but more on that below.

Once your WEC servers are up and running, you can easily configure WEF settings on your log sources via Group Policy. However, WEF’s most significant weaknesses are management, health monitoring, and capacity planning. Unfortunately, not only does Event Viewer fail to offer any viable management options, its performance also takes a noticeable hit when you’re dealing with multiple WEC servers, causing it to slow down and become unresponsive. Third-party options to manage a WEF setup are also minimal. What about large environments with several collector servers? How do you distribute the load from WEF clients, you may be asking? Well, that is something you’ll have to figure out yourself because WEF does not support load balancing of any kind, not even third-party load balancers.

With NXLog, you can implement a complete log collection solution that is easy to deploy, monitor, and manage. Besides deploying agents via Group Policy, there are other options you can choose from. NXLog Manager is a web-based management system that facilitates agent deployment and configuration and supports clustered setups. If your organization prefers an API-first agent management solution, there’s NXLog Minder (in closed beta testing). It’s an automation-friendly solution that can easily integrate with software deployment and management platforms.

It is common knowledge that WEF has a steep learning curve, and deploying it is not easy. Multiple server and client operating system components need to be preconfigured for WEF to work successfully, making it a daunting task to undertake. In fact, when browsing online support platforms, you often come across advice cautioning you to think twice about using the technology. Troubleshooting WEF is also not straightforward. When things are not working as expected, you are limited to a few Windows events with bogus error codes and messages.

Suppose you plan on forwarding logs from your WEC servers to a SIEM or a log analytics solution, which is standard practice for businesses that take security seriously; you will find that WEC servers have no capabilities to forward logs. In this case, you have no other option than to use another tool to collect the forwarded logs from WEC servers. Unfortunately, such a setup introduces a delay in getting your logs to their destination, which is undesirable for threat analysis and management.

In contrast, the NXLog configuration system is much easier to learn. The NXLog Documentation is well-maintained and covers the entire gamut: from getting you up and running to detailed integration guides, complete with examples and use cases for well over 100 popular, third-party solutions. In addition, NXLog’s rich feature-set allows you to achieve everything you need for a centralized log collection solution with ample benefits over WEF.

While WEF is a valuable tool and can be a viable solution for collecting logs from remote Windows machines, implementing it for large-scale log collection and centralization can be a headache. As highlighted in this blog post, a WEF setup does not scale well and becomes difficult to manage. Apart from that, you still require a solution to forward logs from your WEC servers to third-party platforms.

NXLog Enterprise Edition gives you the ability to implement an agent-based or hybrid solution that you can tailor to handle all your logging needs. Its ease of deployment, configuration, and management will reduce your operational costs, which in turn can significantly reduce the Total Cost of Ownership (TCO) of this scalable, enterprise-ready logging solution when compared with other, less capable alternatives. If you are looking for a centralized Windows log collection solution, we invite you to take NXLog for a test drive and see for yourself.