News and blog
NXLog main page
  • Products
    NXLog Platform
    Log collection
    Log management and analytics
    Log storage
    NXLog Community Edition
    Integrations
    Professional Services
  • Solutions
    Use cases
    Specific OS support
    SCADA/ICS
    Windows event log
    DNS logging
    MacOS logging
    Solutions by industry
    Financial Services
    Government & Education
    Entertainment & Gambling
    Telecommunications
    Medical & Healthcare
    Military & Defense
    Law Firms & Legal Counsel
    Industrial & Manufacturing
  • Plans
  • Partners
    Find a Reseller
    Partner Program
  • Resources
    Documentation
    Blog
    White papers
    Videos
    Webinars
    Case Studies
    Community Program
    Community Forum
  • About
    Company
    Careers
  • Support
    Support portals
    Contact us

NXLog Platform
Log collection
Log management and analytics
Log storage
NXLog Community Edition
Integrations
Professional Services

Use Cases
Specific OS support
SCADA/ICS
Windows event log
DNS logging
MacOS logging
Solutions by industry
Financial Services
Government & Education
Entertainment & Gambling
Telecommunications
Medical & Healthcare
Military & Defense
Law Firms & Legal Counsel
Industrial & Manufacturing


Find a Reseller
Partner Program

Documentation
Blog
White papers
Videos
Webinars
Case Studies
Community Program
Community Forum

Company
Careers

Support portals
Contact us
Let's Talk Start free
NXLog search
  • Loading...
Let's Talk Start free
March 3, 2020 siem

Sending ETW Logs to Splunk with NXLog

By Arielle Bonnici

Share
ALL SIEM STRATEGY SECURITY ANNOUNCEMENT DEPLOYMENT COMPLIANCE COMPARISON RSS

NXLog supports direct collection of Event Tracing for Windows (ETW) data. DNS Analytical logs, for example, can be forwarded to Splunk or another SIEM for monitoring and analysis.

Collecting ETW Logs

Event Tracing for Windows (ETW) is a kernel-level tracing facility that provides high-performance logging of kernel and application events. ETW events can be written to a log file or collected directly from the system in realtime via the Consumers API. ETW is a valuable source of event data that is underutilized compared to the well-known Windows Event Log. The Debug and Analytical channels, which are based on ETW, cannot be collected through Windows Event Log directly. For more information, see our white paper Solving Windows Log Collection Challenges with Event Tracing.

ETW logging is of particular interest for collecting DNS Server activity logs. Prior to Windows Server 2012 R2, activity logs were only available through DNS Server’s debug logging feature (See DNS Debug Logging in the NXLog User Guide.) Unfortunately, debug logging can significantly impact DNS Server performance and it was never intended to be enabled permanently. With the introduction of DNS Server Analytical logs in Server 2012 R2 and 2016, high-performance DNS activity logging is available through ETW.

The logman and tracerpt commands can be used to work with traces:

Creating and Controlling Traces with Logman
> logman query providers
Microsoft-Windows-DNSServer              {EB79061A-A566-4698-9119-3ED2807060E7}
[...]
> logman create trace DNSTrace -p Microsoft-Windows-DNSServer -o c:\trace.etl
> logman start DNSTrace
[Wait for activity to be logged]
> logman stop DNSTrace
> logman delete DNSTrace
Using Tracerpt to Generate a Dump File and Summary
> tracerpt c:\trace_000001.etl
[...]
DumpFile:           dumpfile.xml
Summary:            summary.txt

While these tools are suitable for working with temporary traces, they do not offer a good solution for setting up regular collection of ETW logs. A similar multi-stage process would need to be implemented to use the Splunk Universal Forwarder for Windows, as it does not support ETW directly.

NXLog Enterprise Edition, however, implements the ETW Consumers API to collect ETW events directly from the source for robust, efficient ETW logging. NXLog handles tracing setup, so it is not necessary for the user to run logman or tracerpt commands. NXLog does not require events to be written to disk, reducing the overhead required and allowing NXLog to perform well even at high event rates.

Configuring the HEC Data Source in Splunk

Before adding the NXLog configuration, a new log source needs to be added in Splunk to receive the logs.

  1. In the Settings menu, click Data inputs.

    splunk 1 data inputs
  2. In the Local Input section, for the HTTP Event Collector input type, click Add new, then select New Token.

  3. To configure the input source, enter the Name, then click Next. The remaining settings are optional.

  4. In the input settings, select Automatic or set the Source type to json, choose an App context; for example, Search & Reporting (search). The remaining settings are optional, then click Review.

    splunk hec 2
  5. Once reviewed, select Submit, then copy the Token for future use in the NXLog configuration.

  6. Returning back to the HTTP Event Collector page, select the Global Settings button.

  7. Select Enabled to enable Tokens, select Enable SSL if using SSL, then take note of the HTTP Port Number as it will be used in the NXLog configuration.

    splunk hec 1

    Now that an appropriate Splunk log source has been configured, you can continue by setting up NXLog Enterprise Edition to collect and forward ETW logs.

Configuring NXLog to Collect and Forward ETW Logs via HEC

NXLog can collect logs from ETW with the im_etw module. The Provider directive specifies the ETW provider to collect logs from, while the optional Level directive can be used to select a log level (the default is the "verbose" level). It is also possible to collect kernel trace logs with the KernelFlags directive and a comma-separated list of flags. For full configuration details, see im_etw in the NXLog Reference Manual.

Note
DNS Server Analytical logging is available beginning with Windows Server 2012 R2. For more information, and an alternative method for previous versions of Windows Server, see Windows DNS Server in the NXLog User Guide.
Adding the NXLog ETW Module Input

In the following configuration, NXLog reads DNS Server Analytical logs directly from the Microsoft-Windows-DNSServer ETW provider using the im_etw input module.

nxlog.conf
<Input in_dns>
    Module      im_etw
    Provider    Microsoft-Windows-DNSServer
</Input>
Adding the NXLog HEC Output Module

Send logs to the Splunk HEC endpoint by Configuring HTTP Event Collection (HEC) both on Splunk and NXLog.

Specify the HTTP Port Number and Token using the URL and AddHeader directives, respectively. For example, https://127.0.0.1:8088/services/collector and Authorization: Splunk 452d2e2d-b5c5-4001-bd47-d181de90af99.

Enable SSL via the HTTPSCAFile directive available in the om_http module if it is not already set in the Splunk configuration file.

The configuration sample below shows the output module instance with the necessary connection parameters as well as the processing needed in the <Exec> block to match the metadata and data formats Splunk expects to receive.

nxlog.conf
<Extension _json>
    Module  xm_json
</Extension>

<Output splunk_out_hec>
    Module      om_http
    URL         https://127.0.0.1:8088/services/collector
    AddHeader   Authorization: Splunk 452d2e2d-b5c5-4001-bd47-d181de90af99
    HTTPSCAFile %CERTDIR%/cacert.pem
    <Exec>
        # Rename event fields to what Splunk uses
        if $Severity rename_field($Severity, $vendor_severity);
        if $SeverityValue rename_field($SeverityValue, $severity_id);

        # Convert all fields to JSON and write to $event field
        $event = to_json();

        # Convert $EventTime to decimal seconds since epoch UTC
        $time = string(integer($EventTime));
        $time =~ /^(?<sec>\d+)(?<ms>\d{6})$/;
        $time = $sec + "." + $ms;

        # Specify the log source type
        $sourcetype = "_json";

        # Add other HEC metadata fields if available in the event data
        if $Hostname $host = $Hostname;
        if $SourceName $source = $SourceName;

        # Remove all non-metadata fields (already stored in $event)
        clean_splunk_fields->process();

        # Write to JSON
        to_json();
    </Exec>
</Output>
Adding the Splunk Route

Add a route for the ETW logs. If no route is specified in the configuration, NXLog will automatically create a single default route connecting all inputs to all outputs.

nxlog.conf
<Route splunk>
    Path    in_dns => splunk_out_hec
</Route>

The nxlog service should be restarted to apply the new configuration.

Confirming Log Collection in Splunk

To confirm data collection, go to the Splunk Search & Reporting dashboard and search for the correct logs, using either host name or source name.

splunk out
Figure 1. ETW Sample of a Forward Lookup Zone Deletion Event

Once forwarded to Splunk, administrators can leverage Splunk’s features such as visualizations, alerts, and other Splunk apps.

To read more about Splunk, please see the Splunk User Guide or contact us if you are interested in learning how NXLog can further enhance your Splunk deployments.

  • siem
  • splunk
  • etw
  • log forwarding
Share

Facebook Twitter LinkedIn Reddit Mail
Related Posts

Making the most of Windows Event Forwarding for centralized log collection
6 minutes | December 17, 2018
Agent-based versus agentless log collection - which option is best?
5 minutes | October 22, 2019

Stay connected:

Sign up

Keep up to date with our monthly digest of articles.

By clicking singing up, I agree to the use of my personal data in accordance with NXLog Privacy Policy.

Featured posts

Announcing NXLog Platform 1.6
April 22, 2025
Announcing NXLog Platform 1.5
February 27, 2025
Announcing NXLog Platform 1.4
December 20, 2024
NXLog redefines log management for the digital age
December 19, 2024
2024 and NXLog - a review
December 19, 2024
Announcing NXLog Platform 1.3
October 25, 2024
NXLog redefines the market with the launch of NXLog Platform: a new centralized log management solution
September 24, 2024
Welcome to the future of log management with NXLog Platform
August 28, 2024
Announcing NXLog Enterprise Edition 5.11
June 20, 2024
Raijin announces release of version 2.1
May 31, 2024
Ingesting log data from Debian UFW to Loki and Grafana
May 21, 2024
Announcing NXLog Enterprise Edition 6.3
May 13, 2024
Raijin announces release of version 2.0
March 14, 2024
NXLog Enterprise Edition on Submarines
March 11, 2024
The evolution of event logging: from clay tablets to Taylor Swift
February 6, 2024
Migrate to NXLog Enterprise Edition 6 for our best ever log collection experience
February 2, 2024
Raijin announces release of version 1.5
January 26, 2024
2023 and NXLog - a review
December 22, 2023
Announcing NXLog Enterprise Edition 5.10
December 21, 2023
Raijin announces release of version 1.4
December 12, 2023
Announcing NXLog Enterprise Edition 6.2
December 4, 2023
Announcing NXLog Manager 5.7
November 3, 2023
Announcing NXLog Enterprise Edition 6.1
October 20, 2023
Raijin announces release of version 1.3
October 6, 2023
Upgrading from NXLog Enterprise Edition 5 to NXLog Enterprise Edition 6
September 11, 2023
Announcing NXLog Enterprise Edition 6.0
September 11, 2023
The cybersecurity challenges of modern aviation systems
September 8, 2023
Raijin announces release of version 1.2
August 11, 2023
The Sarbanes-Oxley (SOX) Act and security observability
August 9, 2023
Log Management and PCI DSS 4.0 compliance
August 2, 2023
Detect threats using NXLog and Sigma
July 27, 2023
HIPAA compliance logging requirements
July 19, 2023
Announcing NXLog Enterprise Edition 5.9
June 20, 2023
Industrial cybersecurity - The facts
June 8, 2023
Raijin announces release of version 1.1
May 30, 2023
CISO starter pack - Security Policy
May 2, 2023
Announcing NXLog Enterprise Edition 5.8
April 24, 2023
CISO starter pack - Log collection fundamentals
April 3, 2023
Raijin announces release of version 1.0
March 9, 2023
Avoid vendor lock-in and declare SIEM independence
February 13, 2023
Announcing NXLog Enterprise Edition 5.7
January 20, 2023
NXLog - 2022 in review
December 22, 2022
Need to replace syslog-ng? Changing to NXLog is easier than you think
November 23, 2022
The EU's response to cyberwarfare
November 22, 2022
Looking beyond Cybersecurity Awareness Month
November 8, 2022
GDPR compliance and log data
September 23, 2022
NXLog in an industrial control security context
August 10, 2022
Raijin vs Elasticsearch
August 9, 2022
NXLog provides native support for Google Chronicle
May 11, 2022
Aggregating macOS logs for SIEM systems
February 17, 2022
How a centralized log collection tool can help your SIEM solutions
April 1, 2020

Categories

  • SIEM
  • STRATEGY
  • SECURITY
  • ANNOUNCEMENT
  • DEPLOYMENT
  • COMPLIANCE
  • COMPARISON
logo

Subscribe to our newsletter to get the latest updates, news, and products releases. 

© Copyright 2024 NXLog FZE.

Privacy Policy. General Terms of Use

Follow us

  • Product
  • NXLog Platform 
  • Log collection
  • Log management and analysis
  • Log storage
  • Integration
  • Professional Services
  • Plans
  • Resources
  • Documentation
  • Blog
  • White papers
  • Videos
  • Webinars
  • Case studies
  • Community Program
  • Community forum
  • Support
  • Getting started guide
  • Support portals
  • About NXLog
  • About us
  • Careers
  • Find a reseller
  • Partner program
  • Contact us