116.3. Filtering events

Table of Contents

Systems and services on Windows can generate a large volume of logs, and it is often necessary to collect only a certain portion of those events. There are several ways to implement filtering of events from the Windows Event Log when using the im_msvistalog module.

A specific channel can be specified with the Channel directive to collect all the events written to a single channel.
An XPath query can be given with the QueryXML block (or Query directive). The specified query is then used to subscribe to events. An XPath query can be used to subscribe to multiple channels and/or limit events by various attributes. However, XPath queries have a maximum length, limiting the possibilities for detailed event subscriptions. See XPath filtering below.
A log file can be read by setting the File directive, in which case im_msvistalog will read all events from the file (for example, Security.evtx). This is intended primarily for forensics purposes, such as with nxlog-processor.
After being read from the source, events can be discarded by matching events in an Exec block and discarding them selectively with the drop() procedure.

Subscribing to a restricted set of events with an XPath query can offer a performance advantage because the events are never received by NXLog. However, XPath queries have a maximum length and limited filtering capabilities, so in some cases it is necessary to combine an XPath query with Exec block filtering in an im_msvistalog configuration. For examples, see examples in Event IDs to Monitor.

116.3.1. XPath filtering

XPath queries can be used to subscribe to events matching certain criteria, both in the Event Viewer and with the im_msvistalog QueryXML directive. Windows Event Log supports a subset of XPath 1.0. For more information, see Consuming Events on Microsoft Docs.

The Event Viewer offers the most practical way to write and test query strings. An XPath query can be generated and/or tested by filtering the current log or creating a custom view.

In the Event Viewer, click an event channel to open it, then right-click the channel and choose Filter Current Log from the context menu. Or, click Create Custom View in the context menu. Either way, a dialog box will open and options for basic filtering will be shown in the Filter tab.
Specify the desired criteria. The corresponding XPath query on the XML tab will be updated automatically.
To view the query string, switch to the XML tab. This string can be copied into the im_msvistalog QueryXML directive.
If required, advanced filtering can be done by selecting the Edit query manually checkbox and editing the query. The query can then be tested to be sure it matches the correct events and finally copied to the NXLog configuration with the QueryXML block.

Figure 12. A Custom View Querying the Application Channel for Events With ID 1008

Sometimes it is helpful to use a query with sources that may not be available. In this case, set the TolerateQueryErrors directive to TRUE to ensure that the module will continue to collect logs.

Example 525. Collecting operational events only

Here, NXLog queries the local Windows Event Log for operational events only.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

Example 526. Collecting important system events

This query collects System channel events with levels below 4 (Critical, Error, and Warning).

nxlog.conf


  1
2
3
4
5
6
7
8
9
10

  <Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path='System'>*[System/Level&lt;4]</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

116.3.2. Exec block filtering

NXLog’s built-in filtering capabilities can also be used to filter events, by matching events and using the drop() procedure. Events can be matched against any of the im_msvistalog fields.

Example 527. Filtering Sysmon events in an exec block

This example discards all Sysmon network connection events (event ID 3) regarding HTTP network connections to a particular server and port, and all process creation and termination events (event IDs 1 and 5) for conhost.exe.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

  <Input in>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id="0">
                <Select Path="Microsoft-Windows-Sysmon/Operational">*</Select>
            </Query>
        </QueryList>
    </QueryXML>
    <Exec>
        if ($EventID in (1, 5) and
            $Image == "C:\\Windows\\System32\\conhost.exe") or
           ($EventID == 3 and
            $DestinationPort == 80 and
            $DestinationIp == 10.0.0.1)
           drop();
    </Exec>
</Input>

You are here

116.3. Filtering events | Log Collection Solutions