87. Microsoft Azure Sentinel

Table of Contents

Azure Sentinel is Microsoft’s security information event management (SIEM), which is offered as service within Azure. Because of its presence within Azure and close integration with other Azure services, Microsoft refers to Azure Sentinel as "a scalable, cloud-native, and security orchestration automated response (SOAR) solution." NXLog can be configured as an agent for Azure Sentinel, collecting and forwarding logs to its Azure Log Analytics workspaces. For more information about Azure Sentinel, see Microsoft’s Azure Sentinel documentation.

NXLog Enterprise Edition can send security logs directly to Microsoft Azure Sentinel using the Microsoft Azure (om_azure) module.

87.1. Prerequisites

In order to send any type of logs to Azure Sentinel from NXLog, a few prerequisites need to be met.

A subscription to Microsoft Azure
An Azure Log Analytics workspace in the Azure portal
Contributor permissions to the subscription in which the Azure Sentinel workspace resides
Enabling Azure Sentinel if it is not already activated

87.2. Azure Sentinel authentication

The first step in preparing to configure om_azure is to retrieve the Workspace ID and either the Primary key or the Secondary key. These keys can be found by navigating in the Azure portal to Log Analytics workspace > Settings > Agents management. The same set of keys can be viewed under either the Windows servers or Linux servers tab.

Log analytics > Workspace > Agents management

The following om_azure directives are required in order to authenticate with Azure Sentinel:

Table 106. Mandatory om_azure directives
Directive	Description	Sample Value
`WorkspaceID`	Your organization’s Workspace ID	`18fb21ab-d8d4-4448-bdf6-3748c9c03135`
`SharedKey`	Either your Primary key or your Secondary key	`VfIQqBoz6fxmnI/E4PKVPza2clH/YAdJ20RnCDwzHCqCMnobYdM1/dD1+KJ6cI6AkR4xPJlTIWI/jfwPU6QHmw==`
`URL`	The Azure endpoint	`https://<WorkspaceID>.ods.opinsights.azure.com/api/logs?api-version=2016-04-01`

When assigning values to directives that are subject to change by a third-party vendor — or that might change from one organizational unit to another — using define directives to establish the constants needed for a module instance is a best practice. In the following partial configuration file, the values for the om_azure directives that will rarely change are defined. The contents of this file will be read by the main nxlog.conf per file inclusion:

azure-defines.conf


  1
2
3
4
5

  define WORKSPACE    DUMMY_WORKSPACE
define SHAREDKEY    DUMMY_SHAREDKEY
define SUBDOMAIN    ods.opinsights.azure.com
define RESOURCE     api/logs
define APIVER       api-version=2016-04-01

Likewise, another partial configuration containing a complete, generic om_azure instance is platform-agnostic — and thus well-suited for automated deployments — will also be read by the main nxlog.conf configuration file per file inclusion:

om-azure.conf


  1
2
3
4
5
6
7
8

  <Output azure>
    Module          om_azure
    URL             https://%WORKSPACE%.%SUBDOMAIN%/%RESOURCE%?%APIVER%
    WorkspaceID     %WORKSPACE%
    SharedKey       %SHAREDKEY%
    TableName       "%TABLE%"
    HTTPSCAFile     %CAFILE%
</Output>

These two partial configuration files together comprise an almost complete configuration for sending events to Azure Sentinel and will be used for all of the following configuration examples. Only two om_azure directives are missing values: %TABLE%, because it will need to change frequently depending on the log source being monitored, and %CAFile%, because it can change depending on the platform where NXLog Enterprise Edition is running. They will be defined within the the main nxlog.conf configuration file immediately before the inclusion of om-azure.conf.

87.3. Gathering and forwarding security events to Azure Sentinel

The following three examples collect security logs from three different platforms:

Since the output instances for all three examples are almost identical, but their log sources differ greatly, the primary focus will be on configuring their input instances. Once the security events have been captured and prepared for sending to Azure Sentinel, the steps for verifying the ingested data are basically the same, aside from their respective table names and queries:

After logging in to Microsoft Azure, navigate to Azure Sentinel and select the Workspace that will receive the forwarded security events.
Under the General heading immediately below Overview click Logs, which will open up the following Azure Queries modal. Close this modal.
Under the Tables tab, expand the Custom Logs heading to reveal the list of table names which should correspond to the value assigned define TABLE in each of the configurations. This value is eventually assigned to the TableName directive in the generic om-azure.conf file.
In the input area beneath the Run button, compose the Kusto queries specific to each table for verifying the successful ingestion of the security events.

87.3.1. Forwarding Linux audit events to Azure Sentinel

In this example, the NXLog Linux Audit System im_linuxaudit module is used as the event source. The xm_resolver module is needed for the ResolveValues directive in the LinuxAudit input instance, where it is used for resolving some of the numeric values to human readable string values.

The LinuxAudit input instance gets its ruleset from an external rules file that includes rules for monitoring changes to DNS zone files. The Exec block contains a call to the core function hostname() since the Linux Audit system does not include a hostname field by default. This creates a new $Hostname field. The to_json() procedure of the xm_json module is also invoked to format the events as JSON records and enrich them with the NXLog core fields.

Right after the external azure-defines.conf is included, TABLE and CAFILE are defined since they have been intentionally excluded from the generic, external om-azure.conf output instance. They will be referenced by the azure output instance once it is loaded via the next include.

The Linux Audit events are then sent to Azure Sentinel where they will be ingested and stored in Sentinel’s set of Custom Tables. In this case, %TABLE% is set to LinuxAudit. Azure automatically appends _CL to all Custom Tables thus the table will appear as LinuxAudit_CL in the list of Custom Tables.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

  <Extension _json>
    Module          xm_json
</Extension>

<Extension _resolver>
    Module          xm_resolver
</Extension>

<Input LinuxAudit>
    Module          im_linuxaudit
    FlowControl     FALSE
    LoadRule        /opt/nxlog/etc/im_linuxaudit.rules
    ResolveValues   TRUE
    <Exec>
        $Hostname = hostname();
        to_json();
    </Exec>
</Input>

define WORKSPACE    DUMMY_WORKSPACE
define SHAREDKEY    DUMMY_SHAREDKEY
define SUBDOMAIN    ods.opinsights.azure.com
define RESOURCE     api/logs
define APIVER       api-version=2016-04-01

define TABLE        LinuxAudit
define CAFILE       /etc/ssl/certs/ca-certificates.crt

<Output azure>
    Module          om_azure
    URL             https://%WORKSPACE%.%SUBDOMAIN%/%RESOURCE%?%APIVER%
    WorkspaceID     %WORKSPACE%
    SharedKey       %SHAREDKEY%
    TableName       "%TABLE%"
    HTTPSCAFile     %CAFILE%
</Output>

<Route r1>
    Path            LinuxAudit => azure
</Route>

A sample Linux Audit event before it is sent to Azure Sentinel

{
  "type": "PATH",
  "time": "2021-04-28T04:04:04.030000+00:00",
  "seq": 2582,
  "item": 0,
  "name": "/etc/bind/zones/db.example.com",
  "inode": 524363,
  "dev": "fc:02",
  "mode": "file,644",
  "ouid": "root",
  "ogid": "bind",
  "rdev": 0,
  "nametype": "NORMAL",
  "cap_fp": 0,
  "cap_fi": 0,
  "cap_fe": 0,
  "cap_fver": 0,
  "cap_frootid": "0",
  "Hostname": "u20server-1",
  "EventReceivedTime": "2021-04-28T04:04:04.048076+00:00",
  "SourceModuleName": "LinuxAudit",
  "SourceModuleType": "im_linuxaudit"
}

Figure 14. Query to list the Linux Audit events of interest

Figure 15. An expanded view of the fields for the same event shown in the JSON sample above

87.3.2. Forwarding Windows DNS Server audit events to Azure Sentinel

In this example, NXLog collects DNS Server logs via ETW from the Microsoft-Windows-DNSServer provider using the Event Tracing for Windows (im_etw) module. Each record is converted to JSON using to_json() procedure of the xm_json module in order to enrich the event log with the NXLog core fields.

The DNS Server audit events are then sent to Azure Sentinel where they will be ingested and stored in Sentinel’s set of Custom Tables. In this case, %TABLE% is set to DNSServerAudit. Azure automatically appends _CL to all Custom Tables thus the table will appear as DNSServerAudit_CL in the list of Custom Tables.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

  <Extension _json>
    Module          xm_json
</Extension>

<Input DNS_Logs>
    Module          im_etw
    Provider        Microsoft-Windows-DNSServer
    Exec            to_json();
</Input>

include  %INSTALLDIR%\conf\azure-defines.conf

define TABLE        DNSServerAudit
define CAFILE       %CERTDIR%\ca-certificates.crt

include  %INSTALLDIR%\conf\om-azure.conf

<Route r1>
    Path            DNS_Logs => azure
</Route>

The following event is typical of Windows DNS Server Audit events captured by the im_etw module. Based on EventId 516 logged for this particular event, it is evident that a DNS zone record in the example.com domain was deleted.

A sample Windows DNS Server audit event before it is sent to Azure Sentinel

{
  "SourceName": "Microsoft-Windows-DNSServer",
  "ProviderGuid": "{EB79061A-A566-4698-9119-3ED2807060E7}",
  "EventId": 516,
  "Version": 0,
  "ChannelID": 17,
  "OpcodeValue": 0,
  "TaskValue": 5,
  "Keywords": "4611686018428436480",
  "EventTime": "2020-09-09T21:34:21.600556-05:00",
  "ExecutionProcessID": 1784,
  "ExecutionThreadID": 4056,
  "EventType": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "Domain": "WIN-FFMCPAJ76HP",
  "AccountName": "Administrator",
  "UserID": "S-1-5-21-1830054504-3820897498-340727717-500",
  "AccountType": "User",
  "Flags": "EXTENDED_INFO|IS_64_BIT_HEADER|PROCESSOR_INDEX (577)",
  "Type": "1",
  "NAME": "u18s1.example.com",
  "TTL": "0",
  "BufferSize": "4",
  "RDATA": "0xC0A80119",
  "Zone": "example.com",
  "ZoneScope": "Default",
  "VirtualizationID": ".",
  "EventReceivedTime": "2020-09-09T21:34:23.634177-05:00",
  "SourceModuleName": "DNS_Logs",
  "SourceModuleType": "im_etw"
}

Querying for events having a value of 516 for EventId yields the following list. The second one has the same NAME as the example above: u18s1.example.com.

Figure 16. Query to list the Windows DNS Server audit events of interest

Figure 17. An expanded view of the fields for the same event shown in the JSON sample above (part 1)

Figure 18. An expanded view of the fields for the same event shown in the JSON sample above (part 2)

87.3.3. Forwarding BSM audit events from macOS to Azure Sentinel

NXLog supports capturing kernel audit events on macOS with the Basic Security Module Auditing (im_bsm) module. This configuration reads BSM audit events directly from the kernel via the (default) /dev/auditpipe device. Since the hostname is not included in these events, the core function hostname() is called to create a new $Hostname field.

Each record is converted to JSON using to_json() procedure of the xm_json module in order to enrich the event log with the NXLog core fields.

The BSM audit events are then sent to Azure Sentinel where they will be ingested and stored in Sentinel’s set of Custom Tables. In this case, %TABLE% is set to BSMmacOS. Azure automatically appends _CL to all Custom Tables thus the table will appear as BSMmacOS_CL in the list of Custom Tables.

nxlog.conf


  1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

  <Extension _json>
    Module          xm_json
</Extension>

<Input BSMmacOS>
    Module          im_bsm
    DeviceFile      /dev/auditpipe
    <Exec>
        $Hostname = hostname();
        to_json();
    </Exec>
</Input>

define WORKSPACE    DUMMY_WORKSPACE
define SHAREDKEY    DUMMY_SHAREDKEY
define SUBDOMAIN    ods.opinsights.azure.com
define RESOURCE     api/logs
define APIVER       api-version=2016-04-01
define TABLE        BSMmacOS
define CAFILE       /etc/ssl/cert.pem

<Output azure>
    Module          om_azure
    URL             https://%WORKSPACE%.%SUBDOMAIN%/%RESOURCE%?%APIVER%
    WorkspaceID     %WORKSPACE%
    SharedKey       %SHAREDKEY%
    TableName       "%TABLE%"
    HTTPSCAFile     %CAFILE%
</Output>

<Route r1>
    Path            BSMmacOS => azure
</Route>

A sample macOS BSM audit event before it is sent to Azure Sentinel

{
  "TokenVersion": "11",
  "EventType": "AUE_ssauthorize",
  "EventName": "SecSrvr AuthEngine",
  "EventModifier": "",
  "EventTime": "2021-04-27 02:49:21",
  "SubjectAuditID": "4294967295",
  "SubjectUID": "root",
  "SubjectGID": "wheel",
  "SubjectRealUID": "root",
  "SubjectRealGID": "wheel",
  "SubjectPID": "5515",
  "SubjectSID": "100019",
  "SubjectTerminal": "",
  "SubjectTerminal.Port": "1:22136",
  "SubjectTerminal.Host": "0.0.0.0",
  "Text": "com.apple.ServiceManagement.daemons.modify",
  "ReturnErrno": "success",
  "ReturnRetval": "0",
  "Identity": "",
  "Identity.SignerType": "1",
  "Identity.SignerId": "com.apple.authd",
  "Identity.SignerIdTruncated": "0",
  "Identity.TeamId": "",
  "Identity.TeamIdTruncated": "0",
  "Identity.CDHash": "0x46e7a142b069906a06813cc80c6b024e0580a656",
  "TrailerCount": "210",
  "EventReceivedTime": "2021-04-27T02:49:21.210462-05:00",
  "SourceModuleName": "BSMmacOS",
  "SourceModuleType": "im_bsm",
  "Hostname": "mm23a"
}

Figure 19. Query to list the macOS BSM audit events of interest

Figure 20. An expanded view of the fields for the same event shown in the JSON sample above (part 1)

Figure 21. An expanded view of the fields for the same event shown in the JSON sample above (part 2)

Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

NXLog 5.3.6720
Apple macOS 11.2.3 (Big Sur)
Ubuntu 20.04 LTS
Windows Server 2019

Last revision: 28 April 2021

You are here

87. Microsoft Azure Sentinel | Log Collection Solutions