Convert and forward Windows Event Log via Syslog for log collection

Share

Convert and forward Windows Event Log via Syslog for log collection

Log collection requires working with many different formats and protocols. Due to architectural and design differences, Windows Event Log does not communicate with Unix-based Syslog out of the box. However, converting Windows Event Log data to Syslog can be very helpful for centralized log collection.

Windows Event Log

The history of Windows Event Log dates back to Microsoft Windows NT in 1993 with the initial introduction of logging on Windows systems. Over time, it has evolved to its current format and features. For example, support for defining the event source has been added. Windows Event Log is a proprietary binary format where the raw event log data can be translated into XML using the Windows Event Log API.

A Windows Event Log event in text format
Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          10/30/2018 3:52:52 PM
Event ID:      4624
Task Category: Logon
Level:         Information
Keywords:      Audit Success
User:          N/A
Computer:      USER-WORKSTATION
Description:   An account was successfully logged on.
A Windows Event Log event in XML format
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Security-Auditing"
              Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
    <EventID>4624</EventID>
    [...]
    <Channel>Security</Channel>
    <Computer>USER-WORKSTATION</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="SubjectUserSid">S-1-5-18</Data>
    [...]
  </EventData>
</Event>

Windows Event Log uses message templates that are similar to format strings. Variable parts—such as the username or IP address—are denoted by the template’s percent sign (%1, %2, etc.). The actual log record only stores the template identifier and the variable fields. The Windows Event Log API renders this data when viewing the logs by substituting the variable field values into the message. This has many benefits, including localization and reducing the storage space required. However, the format becomes more complex, and the processing of event records can have a significant overhead.

Syslog

As the standard choice for Unix-based systems, administrators may have Syslog as part of an arsenal of tools to cover their log collection, aggregation, and management needs. Syslog is common and can be ingested into other systems, stored in files, sent to other Syslog daemons over the network, and more.

BSD Syslog

The original BSD Syslog format was developed in the 1980s. It later became the de facto standard logging system for Unix-based systems and has been implemented across many operating systems and applications. In 2001, the Internet Engineering Task Force (IETF) officially documented the protocol in informational RFC 3164.

BSD Syslog uses a simple format comprised of three basic parts: priority, header, and message. BSD Syslog uses UDP as its transport layer.

BSD Syslog (RFC 3164)
<30>Nov 21 11:40:27 myserver sshd[26459]: Accepted publickey for john from 192.168.1.1 port 41193 ssh2

IETF Syslog

Due to limitations in the BSD Syslog protocol, in 2009, the IETF released RFCs 5424, 5425, and 5426, which document a replacement for the "legacy" BSD Syslog. The newer IETF Syslog provides a higher-precision timestamp with year, optional structured data, TLS transport, and other improvements.

IETF Syslog (RFC 5424)
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] An application event log entry...

Snare Syslog

Snare Syslog is a Snare formatted message used with a Syslog header. Syslog with a Snare-formatted message is a simple way to send Windows Event Log data to many SIEMs.

Snare with Syslog header
<13>Nov 21 11:40:27 myserver MSWinEventLog	0	Security	32	Mon Nov 21 11:40:27 2016	592	Security	Andy	User	Success Audit	MAIN	DetailedTracking	Process ended	Ended process ID: 2455

Other Syslog Formats

Many other log formats use the Syslog header to define a specific message field syntax. Here are a few:

JSON

It is common to encapsulate JSON-formatted log data by adding a Syslog header, as covered in JSON over Syslog. This is an excellent way to send structured data over Syslog and can be used with Windows Event Log.

ArcSight CEF

The HP ArcSight Common Event Format (CEF) uses Syslog for transport. By converting Windows Event Log data to Syslog-encapsulated CEF, it can be sent to ArcSight products.

LEEF

The LEEF log format, used by IBM QRadar Security products, also uses Syslog for transport.

Converting Windows Event Log to Syslog

Unlike Windows Event Log, Syslog stores the actual rendered text instead of using message templates. When Windows Event Log is converted to Syslog, the fields are mapped and concatenated into a Syslog-formatted string as a single line of text. This conversion allows the Windows events to be used with SIEM suites and other software tools that understand the Syslog format.

Example 1. Windows Event Log to Snare

This configuration reads events from the Security channel, converts each event to the Snare format (with a Syslog header), and forwards the log data via TCP.

nxlog.conf
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='0'><Select Path='Security'>*</Select></Query>
        </QueryList>
    </QueryXML>
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.0.1
    Port    514
    Exec    to_syslog_snare();
</Output>

<Route eventlog_to_tcp>
    Path    eventlog => tcp
</Route>
Output Sample
<11>Nov  1 18:52:56 NXLOG-AGENT MSWinEventLog	3	Security	2	Thu Nov 01 18:52:56 2018	4625	Microsoft-Windows-Security-Auditing	N/A	N/A	Failure Audit	NXLOG-AGENT	Logon		An account failed to log on.    Subject:   Security ID:  S-1-0-0   Account Name:  -   Account Domain:  -   Logon ID:  0x0    Logon Type:   3    Account For Which Logon Failed:   Security ID:  S-1-0-0   Account Name:  ADMINISTRATOR   Account Domain:      Failure Information:   Failure Reason:  Unknown user name or bad password.   Status:   0xC000006D   Sub Status:  0xC000006A    Process Information:   Caller Process ID: 0x0   Caller Process Name: -    Network Information:   Workstation Name: -   Source Network Address: XXXX   Source Port:  0    Detailed Authentication Information:   Logon Process:  NtLmSsp    Authentication Package: NTLM   Transited Services: -   Package Name (NTLM only): -   Key Length:  0
Example 2. Windows Event Log to BSD Syslog

This configuration converts Windows Event Log data to the BSD Syslog format.

nxlog.conf
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.0.1
    Port    514
    Exec    to_syslog_bsd();
</Output>
Output Sample
<11>Nov  1 18:19:56 NXLOG-AGENT Microsoft-Windows-Security-Auditing[0x0]: An account failed to log on.    Subject:  	Security ID:		S-1-0-0  	Account Name:		-  	Account Domain:		-  	Logon ID:		0x0    Logon Type:			3    Account For Which Logon Failed:  	Security ID:		S-1-0-0  	Account Name:		TESTUSER  	Account Domain:		    Failure Information:  	Failure Reason:		Unknown user name or bad password.  	Status:			0xC000006D  	Sub Status:		0xC0000064    Process Information:  	Caller Process ID:	0x0  	Caller Process Name:	-    Network Information:  	Workstation Name:	-  	Source Network Address:	XXXX  	Source Port:		0    Detailed Authentication Information:  	Logon Process:		NtLmSsp   	Authentication Package:	NTLM  	Transited Services:	-  	Package Name (NTLM only):	-  	Key Length:		0
Example 3. Windows Event Log to IETF Syslog

Here, events are converted to the IETF Syslog format instead.

nxlog.conf
<Extension _syslog>
    Module  xm_syslog
</Extension>

<Input eventlog>
    Module  im_msvistalog
</Input>

<Output tcp>
    Module  om_tcp
    Host    192.168.0.1
    Port    514
    Exec    to_syslog_ietf();
</Output>
Output Sample
<11>1 2018-11-01T18:50:16.967176+00:00 NXLOG-AGENT Microsoft-Windows-Security-Auditing 0x0 - [NXLOG@14506 Keywords="9227875636482146304" EventType="AUDIT_FAILURE" EventID="4625" ProviderGuid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Version="0" TaskValue="12544" OpcodeValue="0" RecordNumber="133819" ActivityID="{70C9BD71-705D-0000-97BD-C9705D70D401}" ExecutionProcessID="712" ExecutionThreadID="2452" Channel="Security" Category="Logon" Opcode="Info" SubjectUserSid="S-1-0-0" SubjectUserName="-" SubjectDomainName="-" SubjectLogonId="0x0" TargetUserSid="S-1-0-0" TargetUserName="ADMINISTRATOR" Status="0xc000006d" FailureReason="%%2313" SubStatus="0xc000006a" LogonType="3" LogonProcessName="NtLmSsp " AuthenticationPackageName="NTLM" WorkstationName="-" TransmittedServices="-" LmPackageName="-" KeyLength="0" ProcessName="-" IpAddress="XXXX" IpPort="0" EventReceivedTime="2018-11-01 18:50:18" SourceModuleName="eventlog" SourceModuleType="im_msvistalog"] An account failed to log on.    Subject:  	Security ID:		S-1-0-0  	Account Name:		-  	Account Domain:		-  	Logon ID:		0x0    Logon Type:			3    Account For Which Logon Failed:  	Security ID:		S-1-0-0  	Account Name:		ADMINISTRATOR  	Account Domain:		    Failure Information:  	Failure Reason:		Unknown user name or bad password.  	Status:			0xC000006D  	Sub Status:		0xC000006A    Process Information:  	Caller Process ID:	0x0  	Caller Process Name:	-    Network Information:  	Workstation Name:	-  	Source Network Address:	XXXX  	Source Port:		0    Detailed Authentication Information:  	Logon Process:		NtLmSsp   	Authentication Package:	NTLM  	Transited Services:	-  	Package Name (NTLM only):	-  	Key Length:		0

Read more about Windows Event Log and Syslog log collection

Be one step closer to centralized log collection. The NXLog User Guide has more detailed information about setting up your log collection, with dedicated sections on Windows Event Log, Syslog, and many other formats.

NXLog Ltd. develops multi-platform log collection tools that support many different log sources, formats, transports, and integrations. The tools help administrators collect, parse, and forward logs so they can more easily respond to security issues, investigate operational problems, and analyze event data. NXLog distributes the free and open source NXLog Community Edition and offers additional features and support with the NXLog Enterprise Edition.

This document is provided for informational purposes only and is subject to change without notice. Trademarks are the properties of their respective owners.