DNS Log Collection and Parsing

DNS log collection and parsing should be part of the log collection strategy of every modern IT infrastructure. There are numerous reasons why you should be concerned enough to collect as well as parse the DNS logs collected, some of which include:

Operations and Support	Parsing DNS server logs can be used to track active DNS clients, while parsing complex and noisy logs can be helpful in troubleshooting support issues.
Performance	Collecting all DNS logs and all the metadata they contain could pose major resource utilization concerns. The collection should be based on parsing logic and rules to only collect what is really needed, therefore minimizing the required resources.
Security	By parsing the logs, you can drill down and find the right data for security issues such as finding signs of an intrusion or an attack, or tracking the source of an attack. It also helps to detect malware from lookups on recently registered, esoteric domain names as well as consistent lookup failures.
Monitoring	Parsing the collected logs could reveal unknown devices that appear on the network and identify critical devices that have not issued any queries within a predefined time span.

For more information on this topic see, The Importance of DNS Logging in Enterprise Security white paper.

Challenges and Opportunities

Hardening IT security through DNS log collection can be challenging due to the following reasons:

The sheer number of DNS logs generated by both DNS servers and DNS clients.
The diversity of the log sources as well as the formats of DNS logs.
The privacy issues associated with metadata since it can contain personally identifiable information.
The resources needed to collect and parse logs from numerous endpoints.

Despite these obstacles, there are good security trade-offs that can be made by collecting DNS logs. The benefits associated with DNS parsing include:

Finding compromise indicators like any unauthorized changes made to the DNS server.
Finding attack indicators to identify a live attack on a system calling to a C2 server, or methods like DNS hijacking.
Finding indicators of post-exploitation—while not a live attack on a system, it can discover cases like DNS spoofing.

Security and DNS Log Parsing

Parsing DNS server logs can provide additional security insights. For instance, known malicious domain lists are available for filtering log files for signs of the presence of such domains. Other use cases include finding requests to C2 servers, detecting potential DoS attacks, DNS hijacking attacks, as well as attacks using DGA (Domain generation algorithm) which generate more opportunities for C2 rendezvous points.

Example: Parsing DNS Logs for NXDOMAIN Responses

An administrator used the dig command which showed a seemingly harmless NXDOMAIN related result:

;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 16694
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

Detection techniques for the above example include reactionary detection which relies on finding contextual metadata in the DNS logs. One such indicator is the parsing of the DNS logs to locate network NXDOMAIN responses generated for cases when there are errors in the request, such as the domain not being registered. It can either be benign, such as an application using DNS anti-spoofing mechanism, or it can be a sign of something more malicious.

DNS Log Collection and Parsing in Practice

Excessive DNS logging can be resource intensive. It can slow down production systems by negatively affecting local storage. Therefore, consolidating DNS logs on a centralized logging server should be considered, while keeping in mind restrictions on logging rates, such as maximum EPS (events per second).

Failing to place adequate measures on parsing could result in an excessive amount of logs that may prove difficult to manage. It might also present the additional challenge of log standardization which is essential to the proper correlation of data.

The following are tips to consider when planning DNS logging:

Architecture	Logs either need to be forwarded to a centralized log server before log enrichment, or log enrichment could happen earlier in the stage of a log collection cycle.
Standardization	DNS logs as such, as they are generated from various sources, need to be standardized. The default log fields may need to be rewritten to be consumed and processed by a SIEM or any other system for event correlation.
Data Protection	DNS logs can contain personally identifiable information. Depending on the industry or jurisdiction, log rollover and archiving policies may need to be implemented. Ideally such compliance concerns should be addressed as part of a cross-team effort with other departments.

DNS Log Monitoring Considerations

Planning the monitoring and collection of DNS logs might not seem straightforward at first sight given all the considerations that must be addressed, but the rewards can be substantial: reducing operating costs, more effectively securing sensitive data, and greater ease in meeting regulatory compliance requirements, just to mention a few.

In many cases, having to collect and process multiple log formats from multiple types of log sources is a daunting task requiring multiple tools running on multiple operating systems. Increased complexity normally equates to an increased budget for the additional resources required.

Implementing the following two strategies is pivotal for effectively monitoring DNS Logs.

Monitor changes to important assets: Administrators need to implement monitoring for changes made to critical DNS configuration files. FIM (File Integrity Monitoring) on files that hold DNS related data must be implemented. Additionally, to preserve the integrity of DNS logs, they too need to be monitored for changes. Furthermore changes to Registry hives on Windows should be supervised as well.
Collect events generated by the DNS servers: Security Administrators need to be able to collect important DNS-related events on Windows using both the Windows Event Log API as well as ETW, which includes Analytical and Debug channels if they are enabled. DNS Audit Logs from Linux DNS servers need to be collected too.

Ideally, once all the requisite DNS events and logs have been collected, they need to be parsed, enriched, and forwarded to a centralized logging server where they can be easily accessed and audited by security analysts.

Next in this series

Now that some of the benefits, challenges, and goals of DNS log collection in general have been established, the next two parts, Part 2 and Part3 of this series will address the various DNS log sources and some approaches for effectively collecting them from different DNS server environments on Windows and Linux systems respectively.