The Windows DNS debug log contains valuable information on DNS queries and activity that is especially useful for monitoring and analyzing malicious traffic. This requires some configuration changes for the DNS service in order to enable debug logging.
Here is a short description on how to enable debug logging for the DNS service on windows, this also applies to Windows Server 2008 and later. It is possible to specify the file and path name of the DNS debug log file as well as the maximum size of the file.
Since the main purpose of this post is to provide you with a solution to this very specific, file-based logging problem, please read on. However, if you are interested in a more performance-driven solution to Windows DNS Server event monitoring, fast forward to an alternative, more modern approach that bypasses log files completely, and is much easier to configure.
Windows DNS debug log rollover
DNS debug log is usually configured with a maximum size. When this limit is reached the log file is cleared to start over. Unfortunately this is not a friendly behavior when you need to monitor this data since the log collector might miss some events when the rollover occurs. Some other services are capable of creating a new log file, the DNS service doesn’t. The bigger problem is that the DNS debug log could disappear when it is monitored by log collector tools.
This issue is not NXLog specific and it affects any other tool that opens and reads the log file, for example Splunk users and Trellix SIEM collector users are running into the same issue.
The DNS debug log only disappears if it is monitored, so the conclusion would be to blame the log monitoring tool. The im_file module in NXLog does not delete files and it does not lock log files. Files are opened with READ access only. NXLog and most other log collectors work fine collecting log files being written by most other software.
Some software requests exclusive locking on a log file that it writes, this of course will prevent the file from being opened and monitored. Locking isn’t the problem in this case as the DNS service does not lock the debug log file.
The default behavior of NXLog’s im_file module is to keep the monitored file open. The CloseWhenIdle configuration option can be used to instruct it to close the log file after it’s done reading the file. Unfortunately this does not solve the disappearing DNS log file issue.
Why does the DNS debug log disappear?
We used Windows Sysinternals Process Monitor to check what’s going on.
Below is a screenshot showing the operations from both dns.exe and nxlog.exe.
When the rollover occurs dns.exe creates a backup of the debug log file under C:\Windows\System32\dns\backup\dns.log and then recreates the debug log file by deleting and opening it for read/write.
Unfortunately NXLog is still reading data from the file and holds an open handle thus the delete operation does not complete until NXLog is done reading the file and closes it.
The DNS service tries to create the new file but receives a DELETE PENDING error and this causes the debug log file to disappear.
Solutions
The following workaround will eliminate this problem if you wish to stay with a traditional, file-based approach to monitoring Windows DNS Server events.
> mkdir D:\Data\Logs\DNSServer
> rename-item C:\Windows\system32\dns\backup backup.original
> cmd /c mklink /d C:\Windows\System32\dns\backup D:\Data\Logs\DNSServer`Point your DNS Server debug log to D:\Data\Logs\DNSServer\DNSServer_debug.log.
When the rename operation happens, the target will be redirected back to the same drive the debug log is on, so a rename operation can be used and will prevent the lock conflict and resulting delete.
A more modern approach
A more modern approach to monitoring Windows DNS Server events would be to leverage Microsoft’s Event Tracing for Windows (ETW) capabilities and bypass this issue altogether.
Microsoft offers a native ETW Provider, Microsoft-Windows-DNSServer, especially for this purpose.
It can monitor two categories of Windows DNS Server events: Audit events, which are enabled by default, and Analytic events, which must be explicitly enabled.
With NXLog, setting up Windows DNS logging via ETW Providers has never been easier.
