Let's Talk
Let's Talk

Transfer TLS Windows Server 2012R2 DNS logs by nxlog towards ELK pile on debian

Tags:
#1 OncleThorgal

Hello everyone!

I'm new to the forum, so, I appeal to you because I meet a problem in viewing my DNS logs on ELK stack.

Here is my problem: I have Windows Server 2012R2 VM with nxlog above . The configuration file is the following :

define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
 
<Extension _json>
    Module      xm_json
</Extension>
 
<Input dnslog>
    Module      im_file
    File        "C:\\dns-log.log"
    InputType    LineBased
    Exec $Message = $raw_event;
    SavePos TRUE
</Input>
 
<Output out>
    Module      om_ssl
    Host        IP_DU_SERVEUR_LOGSTASH
    Port        PORT_DU_SERVEUR_LOGSTASH
    CAFile      %CERTDIR%\logstash-forwarder.crt
    Exec        $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Output>
 
<Route 1>
    Path        dnslog => out
</Route>

And when I start it :
3,5 Ko

My ELK stack run on debian. This are config files :

input {
tcp {
  codec =>line { charset => CP1252 }
         port => PORT_DU_SERVEUR_LOGSTASH
  ssl_verify => false
  ssl_enable => true
  ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"
  ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  type => "nxlog"
}
filter {
if [type] == "nxlog" {
  grok {
   match => [ "message", "(?<date_n_time_us>%{DATE_US} %{TIME} (?:AM|PM))%{SPACE}%{WORD:dns_thread_id}%{SPACE}%{WORD:dns_context}%{SPACE}%{WORD:dns_internal_packet_identifier}%{SPACE}%{WORD:dns_protocol}%{SPACE}%{WORD:dns_direction}%{SPACE}%{IP:dns_ip}%{SPACE}%{WORD:dns_xid}%{SPACE}(?<dns_query_type>(?:Q|R Q))%{SPACE}[%{NUMBER:dns_flags_hex}%{SPACE}%{WORD:dns_flags_chars}%{SPACE}%{WORD:dns_response_code}]%{SPACE}%{WORD:dns_question_type}%{SPACE}%{GREEDYDATA:dns_question_name}" ]
  }
}
}
output {
elasticsearch {
  hosts => ["localhost:9200"]
  sniffing => true
  manage_template => false 
  index => "%{[@metadata][nxlog]}-%{+YYYY.MM.dd}"
  document_type => "%{[@metadata][type]}"
}
stdout {
  codec => rubydebug
}
}

Issue : I can not view my DNS logs on Kibana. Also configure a dashboard . I'm not sure of my configuration files for Logstash , especially the "filter" section and "output". However, when I type the command ngrep INTERFACE -d -t -W byline on my debian, I have queries that appears to be from my WS, so my logs are well received. Could you help me ?

Thank you very much for your time ! And sorry for my english writing...

Permalink
#2 admDeactivated Nxlog ✓ (Last updated )
#1 OncleThorgal
Hello everyone! I'm new to the forum, so, I appeal to you because I meet a problem in viewing my DNS logs on ELK stack. Here is my problem: I have Windows Server 2012R2 VM with nxlog above . The configuration file is the following : define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert   Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log   <Extension _json>     Module      xm_json </Extension>   <Input dnslog>     Module      im_file     File        "C:\\dns-log.log"     InputType    LineBased     Exec $Message = $raw_event;     SavePos TRUE </Input>   <Output out>     Module      om_ssl     Host        IP_DU_SERVEUR_LOGSTASH     Port        PORT_DU_SERVEUR_LOGSTASH     CAFile      %CERTDIR%\logstash-forwarder.crt     Exec        $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); </Output>   <Route 1>     Path        dnslog => out </Route> And when I start it : My ELK stack run on debian. This are config files : input { tcp {   codec =>line { charset => CP1252 }          port => PORT_DU_SERVEUR_LOGSTASH   ssl_verify => false   ssl_enable => true   ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"   ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"   type => "nxlog" } filter { if [type] == "nxlog" {   grok {    match => [ "message", "(?<date_n_time_us>%{DATE_US} %{TIME} (?:AM|PM))%{SPACE}%{WORD:dns_thread_id}%{SPACE}%{WORD:dns_context}%{SPACE}%{WORD:dns_internal_packet_identifier}%{SPACE}%{WORD:dns_protocol}%{SPACE}%{WORD:dns_direction}%{SPACE}%{IP:dns_ip}%{SPACE}%{WORD:dns_xid}%{SPACE}(?<dns_query_type>(?:Q|R Q))%{SPACE}[%{NUMBER:dns_flags_hex}%{SPACE}%{WORD:dns_flags_chars}%{SPACE}%{WORD:dns_response_code}]%{SPACE}%{WORD:dns_question_type}%{SPACE}%{GREEDYDATA:dns_question_name}" ]   } } } output { elasticsearch {   hosts => ["localhost:9200"]   sniffing => true   manage_template => false    index => "%{[@metadata][nxlog]}-%{+YYYY.MM.dd}"   document_type => "%{[@metadata][type]}" } stdout {   codec => rubydebug } } Issue : I can not view my DNS logs on Kibana. Also configure a dashboard . I'm not sure of my configuration files for Logstash , especially the "filter" section and "output". However, when I type the command ngrep INTERFACE -d -t -W byline on my debian, I have queries that appears to be from my WS, so my logs are well received. Could you help me ? Thank you very much for your time ! And sorry for my english writing...

This sounds like rather an ELK issue than NXLog related so this isn't the best forum to ask, though there might be ELK users around who might be able to help.
Login to see more