Transfer TLS Windows Server 2012R2 DNS logs by nxlog towards ELK pile on debian
Tags:
#1
OncleThorgal
Hello everyone!
I'm new to the forum, so, I appeal to you because I meet a problem in viewing my DNS logs on ELK stack.
Here is my problem: I have Windows Server 2012R2 VM with nxlog above . The configuration file is the following :
define ROOT C:\Program Files (x86)\nxlogdefine CERTDIR %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _json>
Module xm_json
</Extension>
<Input dnslog>
Module im_file
File "C:\\dns-log.log"
InputType LineBased
Exec $Message = $raw_event;
SavePos TRUE
</Input>
<Output out>
Module om_ssl
Host IP_DU_SERVEUR_LOGSTASH
Port PORT_DU_SERVEUR_LOGSTASH
CAFile %CERTDIR%\logstash-forwarder.crt
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Output>
<Route 1>
Path dnslog => out
</Route>
And when I start it :
My ELK stack run on debian. This are config files :
input {
tcp {
codec =>line { charset => CP1252 }
port => PORT_DU_SERVEUR_LOGSTASH
ssl_verify => false
ssl_enable => true
ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
type => "nxlog"
}
tcp {
codec =>line { charset => CP1252 }
port => PORT_DU_SERVEUR_LOGSTASH
ssl_verify => false
ssl_enable => true
ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
type => "nxlog"
}
filter {
if [type] == "nxlog" {
grok {
match => [ "message", "(?<date_n_time_us>%{DATE_US} %{TIME} (?:AM|PM))%{SPACE}%{WORD:dns_thread_id}%{SPACE}%{WORD:dns_context}%{SPACE}%{WORD:dns_internal_packet_identifier}%{SPACE}%{WORD:dns_protocol}%{SPACE}%{WORD:dns_direction}%{SPACE}%{IP:dns_ip}%{SPACE}%{WORD:dns_xid}%{SPACE}(?<dns_query_type>(?:Q|R Q))%{SPACE}[%{NUMBER:dns_flags_hex}%{SPACE}%{WORD:dns_flags_chars}%{SPACE}%{WORD:dns_response_code}]%{SPACE}%{WORD:dns_question_type}%{SPACE}%{GREEDYDATA:dns_question_name}" ]
}
}
}
if [type] == "nxlog" {
grok {
match => [ "message", "(?<date_n_time_us>%{DATE_US} %{TIME} (?:AM|PM))%{SPACE}%{WORD:dns_thread_id}%{SPACE}%{WORD:dns_context}%{SPACE}%{WORD:dns_internal_packet_identifier}%{SPACE}%{WORD:dns_protocol}%{SPACE}%{WORD:dns_direction}%{SPACE}%{IP:dns_ip}%{SPACE}%{WORD:dns_xid}%{SPACE}(?<dns_query_type>(?:Q|R Q))%{SPACE}[%{NUMBER:dns_flags_hex}%{SPACE}%{WORD:dns_flags_chars}%{SPACE}%{WORD:dns_response_code}]%{SPACE}%{WORD:dns_question_type}%{SPACE}%{GREEDYDATA:dns_question_name}" ]
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][nxlog]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
stdout {
codec => rubydebug
}
}
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][nxlog]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
stdout {
codec => rubydebug
}
}
Issue : I can not view my DNS logs on Kibana. Also configure a dashboard . I'm not sure of my configuration files for Logstash , especially the "filter" section and "output". However, when I type the command ngrep INTERFACE -d -t -W byline on my debian, I have queries that appears to be from my WS, so my logs are well received. Could you help me ?
Thank you very much for your time ! And sorry for my english writing...
#1
OncleThorgal
Hello everyone!
I'm new to the forum, so, I appeal to you because I meet a problem in viewing my DNS logs on ELK stack.
Here is my problem: I have Windows Server 2012R2 VM with nxlog above . The configuration file is the following :
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _json>
Module xm_json
</Extension>
<Input dnslog>
Module im_file
File "C:\\dns-log.log"
InputType LineBased
Exec $Message = $raw_event;
SavePos TRUE
</Input>
<Output out>
Module om_ssl
Host IP_DU_SERVEUR_LOGSTASH
Port PORT_DU_SERVEUR_LOGSTASH
CAFile %CERTDIR%\logstash-forwarder.crt
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Output>
<Route 1>
Path dnslog => out
</Route>
And when I start it :
My ELK stack run on debian. This are config files :
input {
tcp {
codec =>line { charset => CP1252 }
port => PORT_DU_SERVEUR_LOGSTASH
ssl_verify => false
ssl_enable => true
ssl_cert => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
type => "nxlog"
}
filter {
if [type] == "nxlog" {
grok {
match => [ "message", "(?<date_n_time_us>%{DATE_US} %{TIME} (?:AM|PM))%{SPACE}%{WORD:dns_thread_id}%{SPACE}%{WORD:dns_context}%{SPACE}%{WORD:dns_internal_packet_identifier}%{SPACE}%{WORD:dns_protocol}%{SPACE}%{WORD:dns_direction}%{SPACE}%{IP:dns_ip}%{SPACE}%{WORD:dns_xid}%{SPACE}(?<dns_query_type>(?:Q|R Q))%{SPACE}[%{NUMBER:dns_flags_hex}%{SPACE}%{WORD:dns_flags_chars}%{SPACE}%{WORD:dns_response_code}]%{SPACE}%{WORD:dns_question_type}%{SPACE}%{GREEDYDATA:dns_question_name}" ]
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
sniffing => true
manage_template => false
index => "%{[@metadata][nxlog]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
stdout {
codec => rubydebug
}
}
Issue : I can not view my DNS logs on Kibana. Also configure a dashboard . I'm not sure of my configuration files for Logstash , especially the "filter" section and "output". However, when I type the command ngrep INTERFACE -d -t -W byline on my debian, I have queries that appears to be from my WS, so my logs are well received. Could you help me ?
Thank you very much for your time ! And sorry for my english writing...
This sounds like rather an ELK issue than NXLog related so this isn't the best forum to ask, though there might be ELK users around who might be able to help.
