Hello to All! I have an issue using the nxlog agent community edition. My output part of the config file looks like this: Module om_udp Host graylog.domain.local Port 12201 OutputType GELF As failover mechanism I use the DNS record for graylog.domain.local. When the primary graylog node is unreachable, the DNS record is updated to the secondary node. For some log sources, the OS is picking up the change almost immediately (there is no cache enable) but the agent doesn't deliver logs to the secondary node until the nxlog service is restarted. Couple of questions: Does the nxlog service keep any DNS cache? If yes, is there any way to bypass it? Any advice on how to fix the issue? Many thanks in advance

Hi, i am using the nxlog-ce-2.9.1716-1_rhel7.x86_64.rpm and i have the following conf <Output graylogout> Module om_udp Host somehost.com Port 12201 OutputType GELF </Output> If i do "host somehost.com" -> It returns 2 IP in different order $>host somehost.com X.X.X.X Y.Y.Y.Y $>host somehost.com Y.Y.Y.Y X.X.X.X But nxlog always send messages to only one host (X.X.X.X). Does nxlog do a DNS request once? if is that the case, how do i balance the traffic?

​ What is the most efficient way to parse Microsoft DNS Server debug logs into something more tidy, say into a CSV or KVP format on the nxlog agent? Consider the following log message: "24/02/2017 16:37:22 09B0 PACKET  0000009657E7BA40 UDP Rcv 10.0.100.15   a490   Q [0001   D   NOERROR] A      (7)example(3)com(0)" First of all, what would be the most efficient way performance-wise to convert this into a CSV or KVP format? And also, is there some other way besides using Exec and replacing parenthesis and numbers in a sed-like manner to get the clean query name? We have tried to use the Exec method before, but we were hitting some serious performance issues on busy DNS servers. I have currently switched on to using the pm_pattern module to drop invalid log lines (the beginning of the log file and empty lines) and I was wondering if there would be some easy way to perform both of the tasks (the formatting and the cleaning) using the pm_pattern module? An example output could look something like the following: datetime=24/02/2017 16:37:22,thread_id=09B0,context=PACKET,packet_id=0000009657E7BA40,protocol=UDP,action=Rcv,remote_ip=10.0.100.15, xid=a490,event_type=-,opcode=Q,flags_hex=0001,is_authorative=-,is_truncated=-,recursion_desired=D,recursion_available=-, response_code=NOERROR,question_type=A,question_name=example.com The empty or "-" values result from fields specified in the DNS debug log format that are not present in the above message (e.g. all possible flags would be "ATDR", and event_type is "-" because "R" marks a response but an empty value (whitespace) marks a query. And of course, if the above even is possible, would it be too resource consuming?

Hello nxlog world, Shamed to say, I've spent entire yesterday trying to figure out how to read Windows DHCP log files and ship the events to ElasticSearch. Problem was with using direct path for folder C:\Windows\System32\dhcp\. Managed to get nxlog to read by sharing the folders (read-only permissions) to the user account used for nxlog service account logon. As the events were not showing in ES, I'm stuck with trying to write the events into another file, in order to confirm that the source files are being read correctly.   OS: Win Srv 2008 R2 Ent nxlog: v 2.9.1347 Here is the nxlog.conf: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log # debugging only: LogLevel DEBUG NoCache TRUE <Input msdhcp> Module im_file File '\\DC5\dhcp\DhcpSrvLog-*.log' SavePos TRUE InputType LineBased Exec if $raw_event =~ /^#/ drop(); Exec $message = $raw_event; </Input> <Input dns> Module im_file File '\\DC5\dns\dns.log' SavePos TRUE InputType LineBased Exec if $raw_event =~ /^#/ drop(); Exec $message = $raw_event; </Input> <Output file_test> Module om_file File 'C:\Program Files (x86)\nxlog\data\test_file_output.txt' # Sync TRUE OutputType LineBased </Output> <Route test> Path msdhcp,dns => file_test </Route> As a result, only DNS events are written in the output file: 21.1.2016. 11:34:00 A6A8 PACKET 0000000003B27E90 UDP Snd 192.168.105.12 3f0d R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0) 21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Rcv 192.168.105.12 3c32 Q [0001 D NOERROR] A (8)PLANKING(3)lab(5)rador(0) 21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Snd 192.168.105.12 3c32 R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0) ..but only the new ones, i.e. as the source DNS log file is being appended. I have tried modifying the SavePos parameter to FALSE of both input modules, but to no avail - same result.   Questions: 1. What would be the correct configuration of global NoCache and module specific SavePos parameters, in order to read and output the complete source file, regardless of prior attempts? 2. What is the reason DHCP logs (using wildcard) are not being read (or at least written in the output), as opposed to the same configuration for DNS logs?     I will provide nxlog debug level log if needed. No visible errors there.   Any help greatly appreciated!