Hi team, I'm sure this should be easy, but I'm not having much luck finding the answer elsewhere, can any of you help me? So I have (McAfee Firewall) log entries that look like this: Time: 10/23/2020 08:09:36 AM Event: Traffic IP Address: 172.19.0.113 Description: SNMP SERVICE Path: C:\Windows\System32\snmp.exe Message: Allowed Incoming UDP - Source 172.19.0.113 : (52676) Destination 172.23.25.135 : snmp (161) Matched Rule: Adaptive Rule - snmp.exe I've got this being parsed as a multiline log entry with the following: <Extension 5f917c0781064d07c2e8486a-multiline> Module xm_multiline HeaderLine /^Time:.*/ EndLine /^Matched Rule:.*/ <Input 5f917c0781064d07c2e8486a> Module im_file File 'C:\ProgramData\McAfee\Endpoint Security\Logs\FirewallEventMonitor.log' PollInterval 1 SavePos True ReadFromLast True Recursive True RenameCheck False Exec $FileName = file_name(); # Send file name with each message InputType 5f917c0781064d07c2e8486a-multiline So far so good - I get a multiline message come through, but I'd now like to parse it. So my first step is to split out the individual lines - ideally in the above eample I'd split the above message into 7 fields: 'Time' -> '10/23/2020 08:09:36 AM' 'Event' -> 'Traffic' 'IP Address' -> 172.19.0.113 ...etc So I'm guessing there's an Exec section required and some regex work, but I've not managed to get anything to work yet - has anyone else done anything similar here? thanks in advance, Jim

Hi, I am trying to collect Windows DNS debug logs with Nxlog xm_multiline. I reference below link: Parsing Detailed DNS Logs With Regular Expressions (https://nxlog.co/documentation/nxlog-user-guide/windows-dns-server.html#parsing-detailed) But, Windows DNS Debug Logs includes Traditional Chinese characters, it won't let me combine multiline into one log, What is correct "HEADER_REGEX" that should I use? DNS Debug Logs sample is (I beleive problem is 上午, By the way, 上午=AM and 下午=PM): 2020/3/6 上午 11:58:01 0E80 PACKET 000001D80FE9BD40 UDP Snd 10.0.35.101 a3f5 R Q [8081 DR NOERROR] A (5)e3998(1)d(10)akamaiedge(3)net(0) UDP response info at 000001D80FE9BD40 Socket = 724 Remote addr 10.0.35.101, port 56423 Time Query=283057, Queued=283057, Expire=283060 Buf length = 0x0200 (512) Msg length = 0x0038 (56) Message: XID 0xa3f5 Flags 0x8180 QR 1 (RESPONSE) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 1 Z 0 CD 0 AD 0 RCODE 0 (NOERROR) QCOUNT 1 ACOUNT 1 NSCOUNT 0 ARCOUNT 0 QUESTION SECTION: Offset = 0x000c, RR count = 0 Name "(5)e3998(1)d(10)akamaiedge(3)net(0)" QTYPE A (1) QCLASS 1 ANSWER SECTION: Offset = 0x0028, RR count = 0 Name "C00Ce3998(1)d(10)akamaiedge(3)net(0)" TYPE A (1) CLASS 1 TTL 20 DLEN 4 DATA 96.7.252.200 AUTHORITY SECTION: empty ADDITIONAL SECTION: empty Nxlog configuration sample is: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _charconv> Module xm_charconv AutodetectCharsets BIG-5, utf-8, utf-16, utf-32, iso8859-2 </Extension> <Extension gelf> Module xm_gelf </Extension> define EVENT_REGEX /(?x)(?<Date>\d+(?:/\d+){2})\s (?<Time>\d+(?::\d+){2})\s (?<ThreadId>\w+)\s+ (?<Context>\w+)\s+ (?<InternalPacketIdentifier>[[:xdigit:]]+)\s+ (?<Protocol>\w+)\s+ (?<SendReceiveIndicator>\w+)\s (?<RemoteIP>[[:xdigit:].:]+)\s+ (?<Xid>[[:xdigit:]]+)\s (?<QueryType>\s|R)\s (?<Opcode>[A-Z]|?)\s (?<QFlags>[(.?)])\s+ (?<QuestionType>\w+)\s+ (?<QuestionName>.)\s+ (?<LogInfo>.+)\s+.+=\s (?<Socket>\d+)\s+ Remote\s+ addr\s (?<RemoteAddr>.+),\sport\s (?<PortNum>\d+)\s+Time\sQuery= (?<TimeQuery>\d+),\sQueued= (?<Queued>\d+),\sExpire= (?<Expire>\d+)\s+.+( (?<BufLen>\d+))\s+.+( (?<MsgLen>\d+))\s+Message:\s+ (?<Message>(?s).*)/ define HEADER_REGEX /(?x)(?<Date>\d+(?:/\d+){2})\s (?<AMPM>\x{e4}\x{b8}\x{8a}\x{e5}\x{8d}\x{88})\s (?<Time>\d+(?::\d+){2})\s (?<ThreadId>\w+)\s+ (?<Context>\w+)\s+ (?<InternalPacketIdentifier>[[:xdigit:]]+)\s+ (?<Protocol>\w+)\s+ (?<SendReceiveIndicator>\w+)\s (?<RemoteIP>[[:xdigit:].:]+)\s+ (?<Xid>[[:xdigit:]]+)\s (?<QueryType>\s|R)\s (?<Opcode>[A-Z]|?)\s (?<QFlags>[(.?)])\s+ (?<QuestionType>\w+)\s+ (?<QuestionName>.)/ <Extension multiline> Module xm_multiline HeaderLine %HEADER_REGEX% </Extension> <Input windnsdetaillog> Module im_file File 'C:\dns.log' Exec convert_fields("BIG-5", "utf-8"); InputType multiline Exec if $raw_event =~ /(\d+)/(\d+/\d+)\s(上午\s)(\d+:\d+:\d+\s)((.|\n))/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'AM ' + $5; Exec if $raw_event =~ /(\d+)/(\d+/\d+)\s(下午\s)(\d+:\d+:\d+\s)((.|\n))/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'PM ' + $5; <Exec> if $raw_event =~ %EVENT_REGEX% { $EventTime = parsedate($Date + " " + $Time + " " + $AMPM); delete($Date); delete($Time); } </Exec> </Input> <Input wineventin> Module im_msvistalog </Input> <Output windnsdetaillogout> Module om_tcp Host 192.168.11.3 Port 12198 OutputType GELF_TCP </Output> <Output wineventout> Module om_udp Host 192.168.11.3 Port 12196 OutputType GELF </Output> <Route 1> Path wineventin => wineventout </Route> <Route 2> Path windnsdetaillog => windnsdetaillogout </Route>