Greetings All,I'm attempting to collect logs from a Meditech server and having a bit of an issue getting the conf file working. Nxlog.log does not show an error but the conf is not writing to the output file.. Typical log that will need to be parsed looks something like this(There is inconsistent spacing in the log):[2024-06-020T04:03:52.593Z maintenance 3 10152 - - Info] Beginning maintenance cycle...And I'm attempting to parse and send it to a file with this:# # Configuration for converting and sending MediTech Logs # #Configure Root Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log #Load Modules <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> ####################################################################### MediTech ####################################################################### Define the input source: MediTech logs <Input MEDITECH_IN> Module im_file File "C:\MediTech\s1.log" &lt;Exec&gt; if $raw_event =~ /(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\]\s+(.*)$/ { $raw_event = "Time: " + $EventTime + "," + "C1: " + $2 + "," + "C2: " + $3 + "," + "C3: " + $4 + "," + "C4: " + $5 + "," + "C5: " + $6 + "," + "C6: " + $7; } &lt;/Exec&gt; </Input> Define the output destination: forwarding to a central syslog server <Output MEDITECH_OUT> Module om_file File "C:\MediTech\s1_exec_out.txt" Exec $Message = $raw_event; </Output> <Route MEDITECH> Path MEDITECH_IN => MEDITECH_OUT </Route>Also, if I'm way off base with the parsing method please let me know.Any assistance would be appreciated.

I'm attempting to parse a Cerberus FTP log file. What I wind up with:{ "EventReceivedTime": "2024-08-01 16:11:37", "SourceModuleName": "cerberus_log", "SourceModuleType": "im_file", "message": "[\u00002\u00000\u00002\u00004\u0000-\u00000\u00008\u0000-\u00000\u00001\u0000 \u00001\u00006\u0000:\u00001\u00001\u0000:\u00003\u00006\u0000]\u0000:\u0000C\u0000O\u0000N\u0000N\u0000E\u0000C\u0000T\u0000 \u0000[\u00001\u00005\u00002\u00004\u00009\u00002\u0000]\u0000 \u0000-\u0000 \u0000C\u0000o\u0000n\u0000n\u0000e\u0000c\u0000t\u0000i\u0000o\u0000n\u0000 \u0000t\u0000e\u0000r\u0000m\u0000i\u0000n\u0000a\u0000t\u0000e\u0000d\u0000" }I've tried this, to no avail:<Input cerberus_log> Module im_file File "C:\ProgramData\Cerberus LLC\Cerberus FTP Server\log\server.1.log" <Exec> $message = convert($raw_event, "utf-8", "iso8859-2"); if $message =~ s/(.)\\u0000// $message = $1; to_json(); </Exec> </Input>How can I properly parse the log to remove the \u0000 characters before it goes out?

We have a distributed solution and a centralised solutionBoth send events to Splunk (I'm the Splunk Admin)100% of the distributed events have the host field present.About 50% of the centralised events have the host field missing and show :Hostname: ?Any idea why this would be? is this a misconfiguration on the centralised host somewhere?  or on the agentless side?Module:SourceModuleName: in_audit_pipe   SourceModuleType: im_pipeThanks.

I'm encountering some errors with a combination of JSON and UDP inputs/processing. After these errors start occurring, the nxlog process eventually gets killed by the system (or maybe crashes? I'm having a hard time determining that at this point.). It appears that nxlog receives some unexpected data via UDP, and then gets into some sort of loop about “map key name not found” until it eventually crashes. Based on the log below, I'm wondering if it has to do with fragmentation of the UDP packets, maybe receiving them out of order and not being reassembled properly before it gets to the point of being processed by the JSON transformer in NXlog?This particular instance is on a Rocky Linux 8 server, with nxlog-ce-3.2.2329. The general setup here is a local source sends JSON messages over UDP to nxlog via 127.0.0.1, and then nxlog handles the secure connection for sending the message to our Graylog server.Here's the applicable portion of the config. (Edit: I have no idea what's going on with the formatting of this code block; I've tried editing it several times, but I can't seem to get it to look normal. I'll just take it out of a code block and put some markers around it.)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<Output gelf>    Module om_ssl    Host <redacted: jinja2 var>    Port 12202    AllowUntrusted True    OutputType GELF_TCP    <Exec>        # These fields are needed for Graylog        $gl2_source_collector = '<redacted: jinja2 var>';        $collector_node_id = '<redacted: jinja2 var>';    </Exec>    <Schedule>        When <redacted: jinja2 var>        Exec reconnect();    </Schedule></Output><Input local_json_udp>    Module im_udp    Host 127.0.0.1    Port 12201    SockBufSize 150000000</Input><Processor local_json_udp_buffer>    Module pm_buffer    MaxSize 4096    Type Mem    WarnLimit 3072</Processor><Processor local_json_udp_transform>    Module pm_transformer    InputFormat JSON</Processor><Route upd_fwd>    Priority 1    Path local_json_udp => local_json_udp_buffer => local_json_udp_transform => gelf</Route>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~This is a relatively old config, so there are some things that have just been copied from previous versions as it's evolved, such as the 150M socket buffer on the UDP input, but only 4K on the internal buffer. Feel free to comment on changes to those, too.Here's the portion of the log from just before where the issue starts through the process crash, with sensitive data redacted. This is from the system journal. I don't believe the log file rotation ins part of the issue, but I included them for timing reference.Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:21:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:22:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:22:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:22:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:22:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:22:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:02 INFO input file '<redacted: partial file path 1>2024-07-16 08:23:57 ERROR failed to parse json string, parse error: premature EOF; {"short_message":"[backtrace] A; (right here) ------^; [{"short_message":"[backtrace] A query is being a memory hog","full_message":"\n\t\t\t\tSELECT <redacted: mysql fields>\n\t\t\t\tFROM <redacted: mysql table>\n\t\t\t\tLEFT JOIN <redacted: mysql join> \nLEFT JOIN <redacted: mysql join> \nLEFT JOIN <redacted: mysql join> \n\t\t\t\tWHERE d.id IN(8654)\n\t\t\t\t\n\t\t\t\t\n\t\t\t\tORDER BY d.title ASC\n\t\t\t\t\n\t\t\t","SourceModuleType":"app_generated_msg","SourceModuleName":"php_graylog","SourceName":"app_pro_memory_log","memory_usage":200308768,"query_instance":"c32d1cc9f5caa42fa04a63761a736e71","backtrace":"[\n {\n \"file\": \"<redacted: truncated json-escaped path> Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR failed to parse json string, parse error: trailing garbage; 6182,\n 6186,\n ; (right here) ------^; [ 6182,\n 6186,\n 6187,\n 6188,\n 6189,\n 6190,\n 6191,\n 6192,\n 6193,\n 6194,\n 6195,\n 6196,\n 6197,\n 6198,\n 6199,\n 6200,\n 6201,\n 6203,\n 6204,\n 6205,\n 6206,\n 6207,\n 6208,\n 6209,\n 6210,\n 6211,\n 6212,\n 6213,\n 6214,\n 6215,\n 6216,\n Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR map key name not found Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 WARNING data in pm_buffer reached 3073 kbytes Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR map key name not found Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 WARNING pm_buffer is full (4100 kbytes)! Jul 16 08:23:57 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR map key name not found Jul 16 08:24:00 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:00 ERROR last message repeated 296744 times Jul 16 08:24:00 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:00 ERROR map key name not found Jul 16 08:24:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 ERROR map key name not found Jul 16 08:24:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 ERROR map key name not found Jul 16 08:24:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 ERROR map key name not found Jul 16 08:24:05 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:05 ERROR last message repeated 323811 times Jul 16 08:24:05 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:05 ERROR map key name not found Jul 16 08:24:08 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:08 ERROR last message repeated 304470 times Jul 16 08:24:08 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:08 ERROR map key name not found Jul 16 08:24:11 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:11 ERROR last message repeated 265201 times Jul 16 08:24:11 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:11 ERROR map key name not found Jul 16 08:24:14 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:14 ERROR last message repeated 254768 times Jul 16 08:24:14 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:14 ERROR map key name not found Jul 16 08:24:17 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:17 ERROR last message repeated 267045 times Jul 16 08:24:17 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:17 ERROR map key name not found Jul 16 08:24:20 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:20 ERROR last message repeated 333813 times Jul 16 08:24:20 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:20 ERROR map key name not found Jul 16 08:24:23 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:23 ERROR last message repeated 293331 times Jul 16 08:24:23 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:23 ERROR map key name not found Jul 16 08:24:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:26 ERROR last message repeated 274896 times Jul 16 08:24:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:26 ERROR map key name not found Jul 16 08:24:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:29 ERROR last message repeated 274099 times Jul 16 08:24:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:29 ERROR map key name not found Jul 16 08:24:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:32 ERROR last message repeated 215508 times Jul 16 08:24:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:32 ERROR map key name not found Jul 16 08:24:35 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:35 ERROR last message repeated 227820 times Jul 16 08:24:35 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:35 ERROR map key name not found Jul 16 08:24:38 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:38 ERROR last message repeated 215476 times Jul 16 08:24:38 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:38 ERROR map key name not found Jul 16 08:24:41 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:41 ERROR last message repeated 249382 times Jul 16 08:24:41 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:41 ERROR map key name not found Jul 16 08:24:44 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:44 ERROR last message repeated 262639 times Jul 16 08:24:44 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:44 ERROR map key name not found Jul 16 08:24:47 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:47 ERROR last message repeated 228055 times Jul 16 08:24:47 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:47 ERROR map key name not found Jul 16 08:24:50 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:50 ERROR last message repeated 244952 times Jul 16 08:24:50 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:50 ERROR map key name not found Jul 16 08:24:53 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:53 ERROR last message repeated 230802 times Jul 16 08:24:53 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:53 ERROR map key name not found Jul 16 08:24:56 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:56 ERROR last message repeated 270445 times Jul 16 08:24:56 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:56 ERROR map key name not found Jul 16 08:24:59 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:59 ERROR last message repeated 276170 times Jul 16 08:24:59 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:59 ERROR map key name not found Jul 16 08:25:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:02 ERROR map key name not found Jul 16 08:25:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:02 ERROR map key name not found Jul 16 08:25:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:02 ERROR map key name not found Jul 16 08:25:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:02 ERROR map key name not found Jul 16 08:25:05 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:05 ERROR last message repeated 218827 times Jul 16 08:25:05 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:05 ERROR map key name not found Jul 16 08:25:08 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:08 ERROR last message repeated 241344 times Jul 16 08:25:08 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:08 ERROR map key name not found Jul 16 08:25:11 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:11 ERROR last message repeated 239566 times Jul 16 08:25:11 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:11 ERROR map key name not found Jul 16 08:25:14 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:14 ERROR last message repeated 216182 times Jul 16 08:25:14 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:14 ERROR map key name not found Jul 16 08:25:17 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:17 ERROR last message repeated 219382 times Jul 16 08:25:17 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:17 ERROR map key name not found Jul 16 08:25:20 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:20 ERROR last message repeated 210542 times Jul 16 08:25:20 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:20 ERROR map key name not found Jul 16 08:25:23 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:23 ERROR last message repeated 219382 times Jul 16 08:25:23 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:23 ERROR map key name not found Jul 16 08:25:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:26 ERROR last message repeated 215193 times Jul 16 08:25:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:26 ERROR map key name not found Jul 16 08:25:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:29 ERROR last message repeated 218181 times Jul 16 08:25:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:29 ERROR map key name not found Jul 16 08:25:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:32 ERROR last message repeated 218710 times Jul 16 08:25:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:32 ERROR map key name not found Jul 16 08:25:35 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:35 ERROR last message repeated 218381 times Jul 16 08:25:35 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:35 ERROR map key name not found Jul 16 08:25:38 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:38 ERROR last message repeated 279278 times Jul 16 08:25:38 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:38 ERROR map key name not found Jul 16 08:25:41 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:41 ERROR last message repeated 254365 times Jul 16 08:25:41 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:41 ERROR map key name not found Jul 16 08:25:44 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:44 ERROR last message repeated 246005 times Jul 16 08:25:44 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:44 ERROR map key name not found Jul 16 08:25:47 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:47 ERROR last message repeated 232794 times Jul 16 08:25:47 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:47 ERROR map key name not found Jul 16 08:25:50 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:50 ERROR last message repeated 256292 times Jul 16 08:25:50 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:50 ERROR map key name not found Jul 16 08:25:53 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:53 ERROR last message repeated 210924 times Jul 16 08:25:53 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:53 ERROR map key name not found Jul 16 08:25:56 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:56 ERROR last message repeated 210623 times Jul 16 08:25:56 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:56 ERROR map key name not found Jul 16 08:25:59 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:59 ERROR last message repeated 248391 times Jul 16 08:25:59 <redacted: hostname> nxlog[846115]: 2024-07-16 08:25:59 ERROR map key name not found Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:26:02 ERROR last message repeated 195119 times Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:26:02 ERROR map key name not found Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:26:02 ERROR map key name not found Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:26:02 ERROR map key name not found Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: <redacted: partial file path 2>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR last message repeated 4494 times Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:23:57 ERROR last message repeated 821 times Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 ERROR last message repeated 195472 times Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 INFO input file '<redacted: file path>' was truncated, restarting from the beginning Jul 16 08:26:02 <redacted: hostname> nxlog[846115]: 2024-07-16 08:24:02 ERROR last message repeated 35 times ... Jul 16 08:27:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:26 ERROR last message repeated 205422 times Jul 16 08:27:26 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:26 ERROR map key name not found Jul 16 08:27:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:29 ERROR last message repeated 214852 times Jul 16 08:27:29 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:29 ERROR map key name not found Jul 16 08:27:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:32 ERROR last message repeated 104496 times Jul 16 08:27:32 <redacted: hostname> nxlog[846115]: 2024-07-16 08:27:32 ERROR map key name not found Jul 16 08:27:36 <redacted: hostname> systemd[1]: nxlog.service: Main process exited, code=killed, status=9/KILL Jul 16 08:27:36 <redacted: hostname> systemd[1]: nxlog.service: Failed with result 'signal'. -- Subject: Unit failed -- Defined-By: systemd -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The unit nxlog.service has entered the 'failed' state with result 'signal'.The log mostly consisted of the the log file rotation messages previous to this portion of it, with the occasional notification of the reconnect. The <redacted: partial file path 2> is the second half of the previous <redacted: partial file path 1>, so everything between them got inserted in the middle of that message being written to the log.The input JSON looks very GELF-esqe, but modified since NXLog CE doesn't support a GELF input. The exact message varies per error instance, but it does seem to be a consistently larger message (I've determined this by finding the origination point of the message from within the code of the local source), hence my suspicion of the underlying source being a limitation in the network stack.Any guidance on what's going on would be great. The “map key name not found” error is pretty vague, and I can't find much about it online or in this forum, but I also think that it's just a side effect to the issues that appear to happen before it.

Hello, i am trying free alternatives for our old syslog software. Before investing a lot of time can you tell me if it is possible to realize the following syslog configuration?If someone has a similar syslog cofiguration and can post an example config or give me some advice that would be really nice.environment:Windows Server 2022NxLog CE 3.2.2329syslog source:cisco asaconfiguration:filename = [source/host IP]_YYYYMMDD_[idx].txt (idx = 001..\d\d\d)automatic file rotation (max file index 250) when event time has another date than the last log file (day change) -> index starts at 001 againorfile gets bigger than 64Mafter rotation moving file to (another) date folder "YYYYMMDD" (or compressing into an date archive file)remarks:the file rotation / size splitting does not have to happen immediatelythe moving/compressing after creating new log file(s) does not have to happen immediately

Hi,We encounter below error during the installation where directory /opt/nxlog was not created and agent not running.

Hi,I am using this code snipping<Input EventLOG>Module im_msvistalogExec if ($TargetUserName == 'SYSTEM') OR ($EventType == 'VERBOSE') drop();</Input>This is working fine with my nx-Client at Windows 11. It push notification at my debian server with installed rsyslog server.But I want to have some specific ID´s from Windows Eventlog.( e.g. 5013, 10016, 4616, 6869)Can anybody point me to the right way? Thanks in Advance for every help. Heinz  

Hello,We are setting up log collection from Azure Log Analytics workspace but the connection is not possible without B2B proxy. I see that Proxy setup is possible only with om_azure module. We need to read and collect the logs from the Azure environment and not to send it there. What should we do to make it happen? Without proxy the connection is not possible and we can't use nxlog for our new service. Is there any other module which could set the proxy by default etc. ? Thank you,Martin W.

We currently have a central logging server for our Windows Servers collecting and forwarding with NXLog to AlienVault.  We have a working config file currently that I would like to modify to be able to send two of the Windows Event ID's that are being collected to our NDR (Vectra).  What is the best route to be able to configure multiple sources and/or outputs? Currently we have two working configs, one for Vectra and one for AlienVault, but I'd like to “combine” them in a way that allows us to be able to send relevant data to their needed sources.AlienVault uses a patterndb.xml file for what events it wants to collect, Vectra just needs two specific event ID's that it calls out in the log file with the below line. It then outputs to an IP.  This seems like it should be pretty straight forward but I'd like to have it sort of configured/figured out before I bring down my AlienVault feed.We are also currently using the Nxlog community edition if that matters.<Input eventlog>   Module im_msvistalog<QueryXML> <QueryList>  <Query Id="0" Path="ForwardedEvents">  <Select Path="ForwardedEvents">*[System[(EventID=4768 or EventID=4769)]]</Select> </Query></QueryList></QueryXML></Input>

Hi, I am trying to use an ssl connection between client and server in "server-side authentication only" mode. I placed the certificates in the respective /conf folder of nxlog server. these are the current configurations: >>> CLIENT <<< <Output to_syslog_server> Module om_ssl Host 10.1.1.1 Port 516 Exec $Message = to_leef(); to_syslog_ietf(); </Output> >>> SERVER <<< <Input in_syslog_ssl> Module im_ssl Host 0.0.0.0 Port 516 CAFile %CERTDIR%/rootCA.pem CertFile %CERTDIR%/central.crt CertKeyFile %CERTDIR%/central.key KeyPass password FlowControl TRUE AllowUntrusted TRUE <Exec> if $raw_event =~ /LEEF/ parse_leef(); else parse_syslog(); </Exec> </Input> but it makes me this Error: 2019-09-06 17:43:26 ERROR remote ssl socket was reset? (SSL_ERROR_SSL with errno=9); End of file found Do you have any ideas to solve this? Thank you Antonio

All,Hopefully an easy question.I am currently collecting Windows event logs on a dedicated forwarding server (using native WEF) in a dedicated event log (named “Forwarded Events”).  I have NxLog installed on this server and logs are being sent properly to my SIEM.Currently I am having difficulties filtering events where the SubjectUserName field ends with “$” symbol (logs are still reaching my SIEM). Below is a snippet of my configuration.  I am running NXLog Enterprise version 6.2.Please advise.<Extension json> Module xm_json </Extension> <Input windows_security_eventlog> Module im_msvistalog ReadFromLast True SavePos True <QueryXML> <QueryList> <Query Id="0"> <Select Path="ForwardedEvents">*</Select> </Query> </QueryList> </QueryXML> <Exec> ################################################### # Drop noisy machine object access (4662) events.# ################################################### if $EventID == 4662 AND ($SubjectUserName =~ /(.)$/) drop(); </Exec> </Input>

I'm trying to configure an Exclude in im_file. I want to exclude logs with a date and number at the end of the filename. These files are already processed.Config:<Input PPS>  Module im_file  InputType multi_PPS  File '%LOGDIR%\\*.log'  # PPSPortaalManager_24-05-24_2.log  Exclude '*_??-??-??_*.log'  #. I've tried several notations, eg. double quotes instead of single, escaping the questionmarks. But none of them work. Searching for examples leads to 1 of 2 examples, which are very common examples, and none of them with wildcards.Using nxlog-ce-3.2.2329 .Has anyone a good, and working, example how to user Exclude in im_file?

Hello all,  In the below code, “CID.*” is the regex that im using but in order to make the code generic I need to use a variable instead of regex, kindly let me know the fix for it or a way to go arround it . Main issue: I need to run “supressed” function for every different log separately such that one log of every type is suppressed.Thanks!!<Processor dup_drop>   Module pm_evcorr <Suppressed>     Condition $raw_event =~ /CID.*/     Interval 90     Exec $MSG = $raw_event; </Suppressed> </Processor>

I have a Powershell script that is retrieving events from Windows Event logs that are written by a certain application.  It then parses the exception info from the Event_Data portion into separate fields and combines them into JSON (the exception info is written in one big block of text, but each line has fields that we want to separate out into distinct fields so that the data is easier to filter in Kibana.  I have an input im_exec module in nxlog.conf that runs that script and a route to send that input to an om_tcp output for our logstash instance.  This works fine when you restart the service, however I'm having a problem getting this to pick up events after the initial restart.  I've tried adding a schedule both to the input module and adding an xm_exec module to restart the input module on a scheduled basis.  Anyone have any ideas on how to get this script to run repeatedly so that i can pick up and format new events?  I've looked through the schedule documentation and tried some of the suggestions on other discussions that look a bit similar, but none seem to have worked so far. This is what I have in the conf file currently:    define ROOT C:\Program Files (x86)\nxlog      Moduledir %ROOT%\modules   CacheDir %ROOT%\data   Pidfile %ROOT%\data\nxlog.pid   SpoolDir %ROOT%\data   LogFile %ROOT%\data\nxlog.log   LogLevel INFO      <Extension json>         Module xm_json   </Extension>   define LogFile %ROOT%\data\nxlog.log   <Input input_logs_powershell>   Module  im_exec   Restart true    Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe"   Arg     "-ExecutionPolicy"   Arg     "Bypass"   Arg     "-NoProfile"   # This specifies the path to the PowerShell script.   Arg     "-File"   Arg     "D:\Temp\events_parser.ps1"<Exec>       # Parse JSON       parse_json();   </Exec></Input> <Output logs_to_Kibana>      Module om_tcp      Host logstash_hostname      Port 6710   exec to_json();   </Output><Route input_logs_powershell>PATH input_logs_powershell =>logs_to_Kibana</Route> 

Hello there,I am experiencing the same reconnection issues on some VMs (Windows Server 2016 10.0.14393), but Nxlog connects successfully to our SIEM syslog sensor over port 601 from other VMs. When I run the netstat command on the affected servers, I see an “ESTABLISHED” connection between the server and the sensor (syslog server). The Windows firewall is disabled on all our servers.From the syslog server, there's a trace of successful packet exchange between the servers and the sensor. But I cannot see the Windows events in our SIEM platform.I have updated Nxlog CE to the latest version. Please help me resolve this issue.Thank you,Delvain

Hi teams, Hope you are doing well. I have some trouble implementing NXLOG on Windows 2016 server. I have this logs into my nxlog.log file : 2020-03-08 18:46:34 INFO reconnecting in 1 seconds 2020-03-08 18:46:35 INFO connecting to 10.7.x.2:12201 2020-03-08 18:46:35 INFO successfully connected to 10.7.x.2:12201 2020-03-08 18:46:35 INFO reconnecting in 1 seconds 2020-03-08 18:46:36 INFO connecting to 10.7.x.2:12201 2020-03-08 18:46:36 INFO successfully connected to 10.7.x.2:12201 2020-03-08 18:46:36 INFO reconnecting in 1 seconds 2020-03-08 18:46:37 INFO connecting to 10.7.x.2:12201 2020-03-08 18:46:37 INFO successfully connected to 10.7.x.2:12201 2020-03-08 18:46:37 INFO reconnecting in 1 seconds 2020-03-08 18:46:38 INFO connecting to 10.7.x.2:12201 2020-03-08 18:46:38 INFO successfully connected to 10.7.x.2:12201 2020-03-08 18:46:38 INFO reconnecting in 1 seconds 2020-03-08 18:46:39 INFO connecting to 10.7.x.2:12201 2020-03-08 18:46:39 INFO successfully connected to 10.7.x.2:12201 2020-03-08 18:46:39 INFO reconnecting in 1 seconds 2020-03-08 18:46:40 INFO connecting to 10.7.x.2:12201 Under my graylog input, i have following sceen : Throughput / Metrics 1 minute average rate: 13 msg/s Network IO: 308.0B 0B (total: 1.2MiB 0B ) Active connections: 0 (4,228 total) Empty messages discarded: 0 We can see, NXlog seems to send log to Graylog, but as i am in TCP connection i have to see at least 1 active connection. This is not the case. Can you help me to troubleshoot this issue please ? I have check Windows firewall, and network firewall. There is no filtering between NXLogs & my graylog server for sure. Many thanks

New Enterprise NxLog customer here…..hopefully any easy question.Today I am ingesting syslog messages from my PaloAlto Panorama instance into a dedicated syslog (Ununtu) server running syslog-ng.  I am using syslog-ng to parse the incoming logs into 3 distinct log files (traffic, threat, and system).   I am then using “logrotate” and “cron" to rotate, gzip, and retain the logs.I figure I have 2 options in terms of the log files themselves now that I am an nxlog customer.Option 1: Keep things as-is (since it is working now) and just use “im_file”.Option 2: Use nxlog to do the same things I am with syslog-ng.  Being new to nxlog, not sure how to best do this.If I want to go with Option #2, does anyone have a working configuration they would be willing to share on how they parsed the incoming syslog messages from Palo Alto into those 3 distinct files (or came up with a better alternative)? Thank you.

Hello NXlog world! I'm having some challenges to pull data from a SQL 2012 (running on W2K16) database using the im-odbc connector. This is my input: > <Input MSSQL_IN> > Module im_odbc > ConnectionString Driver={ODBC Driver 17 for SQL Server}; Server=XXXXXXXX; > Trusted_Connection=yes; DATABASE=XXXXX; > PollInterval 5 > IdType timestamp > SQL SELECT event_time AS 'id', f., a.name AS action_name > FROM fn_get_audit_file('C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\XXXXX\Log\Audit-File.sqlaudit', default, > default) AS f > INNER JOIN sys.dm_audit_actions AS a > ON f.action_id = a.action_id > WHERE event_time > ? > <Exec> > delete($id); > rename_field($event_time, $EventTime); > </Exec> > > </Input> The error I'm getting when starting NXlog is as follows: 2019-10-16 13:51:03 INFO nxlog-4.4.4431 started 2019-10-16 13:51:03 INFO im_odbc successfully connected to the database 2019-10-16 13:51:03 ERROR SQLExecDirect failed, 42000:2:300:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]CONTROL SERVER permission was denied on object 'server', database 'master'.; 42000:3:297:[Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The user does not have permission to perform this action. (odbc error code: -1)` I have tried to enable of the permissions on the database (GRANT) but the CONTROL SERVER is throwing me off. I'm pretty sure is a stupid thing I haven't thought about, but since I am very new to NXlog, and obviously not a SQL admin, I am defeated! for now, so I'm hopeful someone, can help me with some guidance so I can troubleshoot further. TIA and happy hump day! JC

I am considering using NXLog Community Edition for a commercial application and wanted to confirm if this is permissible under the NXLog Public License. Are there any restrictions or limitations I should be aware of when using the Community Edition in a commercial setting? Additionally, could you please highlight the major differences between the Community and Enterprise editions that could impact a commercial deployment? Thank you for your assistance!

Hello team.We have followed the steps from this link: https://docs.nxlog.co/refman/v5.10/im/ms365.html#config-certkeyfile to obtain the logs from Office365.The following permissions have been applied:However, I am encountering the following errors when trying to extract the information:  2024-05-01 18:59:31 WARNING [im_ms365|microsoft_365] Retrying request Audit.Exchange, attempt 33 failed, error: {"error":{"code":"AF20055","message":"Start time and end time must both be specified (or both omitted) and must be less than or equal to 24 hours apart, with the start time prior to end time and start time no more than 7 days in the past. StartTime:2024-05-01T22:48:58Z, EndTime:2024-05-01T21:21:47Z"}}. (Retrying in 200 seconds). 2024-05-01 18:59:34 WARNING [im_ms365|microsoft_365] Retrying request HealthOverviewsWithServiceHealthIssues, attempt 33 failed, error: {"error":{"code":"UnknownError","message":"","innerError":{"date":"2024-05-01T22:59:34","request-id":"20d6e12b-eb61-4b2b-bffa-b69f8f5c4847","client-request-id":"20d6e12b-eb61-4b2b-bffa-b69f8f5c4847"}}}. (Retrying in 200 seconds). 2024-05-01 18:59:36 WARNING [im_ms365|microsoft_365] Retrying request AuditEvents, attempt 33 failed, error: {"error":{"code":"Forbidden","message":"{\r\n \"_version\": 3,\r\n \"Message\": \"Application is not authorized to perform this operation. Application must have one of the following scopes: DeviceManagementApps.Read.All, DeviceManagementApps.ReadWrite.All - Operation ID (for customer support): 00000000-0000-0000-0000-000000000000 - Activity ID: e188947b-1412-4a09-9b64-548de1c1f6a6 - Url: https://fef.amsua0602.manage.microsoft.com/StatelessAuditingFEService/deviceManagement/auditEvents?api-version=5022-08-30&$filter=activityDateTime+ge+2024-04-01T21%3a21%3a47Z+and+activityDateTime+lt+2024-05-01T21%3a21%3a47Z&$top=50\",\r\n \"CustomApiErrorPhrase\": \"\",\r\n \"RetryAfter\": null,\r\n \"ErrorSourceService\": \"\",\r\n \"HttpHeaders\": \"{}\"\r\n}","innerError":{"date":"2024-05-01T22:59:36","request-id":"e188947b-1412-4a09-9b64-548de1c1f6a6","client-request-id":"e188947b-1412-4a09-9b64-548de1c1f6a 2024-05-01 18:59:44 WARNING [im_ms365|microsoft_365] Retrying request SignIns, attempt 33 failed, error: {"error":{"code":"Authentication_RequestFromNonPremiumTenantOrB2CTenant","message":"Neither tenant is B2C or tenant doesn't have premium license","innerError":{"date":"2024-05-01T22:59:44","request-id":"6eb5223c-948f-42af-b28b-bbf3fbea96fb","client-request-id":"6eb5223c-948f-42af-b28b-bbf3fbea96fb"}}}. (Retrying in 200 seconds). 2024-05-01 19:00:05 WARNING [im_ms365|microsoft_365] Retrying request ReportingWebService/MessageTrace, attempt 33 failed, error: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">; <html xmlns="http://www.w3.org/1999/xhtml">; <head>; <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>; <title>401 - Unauthorized: Access is denied due to invalid credentials.</title>; <style type="text/css">; <!--; body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}; fieldset{padding:0 15px 10px 15px;} ; h1{font-size:2.4em;margin:0;color:#FFF;}; h2{font-size:1.7em;margin:0;color:#CC0000;} ; h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} ; #header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;; background-color:#555555;}; #content{margin:0 0 0 2%;position:relative;}; .content-container{background:#FFF;width:96%;margin-top:8px;padding:1 Could you help me understand these errors?