HiIs it possible to install nxlog-ce-3.1.2319.msi on Windows 2008 R2?Regards.

TEST

Hi, Doing my first steps with NXlog.I have managed to collect all “Security” windows event log and also managed to update the “Version” parameter to my own parameter - Just for a test purposes Now I need to perform 3 tasks Collect all “Security” windows event log - DoneUpdate the “Version” parameter from int to string - Done Update the “Hostname” parameter of specific event ID (for example EventID":4656) to “test”  - Please advice Thank you <Input eventlog>   Module  im_msvistalog   <QueryXML>       <QueryList>           <Query Id='0'>    <Select Path='Security'>*</Select>           </Query>       </QueryList>   </QueryXML><Exec>       $Hostname = "test" ;      # This task should be only for eventID 4656       $Version = string($Version);       to_json();   </Exec></Input> 

I am trying to use the example in https://docs.nxlog.co/ce/current/index.html#om_file for file rotation on Windows (nxlog-ce-3.1.2319).I receive the following errorERROR Couldn't parse Exec block at xxx.conf:104; couldn't parse statement at line 107, character 29 in xxx.conf; module file not found ERROR module 'testfile' has configuration errorsusing this configuration.   The output works fine if I don't use the functions,  so I assume om_file must be loading (by default?).<Output testfile> Module om_file File "E:/nxlog_output/active/nxlog-out.txt" <Exec> # Format output to_json(); # Rotate file based on size, move to staging folder if (file-&gt;file_size() &gt; 10M) { $stagingFolder = 'E:/nxlog_output/staged/'; $newfile = $stagingFolder + 'data_' + strftime(now(), '%Y%m%d%H%M%S') + '.log'; file-&gt;rotate_to($newfile); } &lt;/Exec&gt; </Output> 

Apologies if I'm being dense, but I need some help with navigation of this site.The upper-right search box on this page, (https://nxlog.co/community-forum/) never submits.   Is there another search function I can use?Google search results all point to URLs like https://nxlog.co/question/4970/iis-logs-containing-quotes-are-not-processing,  which return a 404 when I click through.

Hi,I have installed nxlog service (nxlog-ce-3.1.2319.msi) on windows core 2019 machine. I have a config:define EVENT_REGEX /^.*(<EventData>.+<\/EventData>)$/ <Extension xml> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input k8s_containers> Module im_file File "c:\var\log\containers*.log" <Exec> if $raw_event =~ %EVENT_REGEX% { parse_xml($1); } else { drop(); } $log_type = "k8s_container"; $hostname = hostname(); $host_ip = host_ip(); $log_file = file_name(); if $log_file =~ /(.+)_(.+)_(.+)-(.+).log$/ { $k8s_pod = $1; $k8s_namespace = $2; $k8s_container = $3; $k8s_container_id = $4; } to_json(); &lt;/Exec&gt; </Input> <Output file> Module om_file File "c:\k\nxlog.log" </Output> <Route containerlog> Path k8s_containers => file </Route>Everythings work fine, but log line has “log_file”: “unknown”. And because of that I didn't get $k8s_* fields.How should I debug/resolv this issue?

Does nxlog community edition support IBM AIX & SUN Solaris?

Is there a combination of inputs and extensions that can be used to collect RADIUS accounting messages via UDP listener? We use Microsoft NPS today, but could benefit from the forking and advanced parsing of NXLog. We send RADIUS accounting messages from multiple network devices and the differences in data layout are bit too much for NPS.

Forgive my ignorance but I'm looking to use NX Log to capture all windows events under System, Application and Security whether they be Audit, Info, error or critical. Am I correct in my assumption that with no filter's it should collect everything? <Select Path='Application'></Select> <Select Path='Security'></Select> <Select Path='System'></Select> or do I need to specify on single lines each severity level? for example: <Select Path='Application'>[System/Level=4]</Select> <Select Path='Application'>[System/Level=3]</Select> <Select Path='Application'>[System/Level=2]</Select> and so on?

Hi guys, We have NXLog CE 3.0.2272 installed on a server which was originally installed by another user manually. We have tried uninstalling it via the uninstall string MsiExec.exe /X {xxxxx} via ConnectWise which appeared to have uninstalled ok. Since then, we have installed a newer CE version 3.1.2319 however after installation the nxlog service is non-existent. We suspect the uninstallation via string may have broken this. Several attempts using the original installer to repair or uninstall/reboots does not fix this. Is there a way we can start fresh to remove NXLog completely then install? Any help would be appreciated :) TIA Jordan

Hellow everyone! I have a scenario that uses Citrix MCS where I installed the agent on the master image that provides clone images that should go with the nxlog agent installed and running. But the agent goes up with some errors as below: 2022-09-23 13:51:38 ERROR couldn't connect to udp socket on <IP:XYZ:514>; The socket operation was attempted to an unreachable network. 2022-09-23 13:51:46 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-09-23 13:51:46 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: Setup WitnessClientAdmin 2022-09-23 13:52:14 WARNING received a system shutdown request 2022-09-23 13:52:14 WARNING stopping nxlog service 2022-09-23 13:52:14 WARNING nxlog-ce received a termination request signal, exiting... 2022-11-02 23:16:38 INFO nxlog-ce-2.11.2190 started 2022-11-02 23:16:44 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-11-02 23:16:44 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: Setup WitnessClientAdmin 2022-11-02 23:27:15 ERROR EvtNext failed with error 15007: The specified channel could not be found. Check channel configuration. 2022-11-02 23:27:16 WARNING Due to the limitation in the Windows EventLog subsystem, the query cannot contain more than 256 sources. 2022-11-02 23:27:16 WARNING The following sources are omitted to avoid exceeding the limit in the generated query: WitnessClientAdmin 2022-11-02 23:27:16 ERROR Failed to retrieve eventlog fields; The handle is invalid. Has anyone had a problem like this using Citrix MCS? Thanks James \0/

Hello! We have permanent memory leaks on Windows Event Collect server with any 5 version NXLog. If we install any 4 version - it work without memory leak, but very slowly - it's accumulating queue on single filter for windows events. How we can help to fix it in next release?

Hi, I use NXLog to send log of an Oracle Database to Graylog. When i send the log to a INPUT in Graylog, the logs in Graylog are not in the same order as the source logfile. I have configured an output to send event in an other logfile and in the new log the event are ine the same order as the source logfile. Configuration to the UDP Output Graylog where the event are not in the same order as the source logfile. <Extension gelf> Module xm_gelf ShortMessageLength 200 </Extension> <Input ora-database-rman> Module im_file File "/oracle/rman/logs/DATABASE_*.log" </Input> <Output ext-graylog> Module om_udp Host XX.XX.XX.XX Port XXXXX OutputType GELF_UDP </Output> <Route database-rman Path ora-database-rman => ext-graylog </Route> Configuration to the om_file module loca where the event are in the same order as the source logfile. <Input ora-database-rman> Module im_file File "/oracle/rman/logs/DATABASE_*.log" Exec sleep(100); </Input> <Output fileout> File 'tmp/output' Module om_file </Output> <Route database-rman Path ora-database-rman => fileout </Route> Do you have an idea how to get the event in the order to the OUTPUT TCP ? Thanks for your help, Matt

Hello all, I have an application that sends log files to a directory formatted at YYYY-MM-DD.log (year, month, day). I'm watching the directory with the following stanza in the configuration file, but it does not recognize when the date changes and a new file is created. A service restart gets it reading the new file. The configuration is as follows: <Input cvdupdate> Module im_file File "/var/log/cvdupdate/*.log" </Input> I'm running NXLog on a Ubuntu 18.04 system. The version is 3.0.2272.

Hi, I'm using NXLOG Community to transfer logs in and out, from Windows clients to a Linux server with an NXLOG agent for log collection. The logs arrive correctly, the only thing is that for the event viewer (example) 4624, I see logs with the same time, even in milliseconds, but the message varies only for a few lines of the "message" field. I wanted to avoid using the repeat module because I would create the same a log recording increasing the database. This is client configuration file : define ROOT C:\Program Files\nxlog define ROOT_STRING C:\Program Files\nxlog define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log define MonitoredEventIds 4624, 4647 <Extension _syslog> Module xm_syslog </Extension> <Extension json> Module xm_json </Extension> <Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path="Security">*</Select> </Query> </QueryList> </QueryXML> Exec if $EventID NOT IN (%MonitoredEventIds%) drop(); Exec if $TargetUserName == "SYSTEM" drop(); Exec if $TargetUserName =~ /$/ drop(); Exec if $TargetUserName =~ /UMFD/ drop(); Exec if $TargetUserName =~ /DWM/ drop(); Exec if $LogonType == "5" drop(); </Input> <Output out> Module om_tcp Host (10.*****) Port 1514 Exec to_json(); </Output> <Route eventlog_to_out> this is server nxlog.conf User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules # ######################################## <Extension _syslog> Module xm_syslog </Extension> <Extension json> Module xm_json </Extension> <Input in1> Module im_tcp Host 0.0.0.0 Port 1514 <Exec> parse_json(); </Exec> </Input> <Output dbi> Module om_dbi SQL INSERT INTO SystemEvents (ReceivedAt, DeviceReportedTime, EventID, EventUser, EventSource, EventLogType, FromHost, NTSeverity, Priority, Message) VALUES (NOW(), NOW(), $EventID, $TargetUserName, $SourceName, $EventType, $Hostname, $Severity, $SeverityValue, $Message ) Driver mysql Option host 127.0.0.1 Option username **** Option password ******* Option dbname Syslog </Output> ######################################## # Routes # ######################################## <Route 1> Path in1 => dbi </Route> it's possible to not register duplicates at the origin? it's possible to delete duplicates also in mysql database? thanks you

#Hello, tanks in advance . #I am sending multiple logs from windows server to a linux collector #I have no issues with windows system logs , #Seems i can not send via snare windows system logs, and test plain text logs. #Is there any way to do that? But when i switch to snare i can see no description about the warning 2022-10-21T09:21:21+00:00 Winserver MSWinEventLog#0111#011N/A#0111#011Fri Oct 21 09:21:21 2022#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011N/A#011#011N/A#011N/A#015 The same line with snare commented: 2022-10-21T09:18:23.208210+00:00 Winserver WARNING: Can't open file \?\C:...\UPPS\UPPS.BIN: Permission denied#015 #My config: Panic Soft #NoFreeOnExit TRUE define ROOT C:\App\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Input internal> Module im_internal Exec $Hostname = hostname_fqdn(); </Input> <Input vistalog> Module im_msvistalog </Input> <Input eventlog> Module im_mseventlog </Input> <Input testFile> Module im_file SavePos True RenameCheck True Recursive True PollInterval 0.5 #near real time File "C:\test\myfile.txt" ReadFromLast True </Input> <Output out> Module om_tcp Host linux Port 514 #Exec to_syslog_snare(); </Output> <Route r> Path internal, eventlog, vistalog, testFile => out Path testFile => out </Route>

Hi, i have below replace function for replacing "|0" to "Zero". Exec $Message = replace($Message, "|0 ", "Zero"); now, i want to replace "|0" to "|15" with Zero. do i need to add 15 more Exec replace function, or is there is any way to replace using single exec using regex?

Hi all, I think there might be a bug in the im_odbc module in the Linux (Debian11) im_odbc module for NXLog 5.6.7727 EE when used with the FreeTDS driver (via unixODBC) - so for example if you try to connect to MS SQL or Sybase databases. Although all the drivers etc. were configured correctly (it was basically a copy from a system where it used to work) and connecting to the databases via ISQL worked fine, the Agent wasn't able to fetch logs and produced following errors in the nxlog.log: INFO [im_odbc|inputxy] im_odbc successfully connected to the database WARNING [im_odbc|inputxy] im_odbc detected a disconnection, attempting to reconnect in 1200 seconds ERROR [im_odbc|inputxy] SQLDescribeParam failed, IM001:2:0:[unixODBC][Driver Manager]Driver does not support this function (odbc error code: -1) I've tested multiple FreeTDS driver versions with multiple compile options - but none of them worked. Finally I figured to try a different NXLog version, so I downloaded the NXLog Agent EE 4.10.5000 for debian10 and it worked out of the box with the standard FreeTDS driver from the Debian11 repo (v1.2.3) and unixODBC (2.3.6). So my conclusion is that there is a bug in the 5.6 EE NXLog Agent Version regarding the im_odbc module when used with FreeTDS (at least for Debian 11). So while using 4.10 as a fallback is nice, it would be great to use the newer 5.x versions in the future. BR Reinhard

Hello, I am having stability issues with the newest nxlog-ce release 3.0.2284. I have been using nxlog-ce-2.10.2150 for several years and it has been very stable in our environment with few issues. I use papertrail for log collection, and I have a highly customized configuration file. I did a test roll-out of 3.0.2284 to a few servers and did not notice any issues at first. However after rolling out the update to approximately 40 servers it started crashing randomly in ntdll.dll causing the nxlog service to stop and re-start itself. There was no rhyme or reason to it. It would work fine for 15 minutes and then suddenly I would start getting multiple random crashes and service restarts which would then crush papertrail with the previous 30 days of event log history (per crashed server) and my papertrail storage utilization doubled my average daily usage in a few hours. I had to roll-back all the servers back to 2.10.2150 to stop the bleeding. The servers used in the test were a variety of Windows 2012r2, 2016, 2019, and 2022 servers. By far the majority of the servers are running server 2016. Some are hyper-v hosts running on bare metal, others are virtual machines that run on those hyper-v hosts. My papertrail logs are full of these errors, but here is a small sampling. These are server 2019 and 2022 respectively. Oct 10 21:31:21 hv-host19-f4 Application-Error { "Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:libssl-1_1-x64.dll, version:1.1.1.13, time stamp:0x00000000|Exception code:0xc0000005|Fault offset:0x0000000000021b97|Faulting process id:0xf5d0|Faulting application start time:0x01d8dd0fc8bd493e|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:c:\\apps\\nxlog\\libssl-1_1-x64.dll|Report Id:08b0fd02-0e57-44b0-81fc-b1e7fb47f472|Faulting package full name:|Faulting package-relative application ID:", "Hostname": "hv-host19-f4", "EventType": "ERROR", "SeverityValue": 4, "Severity": "ERROR", "EventID": 1000, "SourceName": "Application-Error", "Task": 100, "RecordNumber": 12236, "ProcessID": 0, "ThreadID": 0, "Channel": "Application", "EventTime": "2022-10-10 21:23:29", "Category": "Application Crashing Events", "Opcode": "Info" } Oct 10 21:52:21 ws-ops22-2 nxlog-ce nxlog-ce-3.0.2284 startup profile 2022.01.25 (DEFAULT) Oct 10 21:52:22 ws-ops22-2 nxlog-ce connecting to logs99.papertrailapp.com:12345 Oct 10 21:52:22 ws-ops22-2 nxlog-ce successfully connected to logs99.papertrailapp.com:12345 Oct 10 21:52:23 ws-ops22-2 Application-Error { "Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:ntdll.dll, version:10.0.20348.803, time stamp:0xbee6f04c|Exception code:0xc0000374|Fault offset:0x00000000001044a9|Faulting process id:0x26f4|Faulting application start time:0x01d8ca290a171970|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:C:\\WINDOWS\\SYSTEM32\\ntdll.dll|Report Id:5e5549b8-3c8b-405a-a78f-fd4c1f296a40|Faulting package full name:|Faulting package-relative application ID:", "Hostname": "ws-ops22-2", "EventType": "ERROR", "SeverityValue": 4, "Severity": "ERROR", "EventID": 1000, "SourceName": "Application-Error", "Version": 0, "Task": 100, "OpcodeValue": 0, "RecordNumber": 1100, "ProcessID": 0, "ThreadID": 0, "Channel": "Application", "EventTime": "2022-09-16 21:30:36", "Category": "Application Crashing Events", "Opcode": "Info" } Oct 10 21:54:35 ws-ops22-2 Application-Error { "Message": "Faulting application name:nxlog.exe, version:0.0.0.0, time stamp:0x00000000|Faulting module name:libcrypto-1_1-x64.dll, version:1.1.1.13, time stamp:0x00000000|Exception code:0xc0000005|Fault offset:0x00000000001ba014|Faulting process id:0x30e8|Faulting application start time:0x01d8caef9eba1346|Faulting application path:c:\\apps\\nxlog\\nxlog.exe|Faulting module path:c:\\apps\\nxlog\\libcrypto-1_1-x64.dll|Report Id:12281218-b154-47ae-a426-1495de2adf0d|Faulting package full name:|Faulting package-relative application ID:", "Hostname": "ws-ops22-2", "EventType": "ERROR", "SeverityValue": 4, "Severity": "ERROR", "EventID": 1000, "SourceName": "Application-Error", "Version": 0, "Task": 100, "OpcodeValue": 0, "RecordNumber": 1304, "ProcessID": 0, "ThreadID": 0, "Channel": "Application", "EventTime": "2022-09-17 20:00:44", "Category": "Application Crashing Events", "Opcode": "Info" } The crashing seems to indicate an issue with TLS or crypto but my existing papertrail configuration has been working fine for literally years. Another issue I ran into, while removing nxlog-ce-3.0.2284 is that issuing a stop-service to command to the service returns "The pipe has been ended" error instead of a normal service shutdown gracefully message. This happened every time I tried to stop the service. The service did stop, but given the error I don't know if it was a graceful stop or if it was a hard stop that ends up causing the eventlog to be re-uploaded in its entirety when the service started again. I had a lot of that going on so I can't say for sure if it happened or not. [SC] ControlService FAILED 109: The pipe has been ended. Finally here is a snippet of the bottom of my nxlog.conf file where I set up the connection to papertrail. I've changed the host parameters slightly for security. <Route nxlog> Path from_nxlog => to_papertrail </Route> <Route eventlogs> Path from_eventlog => noisefilter => cleanup => reorder => jsonify => to_papertrail </Route> <Route c_logs> Path from_c_logs => to_papertrail </Route> <Output to_papertrail> Module om_ssl Host logs99.papertrailapp.com Port 12345 CAFile %ROOT%/cert/papertrail-bundle.pem AllowUntrusted FALSE # Convert to syslog format Exec to_syslog_bsd(); </Output> I'm considering pushing the logs to a local linux server with om_udp and let that server relay the logs to papertrail over TLS to workaround the issue but that adds extra complexity to the environment that I would rather not have to support. Thanks Ron

Hi, I'm experimenting with reading from an Azure eventhub with im_kafka. The eventhub receives security data from various security related azure components. The im_kafka module works great after i found out that the username should be $connectionstring ;). The output of the eventhub is a json dict with an array, like this: { records: [ {id: 1, msg: "xyz", etc},{id: 2, msg: "abc", etc}]}. I tried to use extract_json("$.records") but that does not iterate over the array. I also made a python script that writes the logs to a file, one line at a time: from confluent_kafka import Consumer import json c = Consumer({....}) while True: msg = c.poll(1.0) eventhub_records = json.loads(msg.value()) for record in eventhub_records['records']: print(json.dumps(record)) This works great, but i'd like to have something like this in nxlog. Can this be done, or does nxlog not support to split a single record into multiple records? Thanks!