2
responses

We use NXlog on Windows server to send audit logs from MariaDB to a syslog server. On certain messages I execute a powershell script with exec_async. We discoverred that the powershell.exe subprocesses is not closed correctly which leads to excessive memory use which can be seen by using RamMap and then the Page Table value. In the processes tab a lot of powershell.exe's can be seen with 4KB memory use. With Process Explorer I can see a large number of Handles in the NXlog.exe process. A restart of the nxlog service temporary resolves the issue by cleaning up the (zombie?) powershell.exe processes. I could reproduce the issue with a very simple powershell script with only 1 line: "exit". When I run "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -NonInteractive -File C:\temp\test.ps1" from a cmd.exe it exits normally and the powershell.exe subprocess doesn't stay visible in RamMap.

Could this be a bug? Or should I call powershell.exe differently?

Excerpt of my config:

define ROOT     C:\Program Files (x86)\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf
define LOGDIR   %ROOT%\data
define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _exec>
  Module xm_exec
</Extension>

.....

<Processor discard_messages>
    Module  pm_null
    <Exec>
        # Discard messages with 'keepalive'
        if( $auditlog_object == "SELECT 0 FROM DUAL" ) exec_async("C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe", "-ExecutionPolicy", "ByPass", "-NonInteractive", "-File", "C:\\temp\\test.ps1");
        if( $auditlog_object == "SELECT 0 FROM DUAL" ) drop(); 
    </Exec>
</Processor>

....

<Input mariadb_audit_file>
Module im_file
File 'C:\mariadb\data\server_audit.log'
PollInterval 1
SavePos True
ReadFromLast True
Recursive False
RenameCheck False
    Exec $FileName = file_name(); # Send file name with each message    
</Input>

<Output output_graylog>
    Module om_udp
    Host x.x.x.x
    Port 12201
    OutputType  GELF
    Exec $short_message = $raw_event; # Avoids truncation of the short_message field.
    Exec $Hostname = hostname_fqdn();
</Output>

<Route route-0>
  Path mariadb_audit_file => preprocess => extract_fields => discard_messages => whitelist_queries => output_graylog
</Route>
AskedAugust 11, 2021 - 11:57am

Answer (1)

The issue is not limited to powershell.exe but also occurs at other programs (e.g. cmd.exe). It seems to be an issue in exec_async. When I run the command with exec (withoud _async) the handles don't count upwards. I have made this workaround with a batchfile so the calling is as quick as possible (the powershell script could take up to 10 seconds).

nxlog.conf:

if( $auditlog_object == "SELECT 0 FROM DUAL" ) exec("C:\\Windows\\System32\\cmd.exe", "/C", "C:\\temp\\call_testPS.bat");

call_testPS.bat:

start C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy ByPass -NonInteractive -File C:\temp\test.ps1

Comments (1)

  • raf's picture
    (NXLog)

    Hi Mat,

    Thanks for reporting this issue. It seems that - indeed - there might be a bug.

    Our developers are already on it, checking & fixing.

    Thanks again!
    Raf