Logging in UTF8 not working

#1 Carsten_Radke

I installed nxlog with the config below, but the logs I receive with my SysLogServer are not in UTF8 ?

The Windowssystem is a german Windows 11, on this machine nxlog ist installed and running.


Here is a sample output on the syslog server ( KiwiSyslog ) :

10-31-2023 15:37:05 User.Info Oct 31 15:37:04 PC-01 MSWinEventLog   1   Microsoft-Windows-PushNotification-Platform/Operational   1469   Tue Oct 31 15:37:04 2023   1025   Microsoft-Windows-PushNotifications-Platform   SYSTEM   User   Information   PC-01   N/A      Ein Stromversorgungsereignis wurde ausgelst: MonitorSettingChange [PowerEventType] true [Enabled]   1846

You can see that there are the german Umlaut are not displayed, ‘ausgelst:’ have to diplay as 'ausgelöst:'


Hope someone can help me.

here is the nxlog.log:Panic Soft#NoFreeOnExit TRUE

define ROOT     C:\Program Files\nxlogdefine CERTDIR  %ROOT%\certdefine CONFDIR  %ROOT%\conf\nxlog.ddefine LOGDIR   %ROOT%\data

include %CONFDIR%\\*.confdefine LOGFILE  %LOGDIR%\nxlog.logLogFile %LOGFILE%

Moduledir %ROOT%\modulesCacheDir  %ROOT%\dataPidfile   %ROOT%\data\nxlog.pidSpoolDir  %ROOT%\data

<Extension _syslog>   Module      xm_syslog</Extension>

<Extension _charconv>   Module      xm_charconv   AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32</Extension>

<Extension _exec>   Module      xm_exec</Extension>

<Extension _fileop>   Module      xm_fileop

   # Check the size of our log file hourly, rotate if larger than 5MB   <Schedule>       Every   1 hour       Exec    if (file_exists('%LOGFILE%') and \                  (file_size('%LOGFILE%') >= 5M)) \                   file_cycle('%LOGFILE%', 8);   </Schedule>

   # Rotate our log file every week on Sunday at midnight   <Schedule>       When    @weekly       Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);   </Schedule></Extension>

# Snare compatible example configuration# Collecting event log<Input in>   Module      im_msvistalog</Input># # Converting events to Snare format and sending them out over TCP syslog<Output out>   Module      om_tcp   Host   Port        514   Exec        to_syslog_snare();</Output># # Connect input 'in' to output 'out'<Route 1>   Path        in => out</Route>


#2 NenadM


There a couple of things that could have caused this:
1. Your Input encoding doesn't match UTF-8: https://docs.nxlog.co/refman/current/xm/charconv.html#config-inputencoding

2. Your German Windows 11 OS settings produce the logs that are not in UTF-8 or im_msvistalog can't parse that kind of characters. There's a nice directive for im_msvistalog called Language but it's available in NXLog EE and CE or just the EE version. Please check: https://docs.nxlog.co/refman/current/im/msvistalog.html#optional-directives

The best thing to do is to diagnose where the problem starts. You can add a test route that sends the logs to a local test file. Something like the module below. Let me know how the logs look like in the file:

<Output file>
    Module  om_file
    File    "C:\somedir\somefile.txt"

If the Language directive like:

Language de-DE 

doesn't work, and the output file is still in UTF-8 …then maybe a regular expression could help. Something that transliterates the Umlauts.