Random white space characters appearing in the output of a log message


#1 ghillssc01

Hello,

I am using NXLog EE with the im_odbc module to read application logs from an SQL database table.

After writing these logs to a file or forwarding them to a SIEM I seem to see random white space characters in various fields. Is there anyway possible where I can "cut out" this white space so it no longer appears in the log?

For example, in the below there is white space in the USERID field after sa. Example log here:

<13>Jan 4 16:32:56 PAGBSSC1SQL032 2022-01-04 16:32:56 PAGBSSC1SQL032 INFO id="63548" INDEX1="1" PRODNAME=" " CMPNYNAM="ABF plc " USERID="sa " INQYTYPE="2" DATE1="2022-01-04 00:00:00" SECDESC="Successful Attempts to Log In " DEX_ROW_ID="63548"

Here is my NX Log EE config file:

define INSTALLDIR C:\Program Files\nxlog

#ModuleDir %INSTALLDIR%\modules #CacheDir %INSTALLDIR%\data #SpoolDir %INSTALLDIR%\data

define CERTDIR %INSTALLDIR%\cert define CONFDIR %INSTALLDIR%\conf\nxlog.d

Note that these two lines define constants only; the log file location

is ultimately set by the LogFile directive (see below). The

MYLOGFILE define is also used to rotate the log file automatically

(see the _fileop block).

define LOGDIR %INSTALLDIR%\data define MYLOGFILE %LOGDIR%\nxlog.log

If you are not using NXLog Manager, disable the include line

and enable LogLevel and LogFile.

include %CONFDIR%*.conf

LogLevel INFO LogFile %MYLOGFILE%

<Extension _syslog> Module xm_syslog </Extension>

This block rotates %MYLOGFILE% on a schedule. Note that if LogFile

is changed in managed.conf via NXLog Manager, rotation of the new

file should also be configured there.

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    &lt;Exec&gt;
        if ( file_exists('%MYLOGFILE%') and
             (file_size('%MYLOGFILE%') &gt;= 5M) )
        {
             file_cycle('%MYLOGFILE%', 8);
        }
    &lt;/Exec&gt;
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>

<Input odbc> Module im_odbc ConnectionString DSN=NXLog; Driver={ODBC Driver 17 for SQL Server}; Server=PAGBSSC1SQL032;
Trusted_Connection=yes; Database=DYNAMICS IdType integer SQL SELECT DEX_ROW_ID AS id, * FROM DYNAMICS.dbo.SY05000 WHERE DEX_ROW_ID > ? PollInterval 5 Exec delete($id); Exec if not ($raw_event =~ /sa/) drop (); </Input>

<Output udp> Module om_udp Host 10.180.13.28:514 Exec to_syslog_bsd(); </Output>

<Route transfer> Path odbc => udp </Route>

Any help would be greatly appreciated!

TIA

#2 carlos.caro Nxlog ✓
#1 ghillssc01
Hello, I am using NXLog EE with the im_odbc module to read application logs from an SQL database table. After writing these logs to a file or forwarding them to a SIEM I seem to see random white space characters in various fields. Is there anyway possible where I can "cut out" this white space so it no longer appears in the log? For example, in the below there is white space in the USERID field after sa. Example log here: <13>Jan 4 16:32:56 PAGBSSC1SQL032 2022-01-04 16:32:56 PAGBSSC1SQL032 INFO id="63548" INDEX1="1" PRODNAME=" " CMPNYNAM="ABF plc " USERID="sa " INQYTYPE="2" DATE1="2022-01-04 00:00:00" SECDESC="Successful Attempts to Log In " DEX_ROW_ID="63548" Here is my NX Log EE config file: define INSTALLDIR C:\Program Files\nxlog #ModuleDir %INSTALLDIR%\modules #CacheDir %INSTALLDIR%\data #SpoolDir %INSTALLDIR%\data define CERTDIR %INSTALLDIR%\cert define CONFDIR %INSTALLDIR%\conf\nxlog.d Note that these two lines define constants only; the log file location is ultimately set by the LogFile directive (see below). The MYLOGFILE define is also used to rotate the log file automatically (see the _fileop block). define LOGDIR %INSTALLDIR%\data define MYLOGFILE %LOGDIR%\nxlog.log If you are not using NXLog Manager, disable the include line and enable LogLevel and LogFile. include %CONFDIR%*.conf LogLevel INFO LogFile %MYLOGFILE% <Extension _syslog> Module xm_syslog </Extension> This block rotates %MYLOGFILE% on a schedule. Note that if LogFile is changed in managed.conf via NXLog Manager, rotation of the new file should also be configured there. <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour &lt;Exec&gt; if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') &gt;= 5M) ) { file_cycle('%MYLOGFILE%', 8); } &lt;/Exec&gt; &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Input odbc> Module im_odbc ConnectionString DSN=NXLog; Driver={ODBC Driver 17 for SQL Server}; Server=PAGBSSC1SQL032; Trusted_Connection=yes; Database=DYNAMICS IdType integer SQL SELECT DEX_ROW_ID AS id, * FROM DYNAMICS.dbo.SY05000 WHERE DEX_ROW_ID > ? PollInterval 5 Exec delete($id); Exec if not ($raw_event =~ /sa/) drop (); </Input> <Output udp> Module om_udp Host 10.180.13.28:514 Exec to_syslog_bsd(); </Output> <Route transfer> Path odbc => udp </Route> Any help would be greatly appreciated! TIA

Hi Gary,

Since your question refers to the im_odbc input module which is available on NXLog EE only, please consider our Enterprise Support Services, open a ticket and our team will gladly assist you.

Thanks,
Carlos.