5
responses

Since there's no support for floating point data types in nxlog, given a log entry that contains numbers with decimal points, is the best option to convert them to fixed point integers? For example, given a field $value = "123.45" (a string) extracted from a log line using regex or xm_kvp, if I go directly to_json, I end up with a string in JSON.

I don't see any way to put the value, without quotes, into JSON. Am I correct?

One workaround is to convert the value to a fixed point integer, choosing a specific precision. For example, if I were to choose to always store my values * 100, I could do the following:

if $value =~ /^(-)?([0-9]+)\.?([0-9]+)?$/
{
  if not defined($3) or size($3) == 0 $value = integer($2) * 100;
  else if size($3) == 1 $value = integer($2) * 100 + integer($3) * 10;
  else $value = integer($2) * 100 + integer(substr($3,0,2));
  if $1 == '-' $value = $value * -1;
}

Given $value = "123.45", "123", "123.4567", "123.4", or "123.", this code will assign the correct, 2-place integer value.

Is this the best current approach to converting a string representation of a floating point value to something that will result in a non-quoted value in JSON?

Thank you!

AskedMarch 11, 2019 - 5:12pm

Answer (1)

This is probably close to the best way you could run it until we have support for floating point data types native.

You could extend this a bit though and use xm_perl to do the KVP->JSON conversion as well as manipulating your fields as Perl supports floating point.

Comments (4)

  • nimaimalle's picture

    Yes! I got it working much better with Perl... but that was on my Mac. Our production environment runs on Windows, which I don't believe supports xm_perl, right?

    The other thing I wanted to try with xm_perl was s/"([0-9-.]+)"/$1/g on the JSON string, therefore removing all quotes from number-like strings, but can't do it on Windows, and this sort of capture group referencing in regexp is not supported (yet) in nxlog.

  • Zhengshi's picture
    (NXLog)

    Actually xm_perl is in WIndows! https://nxlog.co/documentation/nxlog-user-guide/available-modules.html#Microsoft_Windows_64bit

    While getting this link though I noticed that the Extension Modules section is outdated. I will get that fixed.

    and this sort of capture group referencing in regexp is not supported (yet) in nxlog.

    Correct. There is an open feature request for this, but for now you are stuck with quotemeta, \Q/\E, or similar in xm_perl.

  • nimaimalle's picture

    Thank you for your help! To clarify, is xm_perl available in Windows Community Edition? I only have the following extension module dlls:
    xm_charconv.dll
    xm_csv.dll
    xm_exec.dll
    xm_fileop.dll
    xm_gelf.dll
    xm_json.dll
    xm_kvp.dll
    xm_multiline.dll
    xm_syslog.dll
    xm_xml.dll

  • b0ti's picture
    (NXLog)

    The NXLog Community Edition does not provide xm_perl in the windows msi. We have added perl support to the windows version of the NXLog Enterprise Edition recently.