1
answer

Since there's no support for floating point data types in nxlog, given a log entry that contains numbers with decimal points, is the best option to convert them to fixed point integers? For example, given a field $value = "123.45" (a string) extracted from a log line using regex or xm_kvp, if I go directly to_json, I end up with a string in JSON.

I don't see any way to put the value, without quotes, into JSON. Am I correct?

One workaround is to convert the value to a fixed point integer, choosing a specific precision. For example, if I were to choose to always store my values * 100, I could do the following:

if $value =~ /^(-)?([0-9]+)\.?([0-9]+)?$/
{
  if not defined($3) or size($3) == 0 $value = integer($2) * 100;
  else if size($3) == 1 $value = integer($2) * 100 + integer($3) * 10;
  else $value = integer($2) * 100 + integer(substr($3,0,2));
  if $1 == '-' $value = $value * -1;
}

Given $value = "123.45", "123", "123.4567", "123.4", or "123.", this code will assign the correct, 2-place integer value.

Is this the best current approach to converting a string representation of a floating point value to something that will result in a non-quoted value in JSON?

Thank you!

AskedMarch 11, 2019 - 5:12pm

Answer (1)

This is probably close to the best way you could run it until we have support for floating point data types native.

You could extend this a bit though and use xm_perl to do the KVP->JSON conversion as well as manipulating your fields as Perl supports floating point.

AnsweredMarch 11, 2019 - 11:06pm

Comments (4)

  • nimaimalle's picture

    Yes! I got it working much better with Perl... but that was on my Mac. Our production environment runs on Windows, which I don't believe supports xm_perl, right?

    The other thing I wanted to try with xm_perl was s/"([0-9-.]+)"/$1/g on the JSON string, therefore removing all quotes from number-like strings, but can't do it on Windows, and this sort of capture group referencing in regexp is not supported (yet) in nxlog.

    March 11, 2019 - 11:13pm
  • nimaimalle's picture

    Thank you for your help! To clarify, is xm_perl available in Windows Community Edition? I only have the following extension module dlls:
    xm_charconv.dll
    xm_csv.dll
    xm_exec.dll
    xm_fileop.dll
    xm_gelf.dll
    xm_json.dll
    xm_kvp.dll
    xm_multiline.dll
    xm_syslog.dll
    xm_xml.dll

    March 12, 2019 - 12:24am
  • b0ti's picture
    (NXLog)

    The NXLog Community Edition does not provide xm_perl in the windows msi. We have added perl support to the windows version of the NXLog Enterprise Edition recently.

    March 13, 2019 - 9:13pm