Why is Nxlog not reading the full line from Cerberus FTP log?

Post a different question
6
responses

Hello

I've defined this as an input file in nxlog.conf

<Input cerberus_log>
Module im_file
File "C:\\ProgramData\\Cerberus LLC\\Cerberus FTP Server\\log\\server.1.log"
InputType LineBased
PollInterval 5
<Exec>
log_info("Msg <" + $raw_event + ">");
$date = substr($raw_event,1,10);
$time = substr($raw_event,12,8);
$cmd = substr($raw_event,23,7);
$pid = substr($raw_event,32,6);
$action = substr($raw_event,42);
$Hostname = hostname_fqdn();
$SourceName = "Cerberus FTP Server";
$ProcessID = $pid;
$EventTime = parsedate($date + " " + $time);
$Message = $cmd + ": " + $action;
$SyslogSeverityValue = 6;
$SyslogFacilityValue = 11;
</Exec>
</Input>

When this input is read from the log file

[2018-03-28 09:21:48]: REPLY [ 5445] - 234 Authentication method accepted

[2018-03-28 09:21:48]:CONNECT [ 5445] - SSL connection using TLSv1.2 (ECDHE-RSA-AES256-GCM-SHA384), 256 bit encryption
[2018-03-28 09:21:48]:CONNECT [ 5445] - SSL connection established
[2018-03-28 09:21:48]:COMMAND [ 5445] - USER PandoraManuellt
[2018-03-28 09:21:48]: REPLY [ 5445] - 331 User PandoraManuellt, password please

[2018-03-28 09:21:48]:COMMAND [ 5445] - PASS ***********

the following is logged in nxlog.log

2018-03-28 09:21:51 INFO Msg <[>
2018-03-28 09:21:51 INFO Msg <>
2018-03-28 09:21:51 INFO Msg <[>
2018-03-28 09:21:51 INFO last message repeated 3 times
2018-03-28 09:21:51 INFO Msg <>
2018-03-28 09:21:51 INFO Msg <[>

Empty lines are empty, but when a line that starts with a '[' then $raw_event only contains that character and nothing else. Why?

Mats-Ove

AskedMarch 28, 2018 - 9:36am

Comments (2)

Answer (1)

Perhaps your input is UTF-16/32?

AnsweredMarch 28, 2018 - 12:47pm

Comments (3)

  • matsovef's picture

    For information, this is a Windows Server installation.

    According to Notepad++ the Cerberus log file is encoded in USC-2 BE BOM. I changed to using

    $raw_event = convert($raw_event,'UTF-16BE','UTF-8');

    which improved message convertion but it's still something wrong. The input

    [2018-03-29 11:55:21]:CONNECT [ 6940] - Incoming connection request on FTP interface 6 at 192.168.53.20 rejected from blocked address 80.87.50.170

    results in

    2018-03-29 11:55:21 INFO Msg <[2018-03-29 11:55:21]:CONNECT [ 6940] - Incoming connection request on FTP interface 6 at 192.168.53.20 rejected from blocked address 80.87.50.170??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????>

    after the convertion.

    What does all those '?' characters mean? Is the string filled with NULL values?

    Mats-Ove

    March 29, 2018 - 12:03pm
  • b0ti's picture
    (NXLog)

    UCS-2 and UTF-16 are slightly different. Have you also tried UCS-2 ?

    Also the default LineBased parser treats input as ascii compatible and I assume the trailing ? marks might be a result of the NUL values not getting converted correctly.

    We have recently enhanced xm_charconv in the NXLog EE with the LineReader configuration option that adds better support for reading input in different encodings.

    March 29, 2018 - 1:48pm
  • matsovef's picture

    We are using the Community Edition so the LineReader configuration options are not available.

    I tried with 'UCS-2BE' but that was not recognized by xm_charconv. 'UTF-16BE' is recognized but results in those ? being added.

    Anyway, I found out that I could change the character format used by Cerberus. After changing to 'UTF-8' I started to get lines with correct characters from its log file.

    April 3, 2018 - 12:49pm