3
responses

While evaluating the NXLOG enterprise trial edition, we faced a blocker and I need some clarification/help on the same.

We are using NXLOG’s CEF module (xm_cef, xm_json ) which convert’s CEF messages into JSON. It is working properly for most of the cases but giving unexpected output for few of them.

raw CEF message :-
`
CEF:0|Himanshu Arora|Sample1|10.5.011|195|Process Sample|5|abc=Sample Data suser=XY fname= dvc= shost=10.1.1.1 dhost= duser= externalId= app= reason= cs1Label=Affected User cs1= cs2Label=Safe Name cs2=Notification Sample cs3Label=Device Sample cs3= cs4Label=Database cs4= cs5Label="Other info" cs5= cn1Label=Request Id cn1= cn2Label=Ticket Id cn2= msg=
`
JSON output :-
{
"EventReceivedTime": "2019-04-25T13:43:49.483942+05:30",
"SourceModuleName": "cef_input",
"SourceModuleType": "im_file",
"SyslogFacilityValue": 1,
"SyslogFacility": "USER",
"SyslogSeverityValue": 5,
"SyslogSeverity": "NOTICE",
"SeverityValue": 3,
"Severity": "WARNING",
"EventTime": "2019-04-25T13:43:49.483969+05:30",
"Hostname": "himanshu-VirtualBox",
"SourceName": "CEF",
"CEFVersion": 0,
"CEFDeviceVendor": "Himanshu Arora",
"CEFDeviceProduct": "Sample1",
"CEFDeviceVersion": "10.5.011",
"CEFSignatureID": "195",
"CEFName": "Process Sample",
"CEFSeverity": 5,
"abc": "Sample Data",
"suser": "XY",
"fname": "dvc=",
"shost": "10.1.1.1",
"dhost": "duser="
}

If you notice the raw message has some fields called cs1Label,cs2Label,cs2,cn1Label,cn2Label, cn2 . these fields are missing in the JSON output file.
Moreover in JSON the fields "fname" , "dhost" should have had empty value.

I would like to know
1. If this issue exists only in the enterprise trial edition and it will be resolved if we purchase the Enterprise edition ? or is it issue being fixed and will be released soon?
2. Is there a way to include any third party libraries into NXLOG that can convert CEF to JSON.

AskedApril 25, 2019 - 11:09am

Comments (3)

  • Zhengshi's picture
    (NXLog)

    It could be helpful to have NXLog Version, config, and nxlog.log output when posting issues in the future.

    1. Is there a way to include any third party libraries into NXLOG that can convert CEF to JSON.

    NXLog EE supports external Ruby, Python, Perl , and shell scripts in order to process messages.

    1. If this issue exists only in the enterprise trial edition and it will be resolved if we purchase the Enterprise edition ? or is it issue being fixed and will be released soon?

    We have an internal issue for this currently. It affects trial and EE edition. I do not have an ETA as of yet.

  • himanshu.arora's picture

    Thanks for your reply Zhengshi.
    Yes sure, i will add more details for future technical queries.

    For this issue:-
    1.NXlog version `nxlog-4.3.4308-trial`

    2 log output

    `2019-04-25 20:13:03 ERROR procedure 'parse_cef' failed at line 72, character 21 in /opt/nxlog/etc/nxlog.conf. statement execution has been aborted;cannot parse integer "app=", invalid modifier: '='`

    3. nxlog config

    ```
    User nxlog
    Group nxlog
    Panic Soft

    define CERTDIR /opt/nxlog/var/lib/nxlog/cert
    define CONFDIR /opt/nxlog/var/lib/nxlog

    define LOGDIR /opt/nxlog/var/log/nxlog
    define MYLOGFILE %LOGDIR%/nxlog.log

    include %CONFDIR%/log4ensics.conf

    <Extension _syslog>
    Module xm_syslog
    </Extension>

    <Extension _cef>
    Module xm_cef
    </Extension>

    <Extension _json>
    Module xm_json
    </Extension>

    <Extension _fileop>
    Module xm_fileop

    # Check the size of our log file hourly, rotate if larger than 5MB
    <Schedule>
    Every 1 hour
    <Exec>
    if ( file_exists('%MYLOGFILE%') and
    (file_size('%MYLOGFILE%') >= 5M) )
    {
    file_cycle('%MYLOGFILE%', 8);
    }
    </Exec>
    </Schedule>

    <Schedule>
    When @weekly
    Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
    </Schedule>
    </Extension>

    # CEF parsing

    <Input cef_input>
    Module im_file
    File "/home/himanshu/Desktop/cef_msgs/testCEF.txt"
    <Exec>
    parse_syslog();
    parse_cef($Message);
    </Exec>
    </Input>

    <Output json_cef>
    Module om_file
    File "/home/himanshu/Desktop/cef_msgs/testCEFOutput.txt"
    Exec to_json();
    </Output>

    <Route cefroute>
    Path cef_input => json_cef
    </Route>
    ```

Answers (0)