7
responses

I am trying to use NXlog CE to forward Exchange 2016 message tracking logs via syslog. I am trying to use the example shown here

https://nxlog.co/documentation/nxlog-user-guide#exchange_transport_logs

When I create the conf file and attempt to run it the service starts but there are several errors logged in the nxlog.log file.

My config file is as follows

Panic Soft
#NoFreeOnExit TRUE

define ROOT     C:\Program Files (x86)\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf
define LOGDIR   %ROOT%\data
define LOGFILE  %LOGDIR%\nxlog.log
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension syslog>
 Module xm_syslog
</Extension>

define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15

<Extension csv_parser>
    Module      xm_csv
    Fields      date-time, client-ip, client-hostname, server-ip, server-hostname, \
                source-context, connector-id, source, event-id, \
                internal-message-id, message-id, network-message-id, \
                recipient-address, recipient-status, total-bytes, recipient-count, \
                related-recipient-address, reference, message-subject, \
                sender-address, return-path, message-info, directionality, \
                tenant-id, original-client-ip, original-server-ip, custom-data, \
                transport-traffic-type, log-id, schema-version
</Extension>



<Input messagetracking>
    Module      im_file
    File        '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
    <Exec>
        if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop();
        else
        {
            csv_parser->parse_csv();
            $EventTime = parsedate(${date-time});
        }
    </Exec>
</Input>


<Output out1>
 Module om_udp
 Host 10.1.1.1
 Port 514
 Exec to_syslog_snare();
</Output>


<Route 1>
 Path messagetracking => out1
</Route>

The errors logged in the nxlog.log file are as follows.

2018-07-12 18:06:10 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:39; couldn't parse statement at line 44, character 36 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; invalid character: '$' (0x24)
2018-07-12 18:06:10 ERROR module 'messagetracking' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:59
2018-07-12 18:06:10 ERROR route 1 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:59
2018-07-12 18:06:10 WARNING no routes defined!
2018-07-12 18:06:10 WARNING not starting unused module messagetracking
2018-07-12 18:06:10 WARNING not starting unused module out1
2018-07-12 18:06:10 INFO nxlog-ce-2.10.2102 started

I have yet to come across a working example of how to forward exchange logs and was hoping someone may be able to assist. I cant see what i am missing here.

AskedJuly 13, 2018 - 12:12am

Comments (7)

  • Zhengshi's picture
    (NXLog)

    Your current problem lies in the ${date-time} portion. While this is supported in the EE version, it is not in the CE version as of yet.
    Hyphens (-) are not supported in field names currently, I would rename them to date_time and then call them as $date_time.

  • jdalyasc's picture

    Thank you for your response i tried updating the code as you mentioned and it still appears to not function.

    Panic Soft
    #NoFreeOnExit TRUE
    
    define ROOT     C:\Program Files (x86)\nxlog
    define CERTDIR  %ROOT%\cert
    define CONFDIR  %ROOT%\conf
    define LOGDIR   %ROOT%\data
    define LOGFILE  %LOGDIR%\nxlog.log
    LogFile %LOGFILE%
    
    Moduledir %ROOT%\modules
    CacheDir  %ROOT%\data
    Pidfile   %ROOT%\data\nxlog.pid
    SpoolDir  %ROOT%\data
    
    <Extension syslog>
     Module xm_syslog
    </Extension>
    
    define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15
    
    <Extension csv_parser>
        Module      xm_csv
        Fields      date_time, client_ip, client_hostname, server_ip, server_hostname, \
                    source_context, connector_id, source, event_id, \
                    internal_message_id, message_id, network_message_id, \
                    recipient_address, recipient_status, total_bytes, recipient_count, \
                    related_recipient_address, reference, message_subject, \
                    sender_address, return_path, message_info, directionality, \
                    tenant_id, original_client_ip, original_server_ip, custom_data, \
                    transport_traffic_type, log_id, schema_version
    </Extension>
    
    
    
    <Input messagetracking>
        Module      im_file
        File        '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
        <Exec>
            if $raw_event =~ /^(\xEF\xBB\xBF)?(date_time,|#)/ drop();
            else
            {
                csv_parser->parse_csv();
                $EventTime = parsedate(${date_time});
            }
        </Exec>
    </Input>
    
    
    <Output out1>
     Module om_udp
     Host 10.1.1.1
     Port 514
     Exec to_syslog_snare();
    </Output>
    
    
    <Route 1>
     Path messagetracking => out1
    </Route>
    
    
    2018-07-16 12:07:50 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:39; couldn't parse statement at line 44, character 36 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; invalid character: '$' (0x24)
    2018-07-16 12:07:50 ERROR module 'messagetracking' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:59
    2018-07-16 12:07:50 ERROR route 1 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:59
    2018-07-16 12:07:50 WARNING no routes defined!
    2018-07-16 12:07:50 WARNING not starting unused module messagetracking
    2018-07-16 12:07:50 WARNING not starting unused module out1
    2018-07-16 12:07:50 INFO nxlog-ce-2.9.1716 started
    
    

  • jdalyasc's picture

    Also testing with the trial version of enterprise version with the W3 parser. I should mention we are not beyond purchasing the enterprise edition if needed. We are simply looking for a solution that will be able to message tracking logs. So far both examples appear incorrect.

    Panic Soft
    define ROOT C:\Program Files\nxlog
    ModuleDir %ROOT%\modules
    CacheDir  %ROOT%\data
    PidFile   %ROOT%\data\nxlog.pid
    SpoolDir  %ROOT%\data
    
    define CERTDIR %ROOT%\cert
    define CONFDIR %ROOT%\conf
    
    # Note that these two lines define constants only; the log file location
    # is ultimately set by the `LogFile` directive (see below). The
    # `MYLOGFILE` define is also used to rotate the log file automatically
    # (see the `_fileop` block).
    define LOGDIR %ROOT%\data
    define MYLOGFILE %LOGDIR%\nxlog.log
    
    # By default, `LogFile %MYLOGFILE%` is set in log4ensics.conf. This
    # allows the log file location to be modified via NXLog Manager. If you
    # are not using NXLog Manager, you can instead set `LogFile` below and
    # disable the `include` line.
    #LogFile %MYLOGFILE%
    include %CONFDIR%\log4ensics.conf
    
    <Extension _syslog>
        Module  xm_syslog
    </Extension>
    
    <Extension w3c_parser>
        Module      xm_w3c
        Delimiter   ,
    </Extension>
    
    <Input messagetracking>
        Module      im_file
        File        '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG'
        InputType   w3c_parser
    </Input>
    
    <Output tcp>
        Module      om_udp
        Host        10.1.1.1
        Port        514
        Exec to_syslog_snare()
    </Output>
    
    
    2018-07-16 12:19:53 ERROR Couldn't parse Exec block at C:\Program Files\nxlog\conf\nxlog.conf:44; couldn't parse statement at line 44, character 23 in C:\Program Files\nxlog\conf\nxlog.conf; function 'to_syslog_snare()' does not exist or takes different arguments
    2018-07-16 12:19:53 ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib
    2018-07-16 12:19:53 WARNING no routes defined!
    2018-07-16 12:19:53 INFO nxlog-4.0.3735-trial started
    2018-07-16 12:19:53 WARNING not starting unused module tcp
    2018-07-16 12:19:53 WARNING not starting module messagetracking because it is part of an incomplete route
    2018-07-16 12:19:53 ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib
    

  • Zhengshi's picture
    (NXLog)

    Exec to_syslog_snare()
    

    Missing ; at end. Exec to_syslog_snare();

    2018-07-16 12:19:53 ERROR SSL error, failed to load ca cert from 'C:\Program Files\nxlog\cert\agent-ca.pem', reason: No such file or directory, no such file, system lib

    This part comes from the included include %CONFDIR%\log4ensics.conf

  • jdalyasc's picture

    I was able to get the config working with the CE edition that you provided. I am going to uninstall the CE edition and try testing with the Enterprise edition of the software. Are there any other benefits of the enterprise edition other than the built in W3 parser?\

    Thank you

Answers (0)