NXLog CE - Exchange log

Tags:

#1 gtsintaris

Hi all,

I'm trying to integrate the Exchange Logs to NXLog CE using the example config from https://nxlog.co/documentation/nxlog-user-guide/exchange.html which does'nt work. After reading through the forums I have come up with the following nxlog.conf which still doesn't work producing the errors at the end of this post. Is there any comprehensive guide to setup it up?

---------------- nxlog.conf ----------------------

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log define BASEDIR E:\Exchange Server LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') &gt;= 5M)) \
                file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

</Extension> <Extension csv_parser> Module xm_csv Fields date-time, client-ip, client-hostname, server-ip, server-hostname,
source-context, connector-id, source, event-id,
internal-message-id, message-id, network-message-id,
recipient-address, recipient-status, total-bytes, recipient-count,
related-recipient-address, reference, message-subject,
sender-address, return-path, message-info, directionality,
tenant-id, original-client-ip, original-server-ip, custom-data,
transport-traffic-type, log-id, schema-version,
session, source, Destination, direction, description,
session-id, sequence-number,
local-endpoint, remote-endpoint, event, data, context Exec rename_field('sender-address', 'sender_address'); $sender_address = lc($sender_address); Exec rename_field('date-time', 'date_time'); $date_time = lc($date_time);

</Extension>

<Input messagetracking> Module im_file File '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input connectivity> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input smtp_receive> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input smtp_send> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Extension _json> Module xm_json </Extension>

<Output out> Module om_http URL http://kibanaip:9200 ContentType application/json <Exec> set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); rename_field("timestamp", "@timestamp"); to_json(); </Exec> </Output>

---------------- nxlog.conf ----------------------

Errors: 2021-03-05 11:46:03 WARNING no routes defined! 2021-03-05 11:46:03 WARNING not starting unused module messagetracking 2021-03-05 11:46:03 WARNING not starting unused module connectivity 2021-03-05 11:46:03 WARNING not starting unused module smtp_receive 2021-03-05 11:46:03 WARNING not starting unused module smtp_send 2021-03-05 11:46:03 WARNING not starting unused module out 2021-03-05 11:46:03 INFO nxlog-ce-2.10.2150 started

Thank You in advance! Gregory

#3 rafDeactivated Nxlog ✓
#1 gtsintaris
Hi all, I'm trying to integrate the Exchange Logs to NXLog CE using the example config from https://nxlog.co/documentation/nxlog-user-guide/exchange.html which does'nt work. After reading through the forums I have come up with the following nxlog.conf which still doesn't work producing the errors at the end of this post. Is there any comprehensive guide to setup it up? ---------------- nxlog.conf ---------------------- Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log define BASEDIR E:\Exchange Server LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Extension csv_parser> Module xm_csv Fields date-time, client-ip, client-hostname, server-ip, server-hostname, source-context, connector-id, source, event-id, internal-message-id, message-id, network-message-id, recipient-address, recipient-status, total-bytes, recipient-count, related-recipient-address, reference, message-subject, sender-address, return-path, message-info, directionality, tenant-id, original-client-ip, original-server-ip, custom-data, transport-traffic-type, log-id, schema-version, session, source, Destination, direction, description, session-id, sequence-number, local-endpoint, remote-endpoint, event, data, context Exec rename_field('sender-address', 'sender_address'); $sender_address = lc($sender_address); Exec rename_field('date-time', 'date_time'); $date_time = lc($date_time); </Extension> <Input messagetracking> Module im_file File '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input> <Input connectivity> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input> <Input smtp_receive> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input> <Input smtp_send> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input> <Extension _json> Module xm_json </Extension> <Output out> Module om_http URL http://kibanaip:9200 ContentType application/json <Exec> set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); rename_field("timestamp", "@timestamp"); to_json(); </Exec> </Output> ---------------- nxlog.conf ---------------------- Errors: 2021-03-05 11:46:03 WARNING no routes defined! 2021-03-05 11:46:03 WARNING not starting unused module messagetracking 2021-03-05 11:46:03 WARNING not starting unused module connectivity 2021-03-05 11:46:03 WARNING not starting unused module smtp_receive 2021-03-05 11:46:03 WARNING not starting unused module smtp_send 2021-03-05 11:46:03 WARNING not starting unused module out 2021-03-05 11:46:03 INFO nxlog-ce-2.10.2150 started Thank You in advance! Gregory

Hi,

Without checking other parts of your config, your error is self-explainable:

2021-03-05 11:46:03 WARNING no routes defined!

This means you haven't declared a Route - that's a config section which tell NXLog how to connect your source (implemented in <Input> section) with the destination (<Output>).

Blind guess would be that something like this might work for you:

<Route r1>
Path smtp_send => out
</Route>

Hence, it was quite important part of the config that was missing :)

Hope it helps,
Rafal