CSV-input: converting specific field(s) to lowercase


#1 nomoresecrets

Dear community,

I'm currently working on parsing MS Exchange logs and sending them via GELF to my graylog instance.

I'd like to convert the sender- and recipient-address field to lowercase. Sounds pretty easy, in fact, I need help :(

my current config looks like this (below). Any help is appreciated.

I've tried to work with "Exec       $sender-address = lc($sender-address);" within the input as well as Output backet - neither did work.


define BASEDIR C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\MessageTracking
<Extension csv>
   Module      xm_csv
   Fields      $date-time, $client-ip, $client-hostname, $server-ip, $server-hostname, $source-context, $connector-id, $exchange_source, $event-id, $internal-message-id, $message-id, $recipient-address, $recipient-status, $total-bytes, $recipient-count, $related-recipient-address, $reference, $message-subject, $sender-address, $return-path, $message-info, $directionality, $tenant-id, $original-client-ip, $original-server-ip, $custom-data
   FieldTypes  string, string, string, string, string, string, string, string, string, integer, string, string, string, integer, integer, string, string, string, string, string, string, string, string, string, string, string
   Delimiter   ,
</Extension>

<Input in_exchange>  
   Module     im_file
   File       '%BASEDIR%\MSGTRK????????*-*.LOG'
   SavePos    TRUE
   Exec       if $raw_event =~ /HealthMailbox/ drop();
   Exec       if $raw_event =~ /^#/ drop();
   Exec       csv->parse_csv();
</Input>

<Output out_exchange>  
   Module     om_udp
   Host       graylog.local
   Port       12203
   OutputType GELF
   Exec       $SourceName = 'exchange_msgtrk_log';
</Output>

<Route exchange>  
    Path      in_exchange => out_exchange
</Route> 
#2 adm Nxlog ✓
#1 nomoresecrets
Dear community, I'm currently working on parsing MS Exchange logs and sending them via GELF to my graylog instance. I'd like to convert the sender- and recipient-address field to lowercase. Sounds pretty easy, in fact, I need help :( my current config looks like this (below). Any help is appreciated. I've tried to work with "Exec       $sender-address = lc($sender-address);" within the input as well as Output backet - neither did work. define BASEDIR C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\MessageTracking <Extension csv> Module xm_csv Fields $date-time, $client-ip, $client-hostname, $server-ip, $server-hostname, $source-context, $connector-id, $exchange_source, $event-id, $internal-message-id, $message-id, $recipient-address, $recipient-status, $total-bytes, $recipient-count, $related-recipient-address, $reference, $message-subject, $sender-address, $return-path, $message-info, $directionality, $tenant-id, $original-client-ip, $original-server-ip, $custom-data FieldTypes string, string, string, string, string, string, string, string, string, integer, string, string, string, integer, integer, string, string, string, string, string, string, string, string, string, string, string Delimiter , </Extension> <Input in_exchange> Module im_file File '%BASEDIR%\MSGTRK????????*-*.LOG' SavePos TRUE Exec if $raw_event =~ /HealthMailbox/ drop(); Exec if $raw_event =~ /^#/ drop(); Exec csv->parse_csv(); </Input> <Output out_exchange> Module om_udp Host graylog.local Port 12203 OutputType GELF Exec $SourceName = 'exchange_msgtrk_log'; </Output> <Route exchange> Path in_exchange => out_exchange </Route>

You can do something like this:

Exec rename_field('sender-address', 'sender_address'); $sender_address = lc($sender_address);

rename_field() is needed because the dash causes issues in field names (i.e. $sender-address is not valid). You can rename it back if you want. Else you should specify names that don't contain a dash in xm_csv's Fields.