drop if empty field

Tags:

#1 Armaggedon

I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file:

<Extension csv>
    Module      xm_csv
    Fields      $time, $date, $host, $from, $ip, $loginfo, $color
    FieldTypes  string, string, string, string, string, string, integer
    Delimiter   |
</Extension>

<Input M2P_In>
    Module    im_file
    File    "C:\\M2PLogs\\log*"
    SavePos  TRUE
 
    <Exec>
        if $raw_event =~ /^#/ drop();
        else
        {
            csv->parse_csv();
            if ( not defined $color ) drop();
            $message = $raw_event;
            $raw_event = to_json();
        }
    </Exec>
</Input>

In this case, if the line doesn't have 6 fields, I understand the $color field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text.

Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

#2 MisazivDeactivated Nxlog ✓
#1 Armaggedon
I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file: <Extension csv> Module xm_csv Fields $time, $date, $host, $from, $ip, $loginfo, $color FieldTypes string, string, string, string, string, string, integer Delimiter | </Extension> <Input M2P_In> Module im_file File "C:\\M2PLogs\\log*" SavePos TRUE <Exec> if $raw_event =~ /^#/ drop(); else { csv->parse_csv(); if ( not defined $color ) drop(); $message = $raw_event; $raw_event = to_json(); } </Exec> </Input> In this case, if the line doesn't have 6 fields, I understand the $color field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text. Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

Hi,

Do you have an example of CSV I can use to test?

Regards, Misa