1
answer

I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file:

<Extension csv>
    Module      xm_csv
    Fields      $time, $date, $host, $from, $ip, $loginfo, $color
    FieldTypes  string, string, string, string, string, string, integer
    Delimiter   |
</Extension>

<Input M2P_In>
    Module    im_file
    File    "C:\\M2PLogs\\log*"
    SavePos  TRUE

    <Exec>
        if $raw_event =~ /^#/ drop();
        else
        {
            csv->parse_csv();
            if ( not defined $color ) drop();
            $message = $raw_event;
            $raw_event = to_json();
        }
    </Exec>
</Input>

In this case, if the line doesn't have 6 fields, I understand the $color field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text.

Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

AskedJanuary 9, 2020 - 6:01pm

Comments (5)

  • Armaggedon's picture

    Thanks for your reply Misa,

    Line that should be processed:

    3:06:10 PM|1/9/2020|aaa|bbb|ccc|ddd|100
    

    Line that should not be processed:

    4:34:01 PM|1/9/2020|working on: smime.p7s
    

    In summary, all undesired lines will only and always have 3 fields, so the parse should fail. All desired lines will always have 7 fields.

Answer (1)