I have a file with multiple log lines, but I'm only interested in one type that has 6 fields in CSV format. I want to discard all the rest. So I have this [partial] file:

<Extension csv>
    Module      xm_csv
    Fields      $time, $date, $host, $from, $ip, $loginfo, $color
    FieldTypes  string, string, string, string, string, string, integer
    Delimiter   |

<Input M2P_In>
    Module    im_file
    File    "C:\\M2PLogs\\log*"
    SavePos  TRUE

        if $raw_event =~ /^#/ drop();
            if ( not defined $color ) drop();
            $message = $raw_event;
            $raw_event = to_json();

In this case, if the line doesn't have 6 fields, I understand the $color field will be undefined. But it doesn't work, I get both lines in output: the correct one being processed and the rest in plain text.

Perhaps I'm following the wrong approach, so I'm also open for alternatives. Could you please help?

AskedJanuary 9, 2020 - 6:01pm

Comments (5)

  • Armaggedon's picture

    Thanks for your reply Misa,

    Line that should be processed:

    3:06:10 PM|1/9/2020|aaa|bbb|ccc|ddd|100

    Line that should not be processed:

    4:34:01 PM|1/9/2020|working on: smime.p7s

    In summary, all undesired lines will only and always have 3 fields, so the parse should fail. All desired lines will always have 7 fields.

Answer (1)