How to add additional fields when using im_file module

Post a different question
4
responses

Hi, 

I'm using im_file module to read windows log file to elasticsearch, and I need to get source file name and some fix string like log type to elastic search as well. I used below configuration, but i couldn't find the FileName in elasticsearch, can someone help? thanks a lot!

<Input TestFileInput>
    Module          im_file
    File              'E:\test\app*.log'
    Exec            $FileName = file_name();
</Input>

AskedDecember 10, 2015 - 7:17am

Comments (2)

  • brookssw's picture

    did you ever solve this? I'm having a similar problem, and I'm almost certain it's not being filtered elsewhere; I'm looking at the logstash feed that nxlog pushes to, and there's no sign of the added field, and logstash has no filters whatsoever.

    December 14, 2015 - 10:25pm

Answer (1)

The configuration is correct. Your fields are getting discarded elsewhere in the process when loading ES.

AnsweredDecember 10, 2015 - 11:35am

Comments (1)

  • zpp's picture

    this is what i did, 

    <Input TestFileInput>
        Module          im_file
        File              'E:\test\app*.log'
    Exec $raw_event= file_name() + ',' + $raw_event;;
    </Input>

    at elasticsearch indexer, i used grok to parse the message

    or the other way (think it should work, but i didn't test it)

    <Input TestFileInput>
        Module          im_file
        File              'E:\test\app*.log'
    Exec $FileName = file_name();

    Exec to_json();
    </Input>

    at indexer side add in a json codec.

     

    December 17, 2015 - 12:30pm