im_file - parse if file is newer than X days (or fixed date)

View thread

kumdabur

Hi,

I'm testing nxlog with IIS servers. It works, however I have following issue: If server has IIS since years and lets assume that logs were stored for 1 year, I have bunch of iis logfiles which all together combined results with around 5GB of Data.

While using nxlog as-it-is, it consumes CPU and also SIEM itself is not entirely happy about receiving sudden "bombarding" of new logs within few minutes timeframe. Unfortunately, I have around 50 servers like that and I really do not need to inject past logs.

I wonder if there is an option to either throttle nxlog a bit or in best case I'd like to send ONLY new iis logs, even starting from exact today. Below you may find part of config

   <Input IIS_Logs_1>
        Module   im_file
        File        'C:\inetpub\logs\LogFiles\W3SVC*\u_ex*.log'
        ReadFromLast FALSE
        Recursive TRUE
        PollInterval  1
        Exec     $FileName = file_name();
        Exec if $raw_event =~ /^#/ drop();\
           else\
           {\
            w3c_1->parse_csv();\
            $SourceName = "IIS";\
           }
    </Input>

I'd appreciate any hints.