Using PowerShell to fetch logs and emptying monitored directory at specified intervals

Post a different question
3
responses

I'm looking at a slightly unusual application logging which has turned out quite challenging to handle with NXLog, as is, and for that I've been experimenting of running PS scripts using NXLog.
In principle, I'd like to know if it is possible to build the following scenario using NXLog Enterprise agent.

Running of PS script (using NXLog) to fetch log files at interval from variable directories and putting them into another (a copy of logs not older than 1 hour, PS script would manage this, but needs to be invoked by NXLog agent).
Reading selected events from the fetched logs and dispatching them to another system (note, this is completed in another scenario already so I know this would work).
Deleting of all logs from the import directory after they have been read. This could be managed with the xm_fileop, I believe.

I have been experimenting of running PS scripts, unsuccesfully so far, but I'm going through the docs and examples to understand how would one execute a (any) script using the NXLog agent.

Any advice will be highly appreciated.

AskedNovember 18, 2020 - 2:45pm

Answers (2)

Hi,

If I understood you correctly - I think what you trying to achieve is fairly doable, I suppose using PS might be not needed if solved correctly. The `conf` might be a bit complex in this case, but unless I miss something - shouldn't be a problem.

How about we start with seeing what have you already done? Let's use it as a starting point, and we'll try to give you some hints on what's next.

Regards,
Rafal

AnsweredNovember 19, 2020 - 9:07am

Comments (1)

  • PT_537256's picture

    The process of fetching and deleting is still shaping up and there might be some variation to how I've thought this will be done. In any case, I have a PS script that will copy files from A to B and then I have the NXLog config for reading the events from a local directory. I've only put here the bits that I see relevant in that config:

    <Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _xml>
    Module      xm_xml
</Extension>

<Extension multiline>
        Module      xm_multiline
        HeaderLine  /<?xml/
</Extension>

# This input module is used to collect event logs from App instance_1.
<Input instance1>
    Module  im_file
    File        "C:\\app\\instance1\\eventlog\\XY-*.C*.E-11*.xml"
    File        "C:\\app\\instance1\\eventlog\\XY-*.C*.E-12*.xml"
    File        "C:\\app\\instance1\\eventlog\\XY-*.C*.E-13*.xml"
    InputType   multiline
        <Exec>
            if $raw_event =~ /(.+)(<data)(.+)/
            {
                    $raw_event = $1 + ' instance_id=app_instance_1 ';
        }
            else
            {
                    $raw_event = $raw_event;
            }
        </Exec>
    SavePos     TRUE
    Recursive   TRUE
    ActiveFiles 1
    PollInterval    10
    CloseWhenIdle   TRUE
</Input>

# This input module is used to collect event logs from App instance_2.
<Input instance2>
    Module  im_file
    File        "C:\\app\\instance2\\eventlog\\XY-*.C*.E-11*.xml"
    File        "C:\\app\\instance2\\eventlog\\XY-*.C*.E-12*.xml"
    File        "C:\\app\\instance2\\eventlog\\XY-*.C*.E-13*.xml"
    InputType   multiline
        <Exec>
            if $raw_event =~ /(.+)(<data)(.+)/
            {
                    $raw_event = $1 + ' instance_id=app_instance_2 ';
        }
            else
            {
                    $raw_event = $raw_event;
            }
        </Exec>
    SavePos     TRUE
    Recursive   TRUE
    ActiveFiles 1
    PollInterval    10
    CloseWhenIdle   TRUE
</Input>

# This output module is used to forward App event logs to monitoring.
<Output monitoring>
    Module  om_tcp
    Host    192.168.255.10
    Port    514
    Exec    to_syslog_bsd();
</Output>

#For sending the events out
<Route instance_to_monitoring>
    Path instance1, instance2 => monitoring
</Route>

    So from these pieces I'm looking to take things forward.

    November 20, 2020 - 8:14am

Execution does work using xm_exec.
The problem was that the argument to the script path got changed during the testing process. What worked for the PowerShell terminal did not work for the NXLog agent.
Two backslashes "\\" are required in the path.

AnsweredDecember 2, 2020 - 12:39pm