Delay event processing

Post a different question
6
responses

Hi everyone! I`m trying to delay an event processing for a second, i.e.: if an event has a description I'll delay the output for a period. I found the sleep method in the Documentation, but it's not working as intended. Below is how my input is configured:

<Input input_file>
    Module            im_file
    File            "%LOGPATH%"
    ReadFromLast            TRUE
    Recursive            FALSE

    <Exec>
        if ($raw_event =~ /UPDATE/)
        {
            parse_json();
        }
        else if ($raw_event =~ /DELETE/)
        {
            sleep(2000) # wait 2 seconds before processing this event
            parse_json();
        }
    </Exec>
</Input>

Is is possible to delay/wait for a few seconds before processing an event?

Thank you in advance!

AskedAugust 21, 2020 - 10:26pm

Answer (1)

Hello,

Have you tried the sleep() procedure? It takes a number of microseconds as an argument - I suppose it might help with your task. The docs are here:

https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#core_proc_sleep

Regards,

Rafal

AnsweredAugust 24, 2020 - 7:33am

Comments (5)

  • LukeOrins's picture

    Don't worry. The complete configuration file nxlog.conf is as follows:

    Panic Soft
#NoFreeOnExit TRUE

# NXLOG logging and configuration
define ROOT     C:\Program Files (x86)\nxlog
define CERTDIR  %ROOT%\cert
define CONFDIR  %ROOT%\conf
define LOGDIR   %ROOT%\data
define LOGFILE  %LOGDIR%\nxlog.log
define LOGPATH C:\\Temp\\Input\\test*.log

# NXLOG logging path
LogFile %LOGFILE%
LogLevel ERROR
NoCache TRUE

Moduledir %ROOT%\modules
CacheDir  %ROOT%\data
Pidfile   %ROOT%\data\nxlog.pid
SpoolDir  %ROOT%\data

<Extension _syslog>
    Module      xm_syslog
</Extension>

<Extension _charconv>
    Module              xm_charconv
    AutodetectCharsets  iso8859-2, utf-8, utf-16, utf-32
</Extension>

<Extension _fileop>
    Module      xm_fileop

    # Check the size of our log file hourly, rotate if larger than 5MB
    <Schedule>
        Every   1 hour
        Exec    if (file_exists('%LOGFILE%') and \
                   (file_size('%LOGFILE%') >= 5M)) \
                    file_cycle('%LOGFILE%', 8);
    </Schedule>

    # Rotate our log file every week on Sunday at midnight
    <Schedule>
        When    @weekly
        Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
    </Schedule>
</Extension>

<Extension json>
    Module      xm_json
</Extension>

<Extension multiline>
    Module          xm_multiline
    HeaderLine      /^operationtype: /
</Extension>

<Input input_file>
    Module            im_file
    File            "%LOGPATH%"
    ReadFromLast            TRUE
    Recursive            FALSE

    <Exec>
        if ($raw_event =~ /UPDATE/)
        {
            $raw_event = '{ "action" : "DEVICE_UPDATE" }';
            parse_json();
        }
        else if ($raw_event =~ /DELETE/)
        {
            $raw_event = '{ "action" : "DEVICE_DELETION" }';
            sleep(2000); # wait 2 seconds before processing this event
            parse_json();
        }
        else
        {
            drop();
        }
    </Exec>
</Input>

# Testing output
<Output output_file>
    Module  om_file
    File    "C:\Temp\output.json"
</Output>

# Testing output route
<Route file_to_file>
    Path    input_file => output_file
</Route>

    Sample log content in test.log:

    operationtype: UPDATE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: update
operationTimestamp: 20200427157412
-

operationtype: UPDATE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: update
operationTimestamp: 20200427102538
-

operationtype: DELETE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: delete
operationTimestamp: 20200427199999
-

operationtype: UPDATE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: update
operationTimestamp: 20200427102516
-

operationtype: DELETE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: delete
operationTimestamp: 20200425120214
-

operationtype: UPDATE
authenticatedUser: xxxxx-xxxxx-xxxxx-xxxxx
changetype: update
operationTimestamp: 20200427153003
-

    What I'm receiving in my output.json file:

    { "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_DELETION" }
{ "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_DELETION" }
{ "action" : "DEVICE_UPDATE" }

    What I expect:

    { "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_UPDATE" }
{ "action" : "DEVICE_DELETION" }
{ "action" : "DEVICE_DELETION" }

    Thank you for your time!

    August 25, 2020 - 3:29am
  • raf's picture
    (NXLog)

    Hello,

    There are two main remarks to your case:

    1. The delay() takes microseconds, not milliseconds as the argument, hence, if you want to have seconds - you need to add more "zeros" (000)
    2. I'm not sure what's your aim, but this approach wouldn't give you expected result: since delay() works sequentially, therefore, it will simply pause the whole operations set, without impacting the order.

    Regards,

    Rafal

    August 25, 2020 - 1:16pm
  • LukeOrins's picture

    Thank you for the response. Is there a way for the sleep() operation to affect only some events? For example, process them in a different route in the same nxlog.conf file? The order doesn't matter, I just showed how I expected the outcome to be. What I need is for some events to be processed later, without impacting the others.

    August 25, 2020 - 5:47pm