3
responses

Hello 

I have created a regex expression to extract values from my logs, I have difficulties in writing these values in the nxlog output

The issue is that the output file is identical to the input log, not sure what I'm doing wrong here, any help is welcome

My nxlog.conf is as follow

<Input in>
        Module  im_file
        File    "C:\\tmp\example-log.txt"
        SavePos  TRUE
        Recursive TRUE
        InputType       multiline
        exec if $raw_event =~ /(?:\{"([0-9]+?), ([0-9]+?), "(.+?)", "(.+?)", "(?:.+?)", "(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)New\sLogon:\s*(?:.+?)\n\s*Account\sName:\s*(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)Source\sNetwork\sAddress:\s*([0-9\.]{7,15})\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)"\})/ \
    { \
    $timestamp = $1; \
    $event = $2; \
    $status = $3; \
    $type = $4; \
    $short = $5; \
    $user = $6; \
    $source = $7; \
    }
</Input>

#<Output out>
#    Module      om_tcp
#    Host        192.168.1.15
#    Port        12201
#    OutputType  GELF_TCP
#</Output>

<Processor one>
    Module      pm_null
    Exec        $raw_event = $timestamp + $event + $status + $type + $short + $user + $source;
</Processor>

<Output out>
    Module    om_file
    File    'C:\\tmp\output'
</Output>

<Route 1>
    Path        in => one => out
</Route>

  

Thanks heaps

Mark

 

AskedSeptember 3, 2015 - 2:23pm

Answer (1)

I have difficulties in writing these values in the nxlog output

Please elaborate on this. Are you getting emtpy lines in the output, no output at all, errors in nxlog.log, etc?

Comments (2)

  • mark's picture

    Hello

    Sorry for not being clear,

    The input log file is written to the ouput file exctaly as it is

    Kindly see below the the input log file and the output file

    I have no error in the nxlog.log

     

    Input log file:

    {"1431237504000, 4624, "Success", "Security", "Microsoft-Windows-Security-Auditing", "An account was successfully logged on.

    Subject:
        Security ID:        S-1-0-0
        Account Name:        -
        Account Domain:        -
        Logon ID:        0x0

    Logon Type:            3

    New Logon:
        Security ID:        S-1-5-21-1039142878-192558930-806110124-72544
        Account Name:        xn177n
        Account Domain:        .DOM
        Logon ID:        0x31e2a4d33
        Logon GUID:        {44F1AABB-87B1-6665-DA1F-7D4E1C0E6B2D}

    Process Information:
        Process ID:        0x0
        Process Name:        -

    Network Information:
        Workstation Name:    
        Source Network Address:    192.168.173.159
        Source Port:        55159

    Detailed Authentication Information:
        Logon Process:        Kerberos
        Authentication Package:    Kerberos
        Transited Services:    -
        Package Name (NTLM only):    -
        Key Length:        0

    This event is generated when a logon session is created. It is generated on the computer that was accessed.

    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

    The authentication information fields provide detailed information about this specific logon request.
        - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
        - Transited services indicate which intermediate services have participated in this logon request.
        - Package name indicates which sub-protocol was used among the NTLM protocols.
        - Key length indicates the length of the generated session key. This will be 0 if no session key was requested." "}

     

    Ouput file:

    {"1431237504000, 4624, "Success", "Security", "Microsoft-Windows-Security-Auditing", "An account was successfully logged on.

    Subject:
        Security ID:        S-1-0-0
        Account Name:        -
        Account Domain:        -
        Logon ID:        0x0

    Logon Type:            3

    New Logon:
        Security ID:        S-1-5-21-1039142878-192558930-806110124-72544
        Account Name:        xn177n
        Account Domain:        .DOM
        Logon ID:        0x31e2a4d33
        Logon GUID:        {44F1AABB-87B1-6665-DA1F-7D4E1C0E6B2D}

    Process Information:
        Process ID:        0x0
        Process Name:        -

    Network Information:
        Workstation Name:    
        Source Network Address:    192.168.173.159
        Source Port:        55159

    Detailed Authentication Information:
        Logon Process:        Kerberos
        Authentication Package:    Kerberos
        Transited Services:    -
        Package Name (NTLM only):    -
        Key Length:        0

    This event is generated when a logon session is created. It is generated on the computer that was accessed.

    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

    The authentication information fields provide detailed information about this specific logon request.
        - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
        - Transited services indicate which intermediate services have participated in this logon request.
        - Package name indicates which sub-protocol was used among the NTLM protocols.
        - Key length indicates the length of the generated session key. This will be 0 if no session key was requested." "}

     

    Thanks very much

    Mark

  • adm's picture
    (NXLog)

    Not sure why the modified $raw_event is not being output, have you properly saved the configuration and restarted the service?

    To help debugging you can also try to add the following to the output to get all the fields as a json line:

    Exec to_json();

    There are several similar examples in the Reference Manual.

    Also the processor instance is not necessary (i.e. you can put the Exec $raw_event = ... in the input or output and get rid of pm_null) but this is just cosmetical stuff and should do the same thing.

    See this section in the manual for more debugging tips.