2
responses

Hi,
I try configure regex to send log from localhost_access.log tomcat. I test this in many regex tester online and have error when I use this in nxlog.

Sample:

4.3.2.1 - - [21/Oct/2019:06:29:10 +0200] "GET /webapi?Subsystem=Order&Action=GetTradeReports&Exchange=GRATR&TS=1571632145141&trpt.a=0%2C1%2C3%2C4%2C5%2C6%2C8%2C9%2C10%2C11%2C13%2C15%2C17%2C47%2C52%2C53%2C57%2C58%2C59%2C60%2C61%2C62%2C63&Session=A.z27L1EKuJetOW0RD890Q3DXXK1zwo5eJ HTTP/1.1" 200 89

and config:

<Input tomcat_localhost>
InputType multiline
Module im_file
File '/var/log/tomcat/localhost_access.log'

<Exec>
if $raw_event =~ /^(\S+) (\S+) (\S+) \[([\w:/]+\s[+\-]\d{4})\] "(\S+)\s?(\S+)?\s?(\S+)?" (\d{3}|-) (\d+|-)\s?"?([^"]*)"?\s?"?([^"]*)?"?$/

{
$IP_address = $1;
$EventTime = parsedate($4);
$HTTPMethod = $5;
$HTTPMethod = $4;
$HTTPURL = $5;
$HTTPResponseStatus = $8;
$HTTPProtocol = $7;
$Message = $6;
}
</Exec>
</Input>

error in log:

2019-10-21 10:01:08 ERROR Couldn't parse Exec block at /var/lib/graylog-sidecar/generated/nxlog.conf:61;couldn't parse statement at line 62, character 47 in /var/lib/graylog-sidecar/generated/nxlog.conf;failed to compile regular expression '^(\S+) (\S+) (\S+) \[([\w:', error at position 26: missing terminating ] for character class

Why they expected "]" ?

AskedOctober 21, 2019 - 10:32am

Comments (1)

  • manuel.munoz's picture
    (NXLog)

    Hi Sata,

    I think your regular expression is not correct, you need to escape first slash bar.

    /^(\S+) (\S+) (\S+) \[([\w:/]+\s[+\-]\d{4})\] "(\S+)\s?(\S+)?\s?(\S+)?" (\d{3}|-) (\d+|-)\s?"?([^"]*)"?\s?"?([^"]*)?"?$/

    / An unescaped delimiter must be escaped with a backslash ()

    Should be...

    /^(\S+) (\S+) (\S+) \[([\w:\/]+\s[+\-]\d{4})\] "(\S+)\s?(\S+)?\s?(\S+)?" (\d{3}|-) (\d+|-)\s?"?([^"]*)"?\s?"?([^"]*)?"?$/

    Also you can avoid some variables for better performance (as you only need $1, $4, $5, $6, $7, $8)...

    /^(\S+) \S+ \S+ \[([\w:\/]+\s[+\-]\d{4})\] "(\S+)\s?(\S+)?\s?(\S+)?" (\d{3}|-) \d+|-\s?"?[^"]*"?\s?"?[^"]*?"?$/

    You can verify your regular expessions using https://regex101.com

    Full match  0-296   4.3.2.1 - - [21/Oct/2019:06:29:10 +0200] "GET /webapi?Subsystem=Order&Action=GetTradeReports&Exchang...
    Group 1.    0-7 4.3.2.1
    Group 2.    13-39   21/Oct/2019:06:29:10 +0200
    Group 3.    42-45   GET
    Group 4.    46-279  /webapi?Subsystem=Order&Action=GetTradeReports&Exchange=GRATR&TS=1571632145141&trpt.a=0%2C1%2C3%2C4%...
    Group 5.    280-288 HTTP/1.1
    Group 6.    290-293 200
    

Answer (1)