Hi, my logs:

<188>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.25d.1dd.ddd srcport=59ddd srcintf="portB" dstip=2dd.1dd.dd5.ddd dstport=dd3 dstintf="port9" poluuid=16594dae-dddd-51e7-dddd-da81ec23cc23 sessionid=26ddd5925 proto=6 action=ip-conn policyid=103 policytype=policy appcat="unscanned" crscore=5 craction=262144 crlevel=low devtype="Router/NAT Device" mastersrcmac=00:dd:14:dd:c8:dd srcmac=00:dd:14:dd:c8:ddt="default" appa

<189>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=10.25d.1dd.ddd srcport=59674 srcintf="portB" dstip=2dd.1dd.dd5.ddd dstport=443 dstintf="port9" poluuid=16594dae-dddd-51e7-dddd-da81ec23cc23 sessionid=260ddd925 proto=6 action=close policyid=103 policytype=policy dstcountry="United States" srccountry="Reserved" trandisp=snat transip=10.2dd.1dd.2dd transport=59674 service="HTTPS" appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk=medium applist="default" appact=detected duration=140 sentbyte=1244 rcvdbyte=770 sentpkt=9 rcvdpkt=6 devtype="Router/NAT Device" mastersrcmac=00:2d:dd:6b:dd:60 srcmac=00:dd:14:dd:c8:ddc=88:ad:d2:88:eb

<189>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=10.2dd.dd.dd srcport=42768 srcintf="portB" dstip=2dd.1dd.dd.dd dstport=53 dstintf="port9" poluuid=0dddda60-dddd-51e7-dddd-56c9d0ddde2f sessionid=260ddd113 proto=17 action=accept policyid=84 policytype=policy dstcountry="Hong Kong" srccountry="Reserved" trandisp=snat transip=10.2dd.dd.2d transport=42768 service="DNS" appid=16195 app="DNS" appcat="Network.Service" apprisk=elevated applist="default" appact=detected duration=180 sentbyte=77 rcvdbyte=93 sentpkt=1 rcvdpkt=1 devtype="Router/NAT Device" mastersrcmac=00:dd:14:dd:c8:dd srcmac=00:dd:14:dd:c8:dddd:6060 srcmac=0

I want to use regular expressions:

field  >> regex

action = ^.+\saction=(\S+)\s
app = ^.+\sapp=\"(.+?)\"
appcat = ^.+\sappcat=\"(.+?)\"
applist = ^.+\sapplist=\"(.+?)\"
attack = ^.+\sattack=\"(.+?)\"
devid = ^.+\sdevid=(\S+)\s
dir = ^.+\sdir=(\S+)\s
dstcountry = ^.+\sdstcountry=\"(.+?)\"
dstintf = ^.+\sdstintf=\"(.+?)\"
dstip = ^.+\sdstip=(\S+)\s
dstport = ^.+\sdstport=(\S+)\s
... 175 more

What configuration to use?

<Input i.forti.log>
 Module im_file
 File "/var/log/forti.log"
 InputType LineBased
</Input>

<Output o.forti.log>
 Module om_tcp
 Host 192.168.00.00
 Port XXXXX
 CAFile /data/conf/ca.crt
 AllowUntrusted TRUE
 OutputType LineBased
</Output>

<Route r.forti.log>
 Path i.forti.log => o.forti.log
</Route>

Thank you very much!!

Hello ,

     I'm trying get specific data from some logs of hadoop with REGEX and I recieved this error: ERROR invalid keyword: Output at C:\Program Files (x86)\nxlog\conf\nxlog.conf:45

      Here is my config file:

define ROOT C:\Program Files (x86)\nxlog
# 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
#
<Extension gelf>
    Module         xm_gelf
</Extension>
<Extension fileop>
    Module         xm_fileop
</Extension>
<Extension json>
    Module      xm_json
</Extension>
<Extension multi>
    Module      xm_multiline
    HeaderLine  /^(\d+-\d+-\d+\s\d+:\d+:\d+,\d+)/
    EndLine        /(.*)/
</Extension>
#
<Input hadoop>
  Module         im_file
  File             "E:\\Hadoop\\test\\*.*"
  SavePos         TRUE
  Recursive     TRUE
  InputType        multi
  
  Exec      if $raw_event =~/^(\d+-\d+-\d+\s\d+:\d+:\d+,\d+)\s(?:INFO|ERROR|WARN)\s(org.apache.hadoop.\w+.\w+):\s(.*)/g\
            {\
                $Time = $1;\
                $CStatus = $2;\
                $Process = $3;\
                $Process_result = $4;\
                to_json();\
            }\
            else\
            {\
                drop();\
            }\
</Input>

<Output graylog>
    Module      om_udp
    Host        10.101.78.224
    Port        12201
    OutputType    GELF
 
    #Use the following line for debugging (uncomment the fileop extension above as well)
    #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Route eventlog>
    Path        hadoop => graylog
</Route>

Anyone know what is bad in this config file?.

THank you.