Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.

HELP. Divided into fields Fortinet logs with regular expressions.
Hi, my logs: <188>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.25d.1dd.ddd srcport=59ddd srcintf="portB" dstip=2dd.1dd.dd5.ddd dstport=dd3 dstintf="port9" poluuid=16594dae-dddd-51e7-dddd-da81ec23cc23 sessionid=26ddd5925 proto=6 action=ip-conn policyid=103 policytype=policy appcat="unscanned" crscore=5 craction=262144 crlevel=low devtype="Router/NAT Device" mastersrcmac=00:dd:14:dd:c8:dd srcmac=00:dd:14:dd:c8:ddt="default" appa <189>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=10.25d.1dd.ddd srcport=59674 srcintf="portB" dstip=2dd.1dd.dd5.ddd dstport=443 dstintf="port9" poluuid=16594dae-dddd-51e7-dddd-da81ec23cc23 sessionid=260ddd925 proto=6 action=close policyid=103 policytype=policy dstcountry="United States" srccountry="Reserved" trandisp=snat transip=10.2dd.1dd.2dd transport=59674 service="HTTPS" appid=40568 app="HTTPS.BROWSER" appcat="Web.Client" apprisk=medium applist="default" appact=detected duration=140 sentbyte=1244 rcvdbyte=770 sentpkt=9 rcvdpkt=6 devtype="Router/NAT Device" mastersrcmac=00:2d:dd:6b:dd:60 srcmac=00:dd:14:dd:c8:ddc=88:ad:d2:88:eb <189>date=2017-09-26 time=10:53:04 devname=FT01 devid=************* logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=10.2dd.dd.dd srcport=42768 srcintf="portB" dstip=2dd.1dd.dd.dd dstport=53 dstintf="port9" poluuid=0dddda60-dddd-51e7-dddd-56c9d0ddde2f sessionid=260ddd113 proto=17 action=accept policyid=84 policytype=policy dstcountry="Hong Kong" srccountry="Reserved" trandisp=snat transip=10.2dd.dd.2d transport=42768 service="DNS" appid=16195 app="DNS" appcat="Network.Service" apprisk=elevated applist="default" appact=detected duration=180 sentbyte=77 rcvdbyte=93 sentpkt=1 rcvdpkt=1 devtype="Router/NAT Device" mastersrcmac=00:dd:14:dd:c8:dd srcmac=00:dd:14:dd:c8:dddd:6060 srcmac=0 I want to use regular expressions: field  >> regex action = ^.+\saction=(\S+)\s app = ^.+\sapp=\"(.+?)\" appcat = ^.+\sappcat=\"(.+?)\" applist = ^.+\sapplist=\"(.+?)\" attack = ^.+\sattack=\"(.+?)\" devid = ^.+\sdevid=(\S+)\s dir = ^.+\sdir=(\S+)\s dstcountry = ^.+\sdstcountry=\"(.+?)\" dstintf = ^.+\sdstintf=\"(.+?)\" dstip = ^.+\sdstip=(\S+)\s dstport = ^.+\sdstport=(\S+)\s ... 175 more What configuration to use? <Input i.forti.log>  Module im_file  File "/var/log/forti.log"  InputType LineBased </Input> <Output o.forti.log>  Module om_tcp  Host 192.168.00.00  Port XXXXX  CAFile /data/conf/ca.crt  AllowUntrusted TRUE  OutputType LineBased </Output> <Route r.forti.log>  Path i.forti.log => o.forti.log </Route>   Thank you very much!!

absolis created
Replies: 1
View post »
last updated
ERROR invalid keyword when I tried parse logs with regex.
Hello ,      I'm trying get specific data from some logs of hadoop with REGEX and I recieved this error: ERROR invalid keyword: Output at C:\Program Files (x86)\nxlog\conf\nxlog.conf:45       Here is my config file: define ROOT C:\Program Files (x86)\nxlog #  Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log # <Extension gelf>     Module         xm_gelf </Extension> <Extension fileop>     Module         xm_fileop </Extension> <Extension json>     Module      xm_json </Extension> <Extension multi>     Module      xm_multiline     HeaderLine  /^(\d+-\d+-\d+\s\d+:\d+:\d+,\d+)/     EndLine        /(.*)/ </Extension> # <Input hadoop>   Module         im_file   File             "E:\\Hadoop\\test\\*.*"   SavePos         TRUE   Recursive     TRUE   InputType        multi      Exec      if $raw_event =~/^(\d+-\d+-\d+\s\d+:\d+:\d+,\d+)\s(?:INFO|ERROR|WARN)\s(org.apache.hadoop.\w+.\w+):\s(.*)/g\             {\                 $Time = $1;\                 $CStatus = $2;\                 $Process = $3;\                 $Process_result = $4;\                 to_json();\             }\             else\             {\                 drop();\             }\ </Input> <Output graylog>     Module      om_udp     Host        10.101.78.224     Port        12201     OutputType    GELF       #Use the following line for debugging (uncomment the fileop extension above as well)     #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event); </Output> <Route eventlog>     Path        hadoop => graylog </Route> Anyone know what is bad in this config file?. THank you.

Juan Andrés.Ramirez created
Replies: 1
View post »
last updated