Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.

multiline message
Hello. I have a question. I get multiline messages how can I combine into a single line, multiline message ?? for example this message, In this message 4 lines Jul 21 17:59:10 <14> 1 2016-07-04T00: 53: 02.000000 + 03: 00 node = sec-sflow type = SYSCALL msg = audit (1467579182.055: 3248181): arch = 111 2 syscall = success = yes exit = 4 a0 = 7fc7783127a8 a1 = 2 a2 = a3 = 0 8 items = 1 ppid = 11013 pid = 30363 auid = 0 0 uid = gid = 0 = 0 euid suid = 0 fsuid = 0 = 0 egid sgid = 0 = 0 fsgid tty = (none) ses = 28 comm = "sshd" exe = "/ usr / sbin / sshd" key = "root_action" Thank!

toreno93 created
Replies: 1
View post »
last updated
Multiline Headerline Regex Error
I am trying to use the multlog module in order to start ingesting a custom log: I have the following regex: \^(\d{2}|\d).(\d{2}|\d).(\d{4})\s(\d\d|\d):(\d\d|\d):(\d\d|\d)\s(AM|PM).\[(.*)\](.*) This works in a regex test; however I cannot get it to work with the log file that looks something like this 9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-17] GetStatus for IP: 192.168.0.231 on port: 5016 9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-47] <TRANSACTION> <FUNCTION_TYPE>SECONDARYPORT</FUNCTION_TYPE> <COMMAND>STATUS</COMMAND> <MAC_LABEL>P_061</MAC_LABEL> <MAC>az4FMuLbvrPz720bBeKWz3c+zBh6MsKVo4nJEW96B04=</MAC> <COUNTER>217</COUNTER> </TRANSACTION> 9/10/2015 11:29:16 AM [0-3-1-SecondaryPortStatus.cs-57] <RESPONSE> <RESPONSE_TEXT>Operation SUCCESSFUL</RESPONSE_TEXT> <RESULT>OK</RESULT> <RESULT_CODE>-1</RESULT_CODE> <TERMINATION_STATUS>SUCCESS</TERMINATION_STATUS> <COUNTER>217</COUNTER> <SECONDARY_DATA>10</SECONDARY_DATA> <SERIAL_NUMBER>285498613</SERIAL_NUMBER> </RESPONSE> 9/10/2015 11:29:16 AM [0-1-1-LandingPage.xaml.cs-49] POS opened However when running the nxlog.conf for this I am getting the following error  2015-09-15 08:00:43 ERROR couldn't parse expression at line 12, character 13 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; invalid character: '\' I am unsure what i need to do in order to get this correct; does anyone have any insight or resources I should further explore. Is there a REGEX specific doc for NXLOG? 

chris.ried created
Replies: 1
View post »
last updated
multiline extension not getting the endline regex condition
Hi, I am trying to parse a log4net file into json. Here's my sample log4net: ---------------- 2015-01-27 01:06:18,859 [7] ERROR Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider [(null)] - Get taxonomy Type Failed for Tools 2015-01-27 06:34:31,051 [26] ERROR www.Status404 [(null)] - ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20     UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99 Safari/537.36     HostAddress: 192.168.10.2     RequestUrl: /ErrorPages/404.aspx     MachineName: QA01     Raw Url:/undefined/     Referrer: http://qa1.www.something.com/toolset.aspx 2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled   Now I am using xm_multiline to capture each log entries. ---------------- <Extension multiline>     Module        xm_multiline     HeaderLine    /^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/     EndLine        /\r?\n\r?\n^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}/ </Extension> I use a regex to capture the timestamp as the header then I use a regex to capture twice newline then the next timestamp as endline. However it still treat the second and last entry as ONE log entry. Here's the output: ---------------- {     "EventReceivedTime":"2015-01-27 01:06:35",   "SourceModuleName":"log4net",   "SourceModuleType":"im_file",   "time":"2015-01-27 01:06:18,859",   "thread":"7",   "level":"ERROR",   "logger":"Web.Cms.Content.Base.Taxonomy.TaxonomyDetectionProvider",   "ndc":"(null)",   "message":"Get taxonomy Type Failed for Tools"}{     "EventReceivedTime":"2015-01-27 06:34:35",   "SourceModuleName":"log4net",   "SourceModuleType":"im_file",   "time":"2015-01-27 06:34:31,051",   "thread":"26",   "level":"ERROR",   "logger":"www.Status404",   "ndc":"(null)",   "message":"  ErrorId: 20150127_102b01c6-3208-48c5-8c8b-ae4f92cf2b20\r\n  UserAgent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.99  Safari/537.36\r\n  HostAddress: 192.168.10.2\r\n  RequestUrl: /ErrorPages/404.aspx\r\n  MachineName: QA01\r\n   Raw Url:/undefined/\r\n  Referrer: http://qa1.www.something.com/toolset.aspx\r\n\r\n2015-01-27 06:34:33,270 [26] DEBUG Web.Caching.Core.CacheManagerBase [(null)] - Custom CacheProvider:Web.Caching.Core.AppFabricCacheManager,Web.Caching.Core Disabled"} I used this to produce that output: ---------------- Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- (.*)/s \                 { \                     $time = $1; \                     $thread = $2; \                     $level = $3; \                     $logger = $4; \                     $ndc = $5; \                     $message = $6; \                     to_json(); \                 } \                 else \                 { \                     drop(); \                 }     I've also tried to tweak it by using this to avoid the combining the last two entries as one. However I am not able to get the last entry anymore. ---------------- Exec        if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2},\d{3}) \[(\S+)\] (\S+) (\S+) \[(\S+)\] \- ([\s\S]*?)(\r?\n\r?\n|$)/ \                 { \                     $time = $1; \                     $thread = $2; \                     $level = $3; \                     $logger = $4; \                     $ndc = $5; \                     $message = $6; \                     to_json(); \                 } \                 else \                 { \                     drop(); \                 }​

mvf.right created
Replies: 1
View post »
last updated