multiline in logs
Hi I have been trying to get this to work for 2 days but its not happening! i have a log that sometimes has java call stack errors (multiline) i ship these to graylog using nxlog. As you have guesed i have one log per line. So serched and found the xm_multiline module . Great but i cant get it to work as expected. I have tried many variations of regex but the output i get is not good . To test Im using txt files in and out.. From time to time I point to graylog but its never correct.
Sample data:(stack shortened so not to fill up this forum message) 09:56:29.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 09:56:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 11:49:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error java.lang.NullPointerException: null at org.apache.xmlbeans.impl.store.Cur.listRemove(Cur.java:2788) at org.apache.xmlbeans.impl.store.Locale.embedCurs(Locale.java:2796) at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1953) at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1846) at org.apache.xmlbeans.impl.store.Cur.createHelper(Cur.java:287) at org.apache.xmlbeans.impl.store.Cur.createAttr(Cur.java:211) at org.apache.xmlbeans.impl.store.Xobj.setAttr(Xobj.java:559) at org.apache.xmlbeans.impl.store.Xobj.add_attribute_user(Xobj.java:2272) at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTCellImpl.setR(Unknown Source) at org.apache.poi.xssf.usermodel.XSSFCell.setCellNum(XSSFCell.java:911) at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:226) at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:200) at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:73) at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:60) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticleData(ExcelService.java:441) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:402) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:372) at com.lexisnexis.bis.act.export.service.ExcelService.generateSheetArticles(ExcelService.java:242) at com.lexisnexis.bis.act.export.service.ExcelService.generateExport(ExcelService.java:212) 11:42:30.278 [TEST test2] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 11:42:30.278 [TEST test2] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error
NXLOG Config Panic Soft NoCache TRUE #Loglevel ERROR Loglevel INFO
define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
<Extension _gelf> Module xm_gelf #Avoid truncation of the short_message field to 64 characters. ShortMessageLength 65536 </Extension>
<Extension multi> Module xm_multiline HeaderLine /^\d\d:\d\d:\d\d/ </Extension>
#ACT <Input in> Module im_file FILE "C:\test.log" InputType multi ReadFromLast False SavePos False </Input>
<Output gelf> Module om_tcp Host 192.168.xx Port 12222 OutputType GELF_TCP <Exec> # These fields are needed for Graylog $gl2_source_collector = '72f7289a-7cf5-4ae5-af7f-0428371459f4'; $collector_node_id = hostname(); </Exec> </Output>
<Output out> Module om_file File "C:\test_out.txt" </Output>
<Route 3> Path in => out </Route>
OUTPUT: 09:56:29.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Errorΰ΄ ΰ¨γγ€γ¨γγγ¨γγβΈγγγ βε¬εδεεβεδεεε΄βδεεδΌεβζΌβΈη¨βΈηβΈηηηζ€ζΈζβΈζζΌζ΄ζ΄ζΌζΈβΈδζηζ€ζζεηζζ€ηηββ΄βδ€ζΈηζηζΈζζ°βεζηηζηβδηηζΌηΰ΄ΰ¨ΰ΄ 11:49:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Errorΰ΄ ΰ¨ζ¨ζηζβΈζ°ζζΈζβΈδΈηζ°ζ°εζΌζ€ζΈηζηδη ζζηηζ€ζΌζΈγ¨βζΈηζ°ζ°ΰ΄ΰ¨ΰ΄ at org.apache.xmlbeans.impl.store.Cur.listRemove(Cur.java:2788)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈη ζ΄ζ°ζζζζΈηβΈζ€ζ΄ηζ°βΈηηζΌηζβΈδ°ζΌζζζ°ζβΈζζ΄ζζζδηηηβ δ°ζΌζζζ°ζβΈζ¨ζηζγ¨γγγ€γβ€ΰ΄ΰ¨ΰ΄ at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1953)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈη ζ΄ζ°ζζζζΈηβΈζ€ζ΄ηζ°βΈηηζΌηζβΈδηηβΈζ΄ζΌηζδΈζΌζζβ δηηβΈζ¨ζηζγ¨γγ γγβ€ΰ΄ΰ¨ΰ΄ at org.apache.xmlbeans.impl.store.Cur.createHelper(Cur.java:287)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈη ζ΄ζ°ζζζζΈηβΈζ€ζ΄ηζ°βΈηηζΌηζβΈδηηβΈζηζζηζδηηηβ δηηβΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at org.apache.xmlbeans.impl.store.Xobj.setAttr(Xobj.java:559)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈη ζ΄ζ°ζζζζΈηβΈζ€ζ΄ηζ°βΈηηζΌηζβΈε ζΌζζ¨βΈζζζεΌζηηηζ€ζηηζεΌηηζηβ ε ζΌζζ¨βΈζ¨ζηζγ¨γγγγβ€ΰ΄ΰ¨ΰ΄ at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTCellImpl.setR(Unknown Source)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈηζΌζ€βΈη ηηζβΈηηζηζ΄ζΌζζζ°βΈε εεδδζζ°ζ°βΈηζηδζζ°ζ°δΈηζ΄β ε εεδδζζ°ζ°βΈζ¨ζηζγ¨γ€γγβ€ΰ΄ΰ¨ΰ΄ at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:226)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈζηζζζ ζβΈηζΌζ€βΈη ηηζβΈηηζηζ΄ζΌζζζ°βΈε εεδεζΌηβΈζηζζηζδζζ°ζ°β ε εεδεζΌηβΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:73)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈζη ηζΌηηβΈηηζ€ζ°βΈδη ζζζ°εηζ€ζ°βΈζζζΈζηζηζεζΌηδζζ°ζ°β δη ζζζ°εηζ€ζ°βΈζ¨ζηζγ¨γγβ€ΰ΄ΰ¨ΰ΄ at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticleData(ExcelService.java:441)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈζη ηζΌηηβΈηζηηζ€ζζβΈδη ζζζ°εζηηζ€ζζβΈζζζΈζηζηζεζΌηδηηζ€ζζ°ζβ δη ζζζ°εζηηζ€ζζβΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:372)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈζη ηζΌηηβΈηζηηζ€ζζβΈδη ζζζ°εζηηζ€ζζβΈζζζΈζηζηζεζ ζζηδηηζ€ζζ°ζηβ δη ζζζ°εζηηζ€ζζβΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at com.lexisnexis.bis.act.export.service.ExcelService.generateExport(ExcelService.java:212)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈζη ηζΌηηβΈηζηηζ€ζζβΈδη ζζζ°εζηηζ€ζζββδζηηδζ°ζηηδη€εηηζ€ζΈζδδδ°δ€δββζγγγ€ζγγγβΈζ€ζΈηζΌζ¬ζβ γ°ζζζΈζηζηζζγΈβ€ΰ΄ΰ¨ΰ΄ at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈζζΌηβΈζηζζ΄ζηζΌηζ¬βΈδζζ°ζ€ζδζΌηεηζΌη η€βδζζ°ζ€ζδ΄ζηζ ζΌζδ€ζΈηζΌζζηζ€ζΌζΈβΈζ€ζΈηζΌζ¬ζδ¨ζΌζ€ζΈηζΌζ€ζΈηβ δζζ°ζ€ζδζΌηεηζΌη η€βΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈζζΌηβΈζηζζ΄ζηζΌηζ¬βΈδζζ°ζ€ζδζΌηεηζΌη η€βδζζ°ζ€ζδ΄ζηζ ζΌζδ€ζΈηζΌζζηζ€ζΌζΈβΈηηζΌζζζζβ δζζ°ζ€ζδζΌηεηζΌη η€βΈζ¨ζηζγ¨γγγ€β€ΰ΄ΰ¨ΰ΄ at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈηηζζΈηζζηζ€ζΌζΈβΈζ€ζΈηζηζζηηζΌηβΈεηζζΈηζζηζ€ζΌζΈδ€ζΈηζηζζηηζΌηβΈζ€ζΈηζΌζ¬ζβ εηζζΈηζζηζ€ζΌζΈδ€ζΈηζηζζηηζΌηβΈζ¨ζηζγ¨γγγ β€ΰ΄ΰ¨ΰ΄ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈζζΌηβΈζηζζ΄ζηζΌηζ¬βΈδζζ°ζ€ζδζΌηεηζΌη η€βδζζ°ζ€ζδ΄ζηζ ζΌζδ€ζΈηζΌζζηζ€ζΌζΈβΈηηζΌζζζζβ δζζ°ζ€ζδζΌηεηζΌη η€βΈζ¨ζηζγ¨γγγ€β€ΰ΄ΰ¨ΰ΄ at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈζη ηζΌηηβΈηζηηζ€ζζβΈδη ζζζ°εζηηζ€ζζββδζΈζ ζζΈζζηδη€εηηζ€ζΈζδδδ°δ€δββγγγζγγ€γγ€βΈζζζΈζηζηζδη ηζΌηηβ γ°ζζζΈζηζηζζγΈβ€ΰ΄ΰ¨ΰ΄ at com.lexisnexis.bis.act.service.ArticleService.getExcelExport(ArticleService.java:2200)ΰ΄ ΰ¨ΰ€ζηβζζΌζ΄βΈζ°ζη ζ€ηζΈζη ζ€ηβΈζζ€ηβΈζζηβΈηζηηζ€ζζβΈδηηζ€ζζ°ζεζηηζ€ζζββδζηηδζ°ζηηδη€εηηζ€ζΈζδδδ°δ€δββζγγγγζζγβΈζ€ζΈηζΌζ¬ζβ γ°ζζζΈζηζηζζγΈβ€ΰ΄ΰ¨ΰ΄ at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈζζΌηβΈζηζζ΄ζηζΌηζ¬βΈδζζ°ζ€ζδζΌηεηζΌη η€βδζζ°ζ€ζδ΄ζηζ ζΌζδ€ζΈηζΌζζηζ€ζΌζΈβΈζ€ζΈηζΌζ¬ζδ¨ζΌζ€ζΈηζΌζ€ζΈηβ δζζ°ζ€ζδζΌηεηζΌη η€βΈζ¨ζηζγ¨γγγβ€ΰ΄ΰ¨ΰ΄ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈζζΌηβΈζηζζ΄ζηζΌηζ¬βΈδζζ°ζ€ζδζΌηεηζΌη η€βδζζ°ζ€ζδ΄ζηζ ζΌζδ€ζΈηζΌζζηζ€ζΌζΈβΈηηζΌζζζζβ δζζ°ζ€ζδζΌηεηζΌη η€βΈζ¨ζηζγ¨γγγ€β€ΰ΄ΰ¨ΰ΄ at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366)ΰ΄ ΰ¨ΰ€ζηβζΌηζβΈηηηζ€ζΈζζηζζ΄ζηζΌηζ¬βΈηηζζΈηζζηζ€ζΌζΈβΈζ€ζΈηζηζζηηζΌηβΈεηζζΈηζζηζ€ζΌζΈδ€ζΈηζηζζηηζΌηβΈζ€ζΈηζΌζ¬ζβ εηζζΈηζζηζ€ζΌζΈδ€ζΈηζηζζηηζΌηβΈζ¨ζηζγ¨γγγ β€ΰ΄ΰ¨ΰ΄ at org.springframework.aop.framework.R
Hi Peter,
The current configuration shown in the description was verified and works properly. The sample data was used and the output generated five (5) events converted to GELF format.
Regards,
Carlos.