multiline in logs
Hi I have been trying to get this to work for 2 days but its not happening! i have a log that sometimes has java call stack errors (multiline) i ship these to graylog using nxlog. As you have guesed i have one log per line. So serched and found the xm_multiline module . Great but i cant get it to work as expected. I have tried many variations of regex but the output i get is not good . To test Im using txt files in and out.. From time to time I point to graylog but its never correct.
Sample data:(stack shortened so not to fill up this forum message) 09:56:29.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 09:56:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 11:49:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error java.lang.NullPointerException: null at org.apache.xmlbeans.impl.store.Cur.listRemove(Cur.java:2788) at org.apache.xmlbeans.impl.store.Locale.embedCurs(Locale.java:2796) at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1953) at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1846) at org.apache.xmlbeans.impl.store.Cur.createHelper(Cur.java:287) at org.apache.xmlbeans.impl.store.Cur.createAttr(Cur.java:211) at org.apache.xmlbeans.impl.store.Xobj.setAttr(Xobj.java:559) at org.apache.xmlbeans.impl.store.Xobj.add_attribute_user(Xobj.java:2272) at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTCellImpl.setR(Unknown Source) at org.apache.poi.xssf.usermodel.XSSFCell.setCellNum(XSSFCell.java:911) at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:226) at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:200) at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:73) at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:60) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticleData(ExcelService.java:441) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:402) at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:372) at com.lexisnexis.bis.act.export.service.ExcelService.generateSheetArticles(ExcelService.java:242) at com.lexisnexis.bis.act.export.service.ExcelService.generateExport(ExcelService.java:212) 11:42:30.278 [TEST test2] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 11:42:30.278 [TEST test2] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error
NXLOG Config Panic Soft NoCache TRUE #Loglevel ERROR Loglevel INFO
define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
<Extension _gelf> Module xm_gelf #Avoid truncation of the short_message field to 64 characters. ShortMessageLength 65536 </Extension>
<Extension multi> Module xm_multiline HeaderLine /^\d\d:\d\d:\d\d/ </Extension>
#ACT <Input in> Module im_file FILE "C:\test.log" InputType multi ReadFromLast False SavePos False </Input>
<Output gelf> Module om_tcp Host 192.168.xx Port 12222 OutputType GELF_TCP <Exec> # These fields are needed for Graylog $gl2_source_collector = '72f7289a-7cf5-4ae5-af7f-0428371459f4'; $collector_node_id = hostname(); </Exec> </Output>
<Output out> Module om_file File "C:\test_out.txt" </Output>
<Route 3> Path in => out </Route>
OUTPUT: 09:56:29.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 㤀㨀㔀㘀㨀㌀ ⸀㈀㜀㠀 嬀吀䔀匀吀 吀䔀匀吀崀 䔀刀刀伀刀 漀⸀稀⸀瀀⸀猀瀀爀椀渀最⸀挀漀洀洀漀渀⸀䄀搀瘀椀挀攀吀爀愀椀琀猀 ⴀ 䤀渀琀攀爀渀愀氀 匀攀爀瘀攀爀 䔀爀爀漀爀ഀഀ 11:49:30.278 [TEST TEST] ERROR o.z.p.spring.common.AdviceTraits - Internal Server Error 樀愀瘀愀⸀氀愀渀最⸀一甀氀氀倀漀椀渀琀攀爀䔀砀挀攀瀀琀椀漀渀㨀 渀甀氀氀ഀഀ at org.apache.xmlbeans.impl.store.Cur.listRemove(Cur.java:2788) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀砀洀氀戀攀愀渀猀⸀椀洀瀀氀⸀猀琀漀爀攀⸀䰀漀挀愀氀攀⸀攀洀戀攀搀䌀甀爀猀⠀䰀漀挀愀氀攀⸀樀愀瘀愀㨀㈀㜀㤀㘀⤀ഀഀ at org.apache.xmlbeans.impl.store.Cur.moveNode(Cur.java:1953) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀砀洀氀戀攀愀渀猀⸀椀洀瀀氀⸀猀琀漀爀攀⸀䌀甀爀⸀洀漀瘀攀一漀搀攀⠀䌀甀爀⸀樀愀瘀愀㨀㠀㐀㘀⤀ഀഀ at org.apache.xmlbeans.impl.store.Cur.createHelper(Cur.java:287) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀砀洀氀戀攀愀渀猀⸀椀洀瀀氀⸀猀琀漀爀攀⸀䌀甀爀⸀挀爀攀愀琀攀䄀琀琀爀⠀䌀甀爀⸀樀愀瘀愀㨀㈀⤀ഀഀ at org.apache.xmlbeans.impl.store.Xobj.setAttr(Xobj.java:559) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀砀洀氀戀攀愀渀猀⸀椀洀瀀氀⸀猀琀漀爀攀⸀堀漀戀樀⸀愀搀搀开愀琀琀爀椀戀甀琀攀开甀猀攀爀⠀堀漀戀樀⸀樀愀瘀愀㨀㈀㈀㜀㈀⤀ഀഀ at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTCellImpl.setR(Unknown Source) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀瀀漀椀⸀砀猀猀昀⸀甀猀攀爀洀漀搀攀氀⸀堀匀匀䘀䌀攀氀氀⸀猀攀琀䌀攀氀氀一甀洀⠀堀匀匀䘀䌀攀氀氀⸀樀愀瘀愀㨀㤀⤀ഀഀ at org.apache.poi.xssf.usermodel.XSSFRow.createCell(XSSFRow.java:226) ऀ愀琀 漀爀最⸀愀瀀愀挀栀攀⸀瀀漀椀⸀砀猀猀昀⸀甀猀攀爀洀漀搀攀氀⸀堀匀匀䘀刀漀眀⸀挀爀攀愀琀攀䌀攀氀氀⠀堀匀匀䘀刀漀眀⸀樀愀瘀愀㨀㈀ ⤀ഀഀ at com.lexisnexis.bis.act.export.util.ExcelUtil.generateRowCell(ExcelUtil.java:73) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀攀砀瀀漀爀琀⸀甀琀椀氀⸀䔀砀挀攀氀唀琀椀氀⸀最攀渀攀爀愀琀攀刀漀眀䌀攀氀氀⠀䔀砀挀攀氀唀琀椀氀⸀樀愀瘀愀㨀㘀 ⤀ഀഀ at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticleData(ExcelService.java:441) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀攀砀瀀漀爀琀⸀猀攀爀瘀椀挀攀⸀䔀砀挀攀氀匀攀爀瘀椀挀攀⸀最攀渀攀爀愀琀攀刀漀眀䄀爀琀椀挀氀攀⠀䔀砀挀攀氀匀攀爀瘀椀挀攀⸀樀愀瘀愀㨀㐀 ㈀⤀ഀഀ at com.lexisnexis.bis.act.export.service.ExcelService.generateRowArticle(ExcelService.java:372) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀攀砀瀀漀爀琀⸀猀攀爀瘀椀挀攀⸀䔀砀挀攀氀匀攀爀瘀椀挀攀⸀最攀渀攀爀愀琀攀匀栀攀攀琀䄀爀琀椀挀氀攀猀⠀䔀砀挀攀氀匀攀爀瘀椀挀攀⸀樀愀瘀愀㨀㈀㐀㈀⤀ഀഀ at com.lexisnexis.bis.act.export.service.ExcelService.generateExport(ExcelService.java:212) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀攀砀瀀漀爀琀⸀猀攀爀瘀椀挀攀⸀䔀砀挀攀氀匀攀爀瘀椀挀攀␀␀䘀愀猀琀䌀氀愀猀猀䈀礀匀瀀爀椀渀最䌀䜀䰀䤀䈀␀␀戀㔀㌀㤀戀㌀㐀⸀椀渀瘀漀欀攀⠀㰀最攀渀攀爀愀琀攀搀㸀⤀ഀഀ at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀愀漀瀀⸀昀爀愀洀攀眀漀爀欀⸀䌀最氀椀戀䄀漀瀀倀爀漀砀礀␀䌀最氀椀戀䴀攀琀栀漀搀䤀渀瘀漀挀愀琀椀漀渀⸀椀渀瘀漀欀攀䨀漀椀渀瀀漀椀渀琀⠀䌀最氀椀戀䄀漀瀀倀爀漀砀礀⸀樀愀瘀愀㨀㜀㜀⤀ഀഀ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀愀漀瀀⸀昀爀愀洀攀眀漀爀欀⸀䌀最氀椀戀䄀漀瀀倀爀漀砀礀␀䌀最氀椀戀䴀攀琀栀漀搀䤀渀瘀漀挀愀琀椀漀渀⸀瀀爀漀挀攀攀搀⠀䌀最氀椀戀䄀漀瀀倀爀漀砀礀⸀樀愀瘀愀㨀㜀㐀㤀⤀ഀഀ at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀琀爀愀渀猀愀挀琀椀漀渀⸀椀渀琀攀爀挀攀瀀琀漀爀⸀吀爀愀渀猀愀挀琀椀漀渀䤀渀琀攀爀挀攀瀀琀漀爀⸀椀渀瘀漀欀攀⠀吀爀愀渀猀愀挀琀椀漀渀䤀渀琀攀爀挀攀瀀琀漀爀⸀樀愀瘀愀㨀㠀⤀ഀഀ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀愀漀瀀⸀昀爀愀洀攀眀漀爀欀⸀䌀最氀椀戀䄀漀瀀倀爀漀砀礀␀䌀最氀椀戀䴀攀琀栀漀搀䤀渀瘀漀挀愀琀椀漀渀⸀瀀爀漀挀攀攀搀⠀䌀最氀椀戀䄀漀瀀倀爀漀砀礀⸀樀愀瘀愀㨀㜀㐀㤀⤀ഀഀ at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀攀砀瀀漀爀琀⸀猀攀爀瘀椀挀攀⸀䔀砀挀攀氀匀攀爀瘀椀挀攀␀␀䔀渀栀愀渀挀攀爀䈀礀匀瀀爀椀渀最䌀䜀䰀䤀䈀␀␀㈀㌀攀㈀㤀㘀㤀⸀最攀渀攀爀愀琀攀䔀砀瀀漀爀琀⠀㰀最攀渀攀爀愀琀攀搀㸀⤀ഀഀ at com.lexisnexis.bis.act.service.ArticleService.getExcelExport(ArticleService.java:2200) ऀ愀琀 挀漀洀⸀氀攀砀椀猀渀攀砀椀猀⸀戀椀猀⸀愀挀琀⸀猀攀爀瘀椀挀攀⸀䄀爀琀椀挀氀攀匀攀爀瘀椀挀攀␀␀䘀愀猀琀䌀氀愀猀猀䈀礀匀瀀爀椀渀最䌀䜀䰀䤀䈀␀␀昀㌀㜀㈀㌀攀昀㔀⸀椀渀瘀漀欀攀⠀㰀最攀渀攀爀愀琀攀搀㸀⤀ഀഀ at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀愀漀瀀⸀昀爀愀洀攀眀漀爀欀⸀䌀最氀椀戀䄀漀瀀倀爀漀砀礀␀䌀最氀椀戀䴀攀琀栀漀搀䤀渀瘀漀挀愀琀椀漀渀⸀椀渀瘀漀欀攀䨀漀椀渀瀀漀椀渀琀⠀䌀最氀椀戀䄀漀瀀倀爀漀砀礀⸀樀愀瘀愀㨀㜀㜀⤀ഀഀ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀愀漀瀀⸀昀爀愀洀攀眀漀爀欀⸀䌀最氀椀戀䄀漀瀀倀爀漀砀礀␀䌀最氀椀戀䴀攀琀栀漀搀䤀渀瘀漀挀愀琀椀漀渀⸀瀀爀漀挀攀攀搀⠀䌀最氀椀戀䄀漀瀀倀爀漀砀礀⸀樀愀瘀愀㨀㜀㐀㤀⤀ഀഀ at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) ऀ愀琀 漀爀最⸀猀瀀爀椀渀最昀爀愀洀攀眀漀爀欀⸀琀爀愀渀猀愀挀琀椀漀渀⸀椀渀琀攀爀挀攀瀀琀漀爀⸀吀爀愀渀猀愀挀琀椀漀渀䤀渀琀攀爀挀攀瀀琀漀爀⸀椀渀瘀漀欀攀⠀吀爀愀渀猀愀挀琀椀漀渀䤀渀琀攀爀挀攀瀀琀漀爀⸀樀愀瘀愀㨀㠀⤀ഀഀ at org.springframework.aop.framework.R
Hi Peter,
The current configuration shown in the description was verified and works properly. The sample data was used and the output generated five (5) events converted to GELF format.
Regards,
Carlos.