using xm_multiline

Tags:

#1 kotterbein

Hello-

I'm looking to use the xm_multiline extension to try to concatinate log messages that all fall under the same headerline, but to this point have had little luck.  All messages begin with either -E, -A, -W, -I, and all proceeding lines with "at" are part of the same message.

Any assistance would be appreciated.

Using nxlog.conf:

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension json>
    Module      xm_json
</Extension>
 
<Extension syslog>
   Module xm_syslog
</Extension>

<Extension charconv>
    Module    xm_charconv
    AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2
</Extension>

<Extension multiline>
    Module xm_multiline
    Headerline /^-./
</Extension>
 
<Input internal>
   Module im_internal
   Exec  $Message = to_json(); 
</Input>
 
# Watch any file you'd like
<Input bsi_watch>
   Module   im_file
   File     "D:\\ose\\log\\S.SI.*_*_*_*.log"
   SavePos  TRUE
   InputType LineBased
   Exec $source_server = 'Servername'; $source_file = file_name(); $message = $raw_event; to_json();

</Input>

<Output out>
    Module      om_tcp
    Host        6.x.x.x
    Port        5514
</Output>

<Route 1>
   Path internal, si_watch => out
</Route>

example log:


-E 03-25 04:37:16.477 10992 30 (ISE02E_50013) () GTS_ORA Exception while initializing ReferenceData.  OSE.Library.ITF.ITFMessaging.MessageRequestTimeout: Message request timed out: sessId=I.B.ORA_13.3D3B01ECB, reqId=1
   at OSE.Library.RefData.Client.Singleton`1.Get(Originator orig) in D:\OSE_WD_I\OSE\library\ReferenceDataService\RefDataClient\Singleton.cs:line 93
   at OSE.Applications.Options.OrderRoutingSystem.ORS.ResourceManager..ctor() in D:\OSE_WD_IORS\ISE\Applications\Options\ORS\S-ORA\ResourceManager.cs:line 150
   at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init() in D:\OSE_WD_IORS\OSE\Applications\Options\ORS\S-ORA\S-ORA.cs:line 9160
   at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init()
   at OSE.Library.SIFramework.AdapterBase.AdapterBase.SetTraceAndInit()
#2 adm Nxlog ✓
#1 kotterbein
Hello- I'm looking to use the xm_multiline extension to try to concatinate log messages that all fall under the same headerline, but to this point have had little luck.  All messages begin with either -E, -A, -W, -I, and all proceeding lines with "at" are part of the same message. Any assistance would be appreciated. Using nxlog.conf: ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json>     Module      xm_json </Extension>   <Extension syslog>    Module xm_syslog </Extension> <Extension charconv>     Module    xm_charconv     AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2 </Extension> <Extension multiline>     Module xm_multiline     Headerline /^-./ </Extension>   <Input internal>    Module im_internal    Exec  $Message = to_json();  </Input>   # Watch any file you'd like <Input bsi_watch>    Module   im_file    File     "D:\\ose\\log\\S.SI.*_*_*_*.log"    SavePos  TRUE    InputType LineBased    Exec $source_server = 'Servername'; $source_file = file_name(); $message = $raw_event; to_json(); </Input> <Output out>     Module      om_tcp     Host        6.x.x.x     Port        5514 </Output> <Route 1>    Path internal, si_watch => out </Route> example log: -E 03-25 04:37:16.477 10992 30 (ISE02E_50013) () GTS_ORA Exception while initializing ReferenceData. OSE.Library.ITF.ITFMessaging.MessageRequestTimeout: Message request timed out: sessId=I.B.ORA_13.3D3B01ECB, reqId=1 at OSE.Library.RefData.Client.Singleton`1.Get(Originator orig) in D:\OSE_WD_I\OSE\library\ReferenceDataService\RefDataClient\Singleton.cs:line 93 at OSE.Applications.Options.OrderRoutingSystem.ORS.ResourceManager..ctor() in D:\OSE_WD_IORS\ISE\Applications\Options\ORS\S-ORA\ResourceManager.cs:line 150 at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init() in D:\OSE_WD_IORS\OSE\Applications\Options\ORS\S-ORA\S-ORA.cs:line 9160 at OSE.Applications.Options.OrderRoutingSystem.ORS.S_ORA.Init() at OSE.Library.SIFramework.AdapterBase.AdapterBase.SetTraceAndInit()

You need to make use of the multiline parser:

<Input bsi_watch>
   ...
   InputType multiline
   ...
</Input>