Parsing XML Logs

Tags:

#1 monarch684

I have never had to parse XML files with NxLog so I am new at this process. This config file grabs Windows Event logs and some trace files from XML files. The Windows Event logs works as it should. The XML, I cannot get to work. I am not getting any output from the XML.

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _gelf> Module xm_gelf ShortMessageLength 500 </Extension>

<Extension xml> Module xm_xml </Extension>

<Input EventLog_In> # Use 'im_mseventlog' for Windows XP, 2000 and 2003 Module im_msvistalog # Uncomment the following to collect specific event logs only Query <QueryList>
<Query Id="0">
<Select Path="Application"></Select>
<Select Path="System">
</Select>
<Select Path="Security">*</Select>
</Query>
</QueryList> </Input>

<Input IISTrace_In> Module im_file File 'F:\Fileshare\Infrastructure Department\Logs\W3SVC1\fr001307.xml' Exec parse_xml(); </Input>

<Output EventLog_Out> Module om_tcp Host IP SCRUBBED Port 12201

Exec to_syslog_snare();

OutputType	GELF_TCP

</Output>

<Output IISTrace_Out> Module om_file File 'F:\Fileshare\Infrastructure Department\Logs\NxLog\Test\Test.log' </Output>

<Route EventLog> Path EventLog_In => EventLog_Out </Route>

<Route IISTrace> Path IISTrace_In => IISTrace_Out </Route>

#2 KlevinDeactivated Nxlog ✓
#1 monarch684
I have never had to parse XML files with NxLog so I am new at this process. This config file grabs Windows Event logs and some trace files from XML files. The Windows Event logs works as it should. The XML, I cannot get to work. I am not getting any output from the XML. Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _gelf> Module xm_gelf ShortMessageLength 500 </Extension> <Extension xml> Module xm_xml </Extension> <Input EventLog_In> # Use 'im_mseventlog' for Windows XP, 2000 and 2003 Module im_msvistalog # Uncomment the following to collect specific event logs only Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security">*</Select> </Query> </QueryList> </Input> <Input IISTrace_In> Module im_file File 'F:\Fileshare\Infrastructure Department\Logs\W3SVC1\fr001307.xml' Exec parse_xml(); </Input> <Output EventLog_Out> Module om_tcp Host IP SCRUBBED Port 12201 Exec to_syslog_snare(); OutputType GELF_TCP </Output> <Output IISTrace_Out> Module om_file File 'F:\Fileshare\Infrastructure Department\Logs\NxLog\Test\Test.log' </Output> <Route EventLog> Path EventLog_In => EventLog_Out </Route> <Route IISTrace> Path IISTrace_In => IISTrace_Out </Route>

Hello Sir,

More example you can find here

<Extension xml>
    Module    xm_xml
</Extension>

<Input from_file>
    Module    im_file
      File      '/tmp/input'
      Exec    parse_xml();
</Input>

For testing purposes i would suggest to add

SavePos FALSE
ReadFromLast FALSE

In this way every time you restart the NXLog agent the data will be read from start, most probably you are not seeing any data since the agent have already read the xml file.

Also checking the nxlog.log file should help identifying problem if any specific.

Sincerely Klevin