How parse XML file with multilines (Nessus report)

Tags: xml | nxlog | nessus

#1 ilovegraylog

Hi guys,

I wanted to use NXLog to send data from a XML file into GrayLog.

My first test worked fine with a simple XML file. But with a Nessus Report (in XML) it don't work, because this file is more complexe. Do you have an example configuration nxlog.conf to help me ?

My actual nxlog.conf :

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
Module xm_gelf
</Extension>

<Extension multiline>


    Module	xm_multiline
    HeaderLine	/^<event>/
    EndLine	/^</event>/
</Extension>

<Extension xmlparser>
    Module	xm_xml
</Extension>

<Extension json>
    Module	xm_json
</Extension>

<Input in>
    Module	im_file
    File	"C:\Program Files\Graylog\collector-sidecar\generated\exempleFichier.xml"
    SavePos	FALSE
    ReadFromLast FALSE
    InputType	multiline
    <Exec>
      # Discard everything that doesn't seem to be an xml event   
      if $raw_event !~ /^<event>/ drop();

      # Parse the xml event
      parse_xml();

      # Rewrite some fields 
      $EventTime = parsedate($timestamp);
      delete($timestamp);
      delete($EventReceivedTime);

      # Convert to JSON
      to_json();
    </Exec>
</Input>

<Output out>
    Module      om_udp
    Host        xx.xx.xx.xx
    Port        12201
</Output>

<Route 1>
    Path        in => out
</Route>

And a (verry little) preview of my Nessus File...

<Report name="Scan_Nessus" xmlns:cm="http://www.nessus.org/cm">
<ReportHost name="192.168.1.1"><HostProperties>
<tag name="traceroute-hop-8">192.168.1.1</tag>
<tag name="LastUnauthenticatedResults">1490484150</tag>
<tag name="Credentialed_Scan">false</tag>
<tag name="policy-used">policies_scan</tag>
<tag name="patch-summary-total-cves">1</tag>
<tag name="os">other</tag>
<tag name="system-type">general-purpose</tag>
<tag name="operating-system">CISCO IOS</tag>
<tag name="traceroute-hop-7">?</tag>
<tag name="traceroute-hop-6">192.168.1.1</tag>
<tag name="traceroute-hop-5">192.168.1.2</tag>
<tag name="traceroute-hop-4">192.168.1.3</tag>
<tag name="HOST_END">Sun Mar 26 00:22:30 2017</tag>
<tag name="host-ip">192.168.1.1</tag>
<tag name="HOST_START">Sun Mar 26 00:20:19 2017</tag>
</HostProperties>
<ReportItem port="123" svc_name="ntp" protocol="udp" severity="0" pluginID="10884" pluginName="Network Time Protocol (NTP) Server Detection" pluginFamily="Service detection">
<cpe>cpe:/a:ntp:ntp</cpe>
<description>An NTP server with an insecure configuration is listening on port 123.
It provides information about its version, current date, current time, and possibly system information.</description>
<fname>ntp_open.nasl</fname>
<plugin_modification_date>2017/03/21</plugin_modification_date>
<plugin_name>Network Time Protocol (NTP) Server Detection</plugin_name>
<plugin_publication_date>2015/03/20</plugin_publication_date>
<plugin_type>remote</plugin_type>
<risk_factor>None</risk_factor>
<script_version>$Revision: 1.27 $</script_version>
<see_also>http://www.ntp.org</see_also>
<solution>n/a</solution>
<synopsis>An NTP server with an insecure configuration is listening on the remote host.</synopsis>
<plugin_output>
  Version : unknown
</plugin_output>
</ReportItem>

Thanks !

#2 atmosx
#1 ilovegraylog
Hi guys, I wanted to use NXLog to send data from a XML file into GrayLog. My first test worked fine with a simple XML file. But with a Nessus Report (in XML) it don't work, because this file is more complexe. Do you have an example configuration nxlog.conf to help me ? My actual nxlog.conf : define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> <Extension multiline> Module xm_multiline HeaderLine /^<event>/ EndLine /^</event>/ </Extension> <Extension xmlparser> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "C:\Program Files\Graylog\collector-sidecar\generated\exempleFichier.xml" SavePos FALSE ReadFromLast FALSE InputType multiline <Exec> # Discard everything that doesn't seem to be an xml event if $raw_event !~ /^<event>/ drop(); # Parse the xml event parse_xml(); # Rewrite some fields $EventTime = parsedate($timestamp); delete($timestamp); delete($EventReceivedTime); # Convert to JSON to_json(); </Exec> </Input> <Output out> Module om_udp Host xx.xx.xx.xx Port 12201 </Output> <Route 1> Path in => out </Route> And a (verry little) preview of my Nessus File... <Report name="Scan_Nessus" xmlns:cm="http://www.nessus.org/cm"> <ReportHost name="192.168.1.1"><HostProperties> <tag name="traceroute-hop-8">192.168.1.1</tag> <tag name="LastUnauthenticatedResults">1490484150</tag> <tag name="Credentialed_Scan">false</tag> <tag name="policy-used">policies_scan</tag> <tag name="patch-summary-total-cves">1</tag> <tag name="os">other</tag> <tag name="system-type">general-purpose</tag> <tag name="operating-system">CISCO IOS</tag> <tag name="traceroute-hop-7">?</tag> <tag name="traceroute-hop-6">192.168.1.1</tag> <tag name="traceroute-hop-5">192.168.1.2</tag> <tag name="traceroute-hop-4">192.168.1.3</tag> <tag name="HOST_END">Sun Mar 26 00:22:30 2017</tag> <tag name="host-ip">192.168.1.1</tag> <tag name="HOST_START">Sun Mar 26 00:20:19 2017</tag> </HostProperties> <ReportItem port="123" svc_name="ntp" protocol="udp" severity="0" pluginID="10884" pluginName="Network Time Protocol (NTP) Server Detection" pluginFamily="Service detection"> <cpe>cpe:/a:ntp:ntp</cpe> <description>An NTP server with an insecure configuration is listening on port 123. It provides information about its version, current date, current time, and possibly system information.</description> <fname>ntp_open.nasl</fname> <plugin_modification_date>2017/03/21</plugin_modification_date> <plugin_name>Network Time Protocol (NTP) Server Detection</plugin_name> <plugin_publication_date>2015/03/20</plugin_publication_date> <plugin_type>remote</plugin_type> <risk_factor>None</risk_factor> <script_version>$Revision: 1.27 $</script_version> <see_also>http://www.ntp.org</see_also> <solution>n/a</solution> <synopsis>An NTP server with an insecure configuration is listening on the remote host.</synopsis> <plugin_output>   Version : unknown </plugin_output> </ReportItem> Thanks !

Hello,

The XML supplied does not appear to be valid. Can you give us a complete sample of the XML output you're trying to parse?