Converting XML to syslog

Post a different question
3
responses

Hi,

I'm looking at trying to convert an XML file from one of our filers containing this XML file below (top line is different to rest of the xml) into a syslog output:

`<Events xmlns="http://www.netapp.com/schemas/ONTAP/2007/AuditLog">
<Event><System><Provider Name="NetApp-Security-Auditing" Guid="{3CB2A168-FE19-4A4E-BDAD-DCF422F13473}"/><EventID>4656</EventID><EventName>Open Object</EventName><Version>101.3</Version><Source>CIFS</Source><Level>0</Level><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><Result>Audit Success</Result><TimeCreated SystemTime="2017-12-15T10:34:51.979061000Z"/><Correlation/><Channel>Security</Channel><Computer>server</Computer><ComputerUUID>cf380853-6606-11e6-9638-00a098a5e1db/2fe0edc3-723f-11e7-ab83-00a098a627d4</ComputerUUID><Security/></System><EventData><Data Name="SubjectIP" IPVersion="4">192.168.0.24</Data><Data Name="SubjectUnix" Uid="65534" Gid="65534" Local="false"></Data><Data Name="SubjectUserSid">S-1-5-21-1997283580-3459341067-486214353-122727</Data><Data Name="SubjectUserIsLocal">false</Data><Data Name="SubjectDomainName">Domain</Data><Data Name="SubjectUserName">firstname.lastname</Data><Data Name="ObjectServer">Security</Data><Data Name="ObjectType">Directory</Data><Data Name="HandleID">000000000004cc;00;00000061;2a5f8706</Data><Data Name="ObjectName">(server);/share</Data><Data Name="AccessList">%%4416 %%4423 </Data><Data Name="AccessMask">81</Data><Data Name="DesiredAccess">Read Data; List Directory; Read Attributes; </Data><Data Name="Attributes"></Data></EventData></Event>`

Currently I have the following config but I'm not getting anything sent to the syslog server running on the same box (for testing purposes at present):

define ROOT C:\Program Files (x86)\nxlog

<Extension gelf>
  Module xm_gelf
</Extension>

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension multiline>
    Module  xm_multiline
    HeaderLine  /^<event>/
    EndLine /^</event>/
</Extension>

<Extension xmlparser>
    Module  xm_xml
</Extension>

<Extension json>
    Module  xm_json
</Extension>

<Input in>
    Module  im_file
    File    "C:\\audit.xml"
    SavePos FALSE
    ReadFromLast FALSE
    InputType   multiline
    <Exec>
      # Discard everything that doesn't seem to be an xml event   
      if $raw_event !~ /^<event>/ drop();

      # Parse the xml event
      parse_xml();

      # Rewrite some fields 
      #$EventTime = parsedate($timestamp);
      #delete($timestamp);
      #delete($EventReceivedTime);

      # Convert to JSON
      to_json();
    </Exec>
</Input>

<Output out>
    Module      om_udp
    Host        192.168.0.12
    Port        2548
</Output>

<Route 1>
    Path        in => out
</Route>

Can anyone point me at where I'm going wrong?

Thanks for your help.

AskedDecember 20, 2017 - 9:10pm

Answer (1)

The config with xm_multiline will read XML when the tags are on separate lines like this:

<Event>
  <Foo>foo</Foo>
  <Bar>bar</Bar>
</Event>

In this case it looks like your events are in a single line.

Also looking at the data I see that this is XML extracted from the windows eventlog. You might want to consider using the im_msvistalog module to read that directly. There is a File directive for im_msvistalog in the Enterprise Edition that lets you read .evtx files directly that netapp creates.

AnsweredDecember 22, 2017 - 5:38pm

Comments (2)

  • Callahan's picture

    Hi,

    Due to things beyond my control, I can only receive that event log as an xml file so I am forced to pursue this line in getting NXLog to ingest this XML file. I've adjusted my conf file as shown below. I'm now seeing data landing in my syslog server. However, it's formatted incorrectly. The bulk of the data that I need separated into their own fields ends up in the message field. This is shown below my config.

    define ROOT C:\Program Files (x86)\nxlog

<Extension gelf>
  Module xm_gelf
</Extension>

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
    Module xm_syslog
</Extension>

<Extension xmlparser>
    Module  xm_xml
</Extension>

<Extension json>
    Module  xm_json
</Extension>

<Input in>
    Module im_file
    File    "C:\\ELK\\Import_Logs_Folder\\netapp\\audit.xml"
    SavePos FALSE
    ReadFromLast FALSE
    PollInterval 1
</Input>

<Output out>
    Module      om_tcp
    Host        10.50.20.22
    Port        5519
</Output>

<Route 1>
    Path        in => out
</Route>

Time: Jan 02 16:56:41
IP: 10.50.20.22
Host: <Event><System><Provider
Facility: 
Priority: 
Tag: 
Message: Name="Security-Auditing" Guid="{3CB2A168-FE19-4A4E-BDAD-DCF422F13473}"/><EventID>4656</EventID><EventName>Open Object</EventName><Version>101.3</Version><Source>CIFS</Source><Level>0</Level><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><Result>Audit Success</Result><TimeCreated SystemTime="2017-12-15T10:34:52.681048000Z"/><Correlation/><Channel>Security</Channel><Computer>server\data</Computer><ComputerUUID>cf380853-6606-11e6-9638-00a098a5e1db/2fe0edc3-723f-11e7-ab83-00a098a627d4</ComputerUUID><Security/></System><EventData><Data Name="SubjectIP" IPVersion="4">172.16.0.34</Data><Data Name="SubjectUnix" Uid="65534" Gid="65534" Local="false"></Data><Data Name="SubjectUserSid">S-1-5-21-1997283580-3459341067-486214353-122727</Data><Data Name="SubjectUserIsLocal">false</Data><Data Name="SubjectDomainName">domain</Data><Data Name="SubjectUserName">joe.bloggs</Data><Data Name="ObjectServer">Security</Data><Data Name="ObjectType">Directory</Data><Data Name="HandleID">000000000004cc;00;00004c99;4a277b8a</Data><Data Name="ObjectName">(data_share);/server/Finance</Data><Data Name="AccessList">%%4416 %%4423 </Data><Data Name="AccessMask">81</Data><Data Name="DesiredAccess">Read Data; List Directory; Read Attributes; </Data><Data Name="Attributes"></Data></EventData></Event>

    Could anyone give me a pointer as to how I can divide up the message contents into their component parts?

    Thanks.

    January 2, 2018 - 6:06pm