Strip some data from an XML file

Tags: xml

#1 colaguy44 2 years ago

Nxlog.conf: #NoFreeOnExit TRUE

define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data

define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension xmlparser> Module xm_xml </Extension>

<Extension json> Module xm_json </Extension>

<Input in> Module im_file File "C:\LogFiles\log*.log" InputType LineBased Exec $Message = $raw_event; SavePos TRUE
ReadFromLast TRUE <Exec>

Discard everything that doesn't seem to be an xml event

if $raw_event !~ /^<Event>/ drop(); if $raw_event =~ /^(.+)host(.+)/ drop();

parse_xml();

Convert to JSON

    to_json();

</Exec> </Input>

<Output out> Module om_udp Host yy.xx.xx.xx Port 514 </Output>

Permalink

#2 jeffron Nxlog ✓ 2 years ago

#1 colaguy44

I have an XML that I am trying to strip some data out so it can be pre-processed by software on another machine. XML file is being generated on a windows logging to an XML file. I am at bit of a lost I have tried too many things to list here. What I am trying to do is remove our domain name and our domain email address before it is sent to the machine to be pre-processed. I only want the username. Any records that have host\domain.com don’t need to be sent and I figured out how to drop that data. (number 4). If it helps I am running nxlog-ce-3.0.2284. Here is an example of event: (Removed bunch of xml fields for clarity of this post) 1: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">DOMAIN\username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 2: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 3: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">username@domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> 4: <Event><Timestamp data_type="4">05/25/2022 12:45:43.806</Timestamp><Userid data_type="1">host\domain.com</Userid><IP-Address data_type="3">x.x.x.x</IP-Address><Endtimestamp data_type="5">05/25/2022 12:46:43.806</Endtimestamp> Nxlog.conf: #NoFreeOnExit TRUE define ROOT C:\Program Files\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension xmlparser> Module xm_xml </Extension> <Extension json> Module xm_json </Extension> <Input in> Module im_file File "C:\LogFiles\log*.log" InputType LineBased Exec $Message = $raw_event; SavePos TRUE ReadFromLast TRUE <Exec> Discard everything that doesn't seem to be an xml event if $raw_event !~ /^<Event>/ drop(); if $raw_event =~ /^(.+)host(.+)/ drop(); parse_xml(); Convert to JSON to_json(); </Exec> </Input> <Output out> Module om_udp Host yy.xx.xx.xx Port 514 </Output> <Route 1> Path in => out </Route>

Hi Allen,

Have you considered using the replace function thus?

Exec  $raw_event = replace($raw_event, "DOMAIN", "");

Or simply removing the field thus

Exec delete($Userid);

I hope this helps.

Jeffron