Hi all,

I want to send this text-logfiles to a syslog service. Now I´m stuck with xm_kvp, because it manipualtes the data in a way I don´t expect. The data looks as follows:

"srv1","IAS",08/01/2020,10:33:39,1,"123\xyz","123\xyz","dc-aa-94-96-52-70:WLAN-XYZ","f8-aa-4e-24-bc-7c",,,"wlc01","1.1.1.1",13,0,"1.1.1.1","wlc01",,,19,,,2,5,"wlan-xyz-01",0,"311 1 4.4.4.4 03/25/2020 14:12:43 689",,,,,,,,,"5e8427b3/f8:aa:4e:24:bc:7c/133777",,,,,,,,,13,6,,,,"49",,,,,,,,,,,"Wireless-hj",1,,,,


So there is comma separated values, without fieldnames. I want to create KVP values and send it to syslog via UDP. Lets focus on the formatted data.




So this is my code:

#####################
<Extension csv1>
Module xm_csv
Fields $ComputerName, $ServiceName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Callback-Number, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-Vendor, $Client-IP-Address, $Client-Friendly-Name, $Event-Timestamp, $Port-Limit, $NAS-Port-Type, $Connect-Info, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Class, $Session-Timeout, $Idle-Timeout, $Termination-Action, $EAP-Friendly-Name, $Acct-Status-Type, $Acct-Delay-Time, $Acct-Input-Octets, $Acct-Output-Octets, $Acct-Session-Id, $Acct-Authentic, $Acct-Session-Time, $Acct-Input-Packets, $Acct-Output-Packets, $Acct-Terminate-Cause, $Acct-Multi-Ssn-ID, $Acct-Link-Count, $Acct-Interim-Interval, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Client-Endpt, $Tunnel-Server-Endpt, $Acct-Tunnel-Conn, $Tunnel-Pvt-Group-ID, $Tunnel-Assignment-ID, $Tunnel-Preference, $MS-Acct-Auth-Type, $MS-Acct-EAP-Type, $MS-RAS-Version, $MS-RAS-Vendor, $MS-CHAP-Error, $MS-CHAP-Domain, $MS-MPPE-Encryption-Types, $MS-MPPE-Encryption-Policy, $Proxy-Policy-Name, $Provider-Type, $Provider-Name, $Remote-Server-Address, $MS-RAS-Client-Name, $MS-RAS-Client-Version
#EscapeControl FALSE
Delimiter ,
</Extension>
<Extension csv2>
Module xm_csv
Fields $ComputerName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-IP-Address, $Client-Friendly-Name, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Pvt-Group-ID
Delimiter ;
#EscapeControl False
EscapeChar \n
</Extension>
<Extension kvp1>
Module xm_kvp
#Delimiter ''
#ValueQuoteChar "
QuoteMethod All
#KVDelimiter =
EscapeChar \n
KVPDelimiter ;
IncludeHiddenFields False
</Extension>
<Extension kvp2>
Module xm_kvp
KVPDelimiter ;
</Extension>
<Input in>
Module im_file
File "d:\nxlog\IN2004.log"
InputType LineBased

PollInterval 1
ReadFromLast FALSE
SavePos FALSE
<Exec>
csv1->parse_csv();
if not defined $number $number = 0;
csv2->to_csv();
kvp1->to_kvp();
delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);
</Exec>
</Input>

#####################

Till csv2->to_csv();, it works fine and the output is as expected. Values are now semicolon seperated and surrounded by quotation marks "". This is what I want.

But know, when kvp1->to_kvp(); is also active, quotes are removed from all values but values with spaces in it. I do not want to change the quotes surrounding the values.

The result looks like this:

EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;ComputerName=srv1;ServiceName=IAS;Record-Date=08/01/2020;Record-Time=10:33:39;Packet-Type=1;User-Name=123\xyz;Fully-Qualified-Distinguished-Name=123\xyz;Called-Station-ID=dc-aa-94-96-52-70:WLAN-XYZ;Calling-Station-ID=f8-aa-4e-24-bc-7c;NAS-Identifier=wlc01;NAS-IP-Address=1.1.1.1;NAS-Port=13;Client-Vendor=0;Client-IP-Address=1.1.1.1;Client-Friendly-Name=wlc01;NAS-Port-Type=19;Service-Type=2;Authentication-Type=5;Policy-Name=wlan-xyz-01;Reason-Code=0;Class='311 1 4.4.4.4 03/25/2020 14:12:43 689';Acct-Session-Id=5e8427b3/f8:aa:4e:24:bc:7c/133777;Tunnel-Type=13;Tunnel-Medium-Type=6;Tunnel-Pvt-Group-ID=49;Proxy-Policy-Name=Wireless-hj;Provider-Type=1;number=0;

So does somebody know, why the quotes are beeing removed in general, but only kept for values with spaces in it?

Additionally, I would like to remove these fields: "EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;"

by using

delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);


but it is also not working.

Any ideas?



Thanks!