Can we avoid to_syslog_ietf to include all fields in forwarded message

Tags:

#1 Whok1996

Hello everyone,
I'm new to NXLOG and I'm trying to understand how event creation and forwarding work.

I have a question about the parse_json() function or maybe the to_syslog_ietf() function, I'm not sure.
Is it possible to use parse_json($Message) then to_syslog_ietf() to forward $Message without all the fields obtained with parse_json included in the forwarded syslog event?

Here are more explanations:

  • I receive a syslog event and use parse_syslog_ietf()
  • I get $Message which is json formated, so I use parse_json($Message)
  • As I want to forward a specific field (let's say MsgToForward) I define $Message = $MsgToForward
  • Then use to_syslog_ietf to forward it through syslog.

In the forwarded event everything is ok, except that all the fields in the json (the original $Message field) are included in the forwarded message.
But I don't understand why because if I put a log_info($raw_event) after the parse_syslog and after the parse_json, $raw_event has not changed.
So is it to_parse-ietf() function which includes all the fields available? Can we avoid that?

For instance the original event is:

2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe - {"user":"NT AUTHORITY\\SYSTEM","name":"main","programSource":"send-logs","msgToForward":"Send-logs started"};

Then the forwarded message:

2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp" user="NT AUTHORITY\\SYSTEM" name="main" programSource="send-logs" msgToForward="Send-logs started"] Send-logs started

We can see the fields available in the json in the first event (user, name, programSource and msgToForward) are added in the second event. But there are useless here, so I'm trying to have this result :

2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp"] Send-logs started

Is there a way to avoid parse_json or to_syslog_ietf to add these fields? (The number of fields and their names are not always the same.)
Or should I just not use parse_json and use regexp instead to get the value of MsgToForward?

My configuration if needed.

<Input  inSyslog>
  Module      im_udp
  Host        127.0.0.1
  Port        514
</Input>

<Output outSyslog2>
    Module          om_udp
    Host            127.0.0.1
    Port            1514
	  Exec            parse_syslog_ietf(); parse_json($Message); $Message = $MsgToForward; to_syslog_ietf();
</Output>

<Route	Syslog>
    Path        inSyslog => outSyslog2
</Route>

Any help is welcomed. Thank you :)
Whok.

#2 ArkadiyDeactivated Nxlog ✓
#1 Whok1996
Hello everyone, I'm new to NXLOG and I'm trying to understand how event creation and forwarding work. I have a question about the parse_json() function or maybe the to_syslog_ietf() function, I'm not sure. Is it possible to use parse_json($Message) then to_syslog_ietf() to forward $Message without all the fields obtained with parse_json included in the forwarded syslog event? Here are more explanations: I receive a syslog event and use parse_syslog_ietf() I get $Message which is json formated, so I use parse_json($Message) As I want to forward a specific field (let's say MsgToForward) I define $Message = $MsgToForward Then use to_syslog_ietf to forward it through syslog. In the forwarded event everything is ok, except that all the fields in the json (the original $Message field) are included in the forwarded message. But I don't understand why because if I put a log_info($raw_event) after the parse_syslog and after the parse_json, $raw_event has not changed. So is it to_parse-ietf() function which includes all the fields available? Can we avoid that? For instance the original event is: 2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe - {"user":"NT AUTHORITY\\SYSTEM","name":"main","programSource":"send-logs","msgToForward":"Send-logs started"}; Then the forwarded message: 2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp" user="NT AUTHORITY\\SYSTEM" name="main" programSource="send-logs" msgToForward="Send-logs started"] Send-logs started We can see the fields available in the json in the first event (user, name, programSource and msgToForward) are added in the second event. But there are useless here, so I'm trying to have this result : 2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp"] Send-logs started Is there a way to avoid parse_json or to_syslog_ietf to add these fields? (The number of fields and their names are not always the same.) Or should I just not use parse_json and use regexp instead to get the value of MsgToForward? My configuration if needed. <Input inSyslog> Module im_udp Host 127.0.0.1 Port 514 </Input> <Output outSyslog2> Module om_udp Host 127.0.0.1 Port 1514 Exec parse_syslog_ietf(); parse_json($Message); $Message = $MsgToForward; to_syslog_ietf(); </Output> <Route Syslog> Path inSyslog => outSyslog2 </Route> Any help is welcomed. Thank you :) Whok.

Hello Whok.

You should try and use $raw_event instead of $Message so your code should look like this:

Exec            parse_syslog_ietf(); parse_json($Message); $raw_event = $MsgToForward; to_syslog_ietf();

Hope it solve your issue.
Best regards, Arch