responses
Hello everyone,
I'm new to NXLOG and I'm trying to understand how event creation and forwarding work.
I have a question about the parse_json() function or maybe the to_syslog_ietf() function, I'm not sure.
Is it possible to use parse_json($Message)
then to_syslog_ietf()
to forward $Message
without all the fields obtained with parse_json included in the forwarded syslog event?
Here are more explanations:
- I receive a syslog event and use parse_syslog_ietf()
- I get $Message
which is json formated, so I use parse_json($Message)
- As I want to forward a specific field (let's say MsgToForward) I define $Message = $MsgToForward
- Then use to_syslog_ietf
to forward it through syslog.
In the forwarded event everything is ok, except that all the fields in the json (the original $Message field) are included in the forwarded message.
But I don't understand why because if I put a log_info($raw_event)
after the parse_syslog and after the parse_json, $raw_event has not changed.
So is it to_parse-ietf() function which includes all the fields available? Can we avoid that?
For instance the original event is:
2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe - {"user":"NT AUTHORITY\\SYSTEM","name":"main","programSource":"send-logs","msgToForward":"Send-logs started"};
Then the forwarded message:
2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp" user="NT AUTHORITY\\SYSTEM" name="main" programSource="send-logs" msgToForward="Send-logs started"] Send-logs started
We can see the fields available in the json in the first event (user, name, programSource and msgToForward) are added in the second event. But there are useless here, so I'm trying to have this result :
2020-12-29 03:05:19 INFO <7>1 2020-12-29T03:05:19.692869 admin-PC send-logs 3556 send-logs.exe [NXLOG@14506 MessageSourceAddress="127.0.0.1" EventReceivedTime="2020-12-29 03:05:19" SourceModuleName="inSyslog" SourceModuleType="im_udp"] Send-logs started
Is there a way to avoid parse_json or to_syslog_ietf to add these fields? (The number of fields and their names are not always the same.)
Or should I just not use parse_json and use regexp
instead to get the value of MsgToForward?
My configuration if needed.
<Input inSyslog>
Module im_udp
Host 127.0.0.1
Port 514
</Input>
<Output outSyslog2>
Module om_udp
Host 127.0.0.1
Port 1514
Exec parse_syslog_ietf(); parse_json($Message); $Message = $MsgToForward; to_syslog_ietf();
</Output>
<Route Syslog>
Path inSyslog => outSyslog2
</Route>
Any help is welcomed. Thank you :)
Whok.
Comments (4)
Hello Whok.
You should try and use
$raw_event
instead of$Message
so your code should look like this:Hope it solve your issue.
Best regards, Arch
Hello Arch, thank you for your help :)
Unfortunatly, setting $raw_event instead of $Message was the first thing I tried, but it makes the json fields duplicated. Here is the output:
The json fields are first key=value formated in the information added by
to_syslog_ietf
(I think). Then the whole json is appended. I suppose it re-uses the already created field $Message to create the new syslog event, and as I don't change it, it adds the full json. That's why I was trying to change $Message instead of $raw_event.Again thank you. If you have more ideas, please let me know.
Whok.
Hi Whok,
The
to_syslog_ietf()
procedure is used to generate the$raw_event
field from all fields in the event recordThere is a special module where you can define which fields to keep: xm_rewrite, however this is available only in the EE version.
I see two options:
parse_json()
:Kind regards,
Konstantinos
Hello Konstantinos, Thank you for your help, I used the regexp, but I didn't understand I could use the
delete()
function. Just tried and it worked.Thanks :D
Whok