nxlog to kafka
Tags:
#1
dsta
Is there a way to control the serialisation nxlog does on the logs sent to kafka via om_kafka? Firstly i do not know how they serialise the messages sent to kafka as a result when I read these messages from kafka, i end up facing de-serialisation errors.
#1
dsta
Is there a way to control the serialisation nxlog does on the logs sent to kafka via om_kafka? Firstly i do not know how they serialise the messages sent to kafka as a result when I read these messages from kafka, i end up facing de-serialisation errors.
Hi,
By default the output module will take the contents of $raw_event and send it via kafka
That field is usually populated by the input module. You have control over its format using various to_format() procedures. These take the internal record's fields and convert them to the desired format.
For example
Exec to_json();
would take the record and dump it into $raw_event in JSON format. This in turn will be transmitted by om_kafka.
I hope this helps
Gabor
