Hi everyone,
The user guide section for the om_http module explains:
Each event is transferred in a single POST request.
https://nxlog.co/documentation/nxlog-user-guide#om_http
When https is used, is the TLS handshake also performed for each event? This seems like it would be quite inefficient.
Are there any plans to support bulk / batch exporting of many events in a single http post, specifically for exporting data to the Splunk HEC?
rsyslog and syslog-ng support batch exporting events to Splunk using the external python script omsplunkhec.py, as explained here:
https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splunk.html
NXLog supports batch exporting events to Elasticsearch using their bulk API:
https://nxlog.co/documentation/nxlog-user-guide#om_elasticsearch
om_http maintains a persistent TCP/TLS connection.
Adding bulk export capabilities to om_http is on the roadmap to implement similar functionality to what om_elasticsearch does.
The recent NXLog EE v4.0 release adds support for python and ruby so the python based solution could also work. Note that with om_exec it is already possible to invoke such scripts with the CE and older EE versions as this is the same as the program() destination in syslog-ng.
