Our application writes logs in JSON format so it's quite straightforward to send them to Elasticsearch using om_http module. However we need to enrich JSON logs with additional information like application name. I was searching for the solution and found that I could do the following:

<Output elasticsearch>
    Module      om_http
    URL            (server_url)
    ContentType application/json
    Exec        set_http_request_path(strftime(now(), "/test-%Y.%m/log/"));
    Exec        parse_json(); $Application="MyApp"; to_json();

The last line in the output specification make sure the json payload is first parsed and then generated again, enriched with a new field "Application". I wonder if this is a right approach or there are other alternatives.

Thanks in advance


AskedMarch 24, 2015 - 4:18pm

Answer (1)

Generally this should be fine. Note that parse_json() does not support nested JSON, only flat key-values.

The NXLog Enterprise Edition has an om_elasticsearch module that uses the bulk API. If you have a lot of data to load this will be a lot faster than om_http which submits a single event per http request.

Comments (2)