Our application writes logs in JSON format so it's quite straightforward to send them to Elasticsearch using om_http module. However we need to enrich JSON logs with additional information like application name. I was searching for the solution and found that I could do the following:

<Output elasticsearch>
    Module      om_http
    URL            (server_url)
    ContentType application/json
    Exec        set_http_request_path(strftime(now(), "/test-%Y.%m/log/"));
    Exec        parse_json(); $Application="MyApp"; to_json();

The last line in the output specification make sure the json payload is first parsed and then generated again, enriched with a new field "Application". I wonder if this is a right approach or there are other alternatives.

Thanks in advance


AskedMarch 24, 2015 - 4:18pm

Answer (1)

Generally this should be fine. Note that parse_json() does not support nested JSON, only flat key-values.

The NXLog Enterprise Edition has an om_elasticsearch module that uses the bulk API. If you have a lot of data to load this will be a lot faster than om_http which submits a single event per http request.

AnsweredMarch 24, 2015 - 5:19pm

Comments (2)

  • object's picture

    Thank you for the response. I wasn't aware that parse_json didn't support nested JSONs, it's good to know.

    March 30, 2015 - 11:35am
  • object's picture

    BTW, is there any information about NXLog Enterprise Edition pricing? I couldn'd find anything on your site.

    March 30, 2015 - 11:37am