Hi,

I'm using nxlog community edition nxlog-ce-2.9.1347, I have few questions related om_http module.
We have a centralized log server(Log Insight) and 10 application servers. Log Insight server exposes a REST API to post the logs data.
I'm using NXLOG as a log forwarder to Log Insight Server from all my application servers. Please clarify the following questions.

POST URL : http://loginsight:9000/api/v1/messages/ingest/4C4C4544-0037-5910-805A-C4C04F585831

Request Body:

{"messages": [{

 "fields": [
 {"name": "Channel", "content": "Security"},
 {"name": "EventID", "content": "4688"},
 {"name": "EventRecordID", "content": "33311266"},
 {"name": "Keywords", "content": "Audit Success"},
 {"name": "Level", "content": "Information"},
 {"name": "OpCode","content": "Info"},
 {"name": "ProcessID", "content": "4"},
 {"name": "ProviderName", "content": "Microsoft-Windows-Security-Auditing"},
 {"name": "Task", "content": "Process Creation"},
 {"name": "ThreadID", "content": "64"}
 ],
 "text": "A new process has been created.",
 "timestamp": 1396622879241
 }
 ]
}

1. How do I format my log data into the request as mentioned above in NXLOG. Request should be formated into json with fields and the data accordingly.
   I'm able to parse IIS logs,eventlogs and logs from files into JSON but got stuck with calling REST API with request body. Please find my nxlog.conf below.

2. Is it a good idea to directly send the log data to log server via REST API? if not what are the disadvantages. 
3. Does om_http module support retrying logic\buffering just in case REST API is down or doesn't respond.
4. What is the best architecture for sending the logs to centralized server, I see lot of people online follow NXLOG=>LOG STASH=>ELASTICSEARCH or Some Centralized server(Log Insight in my case).
5. Should I use pm_buffer, as my log files will be rotated after certain memory limit is reached in case if the REST API is down or om_http module handles this automatically.
 

 Following configuration reads IIS, event logs and logs from files.

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension json>
    Module      xm_json
</Extension>
  
<Extension w3c>
  # Map the fields from the IIS log file (you can open the IIS log file to see the header and know what fields to map)
    Module      xm_csv
    Fields   $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs(User-Agent), $sc-status, $sc-substatus, $sc-win32-status, $time-taken
    FieldTypes  string, string, string, string, string, string, integer, string, string, string, integer, integer, integer, integer
    Delimiter   ' '
</Extension>

<Extension multiline>
    Module        xm_multiline
    HeaderLine    /^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}.\d{3}/
</Extension>

<Extension charconv>
    Module    xm_charconv
    AutodetectCharsets utf-8, utf-16, utf-32, iso8859-2
</Extension>

<Input eventlog>
    Module      im_msvistalog
    ReadFromLast    True
    Module      im_msvistalog
        Query       <QueryList>\
                        <Query Id="0">\
                            <Select Path="Application">*</Select>\
                            <Select Path="System">*</Select>\
                            <Select Path="Security">*</Select>\
                        </Query>\
                    </QueryList>
</Input>
 
<Input iis>
    Module      im_file
    File 'C:\inetpub\logs\LogFiles\W3SVC1\u_ex*.log'
    ReadFromLast TRUE
    Exec        if $raw_event =~ /^#/ drop();                \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    to_json ();                                  \
                }
</Input>

<Input webconsole>
    Module im_file
    File 'C:\Stash\WebConsole.log' 
    InputType multiline
    SavePos TRUE
    # The call to convert_fields automatically converts the input to utf-8
    Exec        convert_fields("AUTO","utf-8"); \
            if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}.\d{3})    \[(\S+)\]    \[(\S+)\]    \[(\S+)\]    \[(\S+)\]    \[(.*)\]    \[(.*)\]    (.*)/s \
                    { \
                        $time = $1; \
                        $hostname = $2; \
                        $activityId = $3; \
                        $userIddeviceId = $4; \
                        $threadId = $5; \
                        $level = $6; \
                        $logger = $7; \
                        $message = $8; \
                        to_json(); \
                    } \
                    else \
                    { \
                        drop(); \
                }
</Input>

<Output eventlog-out>
    Module      om_tcp
    Host        127.0.0.1
    Port        3515
    Exec        $EventReceivedTime = integer($EventReceivedTime) / 1000000; \
                to_json();
</Output>
 
<Output iis-out>
    Module      om_tcp
    Host        127.0.0.1
    Port        3516
</Output>

<Output general-out>
    Module      om_tcp
    Host        127.0.0.1
    Port        3517
</Output>

<Route 1>

    Path eventlog => eventlog-out
</Route>
 
<Route 2>
    Path iis => iis-out
</Route>

<Route 3>
    Path webconsole => general-out
</Route>

Sample logs(Webconsole.log):

2015-07-10 10:24:17.424    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Info ]    [TestModule]    Testing log stash3    
2015-07-10 10:24:17.425    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Info ]    [TestModule]    Testing log stash4    
2015-07-10 10:24:17.448    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Error]    [TestModule]    *** EXCEPTION ***
System.DivideByZeroException: Attempted to divide by zero.
   at TCPPublisher.Program.Main(String[] args) in c:\Users\test\Documents\Visual Studio 2013\Projects\TCPDemo\TCPPublisher\Program.cs:line 26

Thanks in advance!
Mohan G