Hi, I use module om_http for send events to host via https, but after start NXlog shows error: ERROR SSL certificate verification failed: self signed certificate in certificate chain (err: 19) Thanks for your ideas!

Hello, is the any way to set custom timeout in om_http? or custom retry mechanism? Thanks

Hi everyone, The user guide section for the om_http module explains: Each event is transferred in a single POST request. https://nxlog.co/documentation/nxlog-user-guide#om_http When https is used, is the TLS handshake also performed for each event? This seems like it would be quite inefficient. Are there any plans to support bulk / batch exporting of many events in a single http post, specifically for exporting data to the Splunk HEC? rsyslog and syslog-ng support batch exporting events to Splunk using the external python script omsplunkhec.py, as explained here: https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splunk.html NXLog supports batch exporting events to Elasticsearch using their bulk API: https://nxlog.co/documentation/nxlog-user-guide#om_elasticsearch

With reference to the following guide for sending NXLog output to Splunk using HEC: https://nxlog.co/documentation/nxlog-user-guide#splunk-hec-collection The NXLog community edition om_http is missing the add_http_header() procedure: https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#om_http This means it is not possible to output nxlog-ce data to http where a header needs to be added to the http post, such as the Splunk HEC: https://docs.splunk.com/Documentation/Splunk/7.0.3/Data/HECWalkthrough Is this functionality deliberately missing from the community edition, or can it please be added?

I have the usual error "Unexpected data from server ..." using om_http to send Windows event logs through Nginx to Redis. The problem is that when randomly this error happens ( once every 5 days) the service unexpectently stops ! From the source code I can understand that NXLog receives a response which cannot map to a correnponding request. It has a response with an appropriate length but the corresponding request is null! Possibly is a bad logic implementation, or even a network problem that could cause request/response mismatch.I really don't know. But I also really can't understand why NxLog preferres to die instead of  just return from the function in case this happens and not execute the code following the condition that drives the service to crash at the end. Stelios

Hi, I'm using nxlog community edition nxlog-ce-2.9.1347, I have few questions related om_http module. We have a centralized log server(Log Insight) and 10 application servers. Log Insight server exposes a REST API to post the logs data.I'm using NXLOG as a log forwarder to Log Insight Server from all my application servers. Please clarify the following questions. POST URL : http://loginsight:9000/api/v1/messages/ingest/4C4C4544-0037-5910-805A-C4C04F585831 Request Body: {"messages": [{  "fields": [  {"name": "Channel", "content": "Security"},  {"name": "EventID", "content": "4688"},  {"name": "EventRecordID", "content": "33311266"},  {"name": "Keywords", "content": "Audit Success"},  {"name": "Level", "content": "Information"},  {"name": "OpCode","content": "Info"},  {"name": "ProcessID", "content": "4"},  {"name": "ProviderName", "content": "Microsoft-Windows-Security-Auditing"},  {"name": "Task", "content": "Process Creation"},  {"name": "ThreadID", "content": "64"}  ],  "text": "A new process has been created.",  "timestamp": 1396622879241  }  ] } 1. How do I format my log data into the request as mentioned above in NXLOG. Request should be formated into json with fields and the data accordingly.    I'm able to parse IIS logs,eventlogs and logs from files into JSON but got stuck with calling REST API with request body. Please find my nxlog.conf below. 2. Is it a good idea to directly send the log data to log server via REST API? if not what are the disadvantages.  3. Does om_http module support retrying logic\buffering just in case REST API is down or doesn't respond. 4. What is the best architecture for sending the logs to centralized server, I see lot of people online follow NXLOG=>LOG STASH=>ELASTICSEARCH or Some Centralized server(Log Insight in my case). 5. Should I use pm_buffer, as my log files will be rotated after certain memory limit is reached in case if the REST API is down or om_http module handles this automatically.      Following configuration reads IIS, event logs and logs from files. ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally and is also available ## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html ## Please set the ROOT to the folder your nxlog was installed into, ## otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension w3c> # Map the fields from the IIS log file (you can open the IIS log file to see the header and know what fields to map) Module xm_csv Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $cs(User-Agent), $sc-status, $sc-substatus, $sc-win32-status, $time-taken FieldTypes string, string, string, string, string, string, integer, string, string, string, integer, integer, integer, integer Delimiter ' ' </Extension> <Extension multiline> Module xm_multiline HeaderLine /^\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}.\d{3}/ </Extension> <Extension charconv> Module xm_charconv AutodetectCharsets utf-8, utf-16, utf-32, iso8859-2 </Extension> <Input eventlog> Module im_msvistalog ReadFromLast True Module im_msvistalog Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> </Input> <Input iis> Module im_file File 'C:\inetpub\logs\LogFiles\W3SVC1\u_ex*.log' ReadFromLast TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c->parse_csv(); \ $EventTime = parsedate($date + " " + $time); \ to_json (); \ } </Input> <Input webconsole> Module im_file File 'C:\Stash\WebConsole.log' InputType multiline SavePos TRUE # The call to convert_fields automatically converts the input to utf-8 Exec convert_fields("AUTO","utf-8"); \ if $raw_event =~ /^(\d{4}\-\d{2}\-\d{2} \d{2}\:\d{2}\:\d{2}.\d{3}) \[(\S+)\] \[(\S+)\] \[(\S+)\] \[(\S+)\] \[(.*)\] \[(.*)\] (.*)/s \ { \ $time = $1; \ $hostname = $2; \ $activityId = $3; \ $userIddeviceId = $4; \ $threadId = $5; \ $level = $6; \ $logger = $7; \ $message = $8; \ to_json(); \ } \ else \ { \ drop(); \ } </Input> <Output eventlog-out> Module om_tcp Host 127.0.0.1 Port 3515 Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; \ to_json(); </Output> <Output iis-out> Module om_tcp Host 127.0.0.1 Port 3516 </Output> <Output general-out> Module om_tcp Host 127.0.0.1 Port 3517 </Output> <Route 1> Path eventlog => eventlog-out </Route> <Route 2> Path iis => iis-out </Route> <Route 3> Path webconsole => general-out </Route> Sample logs(Webconsole.log): 2015-07-10 10:24:17.424    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Info ]    [TestModule]    Testing log stash3     2015-07-10 10:24:17.425    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Info ]    [TestModule]    Testing log stash4     2015-07-10 10:24:17.448    [20EX15736]    [00000000-0000-0000-0000-000000000000]    [0000000-0000000]    [00008]    [Error]    [TestModule]    *** EXCEPTION *** System.DivideByZeroException: Attempted to divide by zero.    at TCPPublisher.Program.Main(String[] args) in c:\Users\test\Documents\Visual Studio 2013\Projects\TCPDemo\TCPPublisher\Program.cs:line 26   Thanks in advance! Mohan G