- Introduction
- Deployment
- Configuration
- OS Support
- Integration
- Troubleshooting
- Enterprise Edition Reference Manual
- 127. Man Pages
- 128. Configuration
- 129. Language
- 130. Extension Modules
- 131. Input Modules
- 132. Processor Modules
- 133. Output Modules
- 133.1. Batched Compression (om_batchcompress)
- 133.2. Blocker (om_blocker)
- 133.3. DBI (om_dbi)
- 133.4. Elasticsearch (om_elasticsearch)
- 133.5. EventDB (om_eventdb)
- 133.6. Program (om_exec)
- 133.7. Files (om_file)
- 133.8. Go (om_go)
- 133.9. HTTP(s) (om_http)
- 133.10. Java (om_java)
- 133.11. Kafka (om_kafka)
- 133.12. Null (om_null)
- 133.13. ODBC (om_odbc)
- 133.14. Perl (om_perl)
- 133.15. Named Pipes (om_pipe)
- 133.16. Python (om_python)
- 133.17. Raijin (om_raijin)
- 133.18. Redis (om_redis)
- 133.19. Ruby (om_ruby)
- 133.20. TLS/SSL (om_ssl)
- 133.21. TCP (om_tcp)
- 133.22. UDP (om_udp)
- 133.23. UDP with IP Spoofing (om_udpspoof)
- 133.24. Unix Domain Sockets (om_uds)
- 133.25. WebHDFS (om_webhdfs)
- 133.26. ZeroMQ (om_zmq)
- NXLog Manager
- NXLog Add-Ons
133.17. Raijin (om_raijin)
This module enables logs to be forwarded to Raijin Database servers for ingestion. It connects to the URL specified in the configuration in either HTTP or HTTPS mode. Raijin accepts HTTP POST requests with multiple JSON records in the request body, assuming that there is a target database table already created on the Raijin side. Although Raijin accepts structured data in flat JSON (i.e. a list of key-value pairs) format, it does not support JSON values comprised of nested data structures or other non-scalar data such as arrays and maps. Raijin currently does not support authorization/SSL but the om_raijin module supports TLS since TLS can be enabled with an HTTP proxy.
133.17.1. Batch processing
Starting with NXLog version 5.2, this module takes advantage of an internal batch processing mechanism which is provided by the NXLog core.
Internal batching works by accumulating multiple successive event records produced by an input or processor module into a batch of records. When a batch is full or after a configurable timeout expires, the batch is forwarded to the next module in the route. For it’s own batching functionality, the om_raijin module now uses the already available batches it receives from the previous module, instead of reorganizing individual event records into batches itself.
Simply put, the new batching functionality works the same as before, with the difference that event records are organized into batches by input and processor modules, whereas in older versions, it was the output modules which did this. Because of this, the FlushInterval and FlushLimit configuration directives have been deprecated for om_raijin. The same functionality is now provided by the BatchFlushInterval and BatchSize directives, which can be used with all input and processor module(s).
Note
|
To examine the supported platforms, see the list of installer packages in the Available Modules chapter. |
Note
|
This module requires the xm_json extension module to be loaded in order to
convert the payload to JSON. If the |
133.17.2. Configuration
The om_raijin module accepts the following directives in addition to the common module directives.
These three om_raijin directives are required:
- DBName
-
This mandatory directive specifies the name of the Raijin database containing the table where logs will be stored.
- DBTable
-
This mandatory directive specifies the name of the database table where log events will be inserted as JSON records.
- URL
-
This mandatory directive is comprised of the REST API URI that Raijin Server uses for receiving bulk, JSON-formatted events via POST requests:
http://<RaijinServerHostname>:<port>/_bulk
If an om_raijin module instance contains multiple URL directives, the hosts will function as a cluster in NXLog failover mode. This directive also supports HTTPS URLs. If the optional port number part of the URL (
:<port>
) is not explicitly included in the URL, it will default to port 80 for HTTP (or port 443 for HTTPS). The/_bulk
endpoint part of the URL is required and will return a400 Bad Request
error if missing.
Note
|
The Raijin Database REST API listens by default on port 2500. If you plan to use this directive without explicitly specifying a port number, make sure the Raijin Database server is configured to listen on the correct default port corresponding to this directive (either 80 or 443). |
The following om_raijin directives are optional.
- FlushInterval
-
This directive has been deprecated. See Batch processing for details.
- FlushLimit
-
This directive has been deprecated. See Batch processing for details.
- HTTPSAllowUntrusted
-
Accepts a boolean value to determine whether the remote connection should be allowed without certificate verification. If set to
TRUE
, the connection will be allowed even if the remote HTTPS server presents an unknown or self-signed certificate. The default value isFALSE
; the remote HTTPS server must present a trusted certificate.
- HTTPSCADir
-
Specifies the path to a directory containing certificate authority (CA) certificates which will be used to check the certificate of the remote HTTPS server. The certificate filenames in this directory must be in the OpenSSL hashed format. A remote’s self-signed certificate (which is not signed by a CA) can also be trusted by including a copy of the certificate in this directory.
- HTTPSCAFile
-
Specifies the path to the certificate authority (CA) certificate which will be used to validate the certificate of the remote HTTPS server. To trust a self-signed certificate presented by the remote host (which is not signed by a CA), provide that certificate instead.
- HTTPSCertFile
-
Specifies the path of the certificate file to be used for the HTTPS handshake.
- HTTPSCertKeyFile
-
Specifies the path of the certificate key file to be used for the HTTPS handshake.
- HTTPSCRLDir
-
Specifies the path to a directory containing certificate revocation lists (CRLs), which will be consulted when checking the certificate of the remote HTTPS server. The certificate filenames in this directory must be in the OpenSSL hashed format.
- HTTPSCRLFile
-
Specifies the path of the certificate revocation list (CRL) which will be consulted when checking the certificate of the remote HTTPS server.
- HTTPSKeyPass
-
Specifies a password for the certificate key file defined in HTTPSCertKeyFile. This directive is not needed for passwordless private keys.
- HTTPSSSLCipher
-
Sets the permitted SSL cipher list, overriding the default. Use the format described in the ciphers man page.
- HTTPSSSLCiphersuites
-
This optional directive can be used to define the permitted SSL cipher list in case the HTTPSSSLProtocol directive is set to
TLSv1.3
. Use the same format as in the HTTPSSSLCipher directive.
- HTTPSSSLCompression
-
Accepts a boolean value to determine whether data compression should be enabled. The compression algorithm is based on the zlib compression library. If not specified, it defaults to
FALSE
(compression will be disabled).
Note
|
Some Linux packages (for example, Debian) use the OpenSSL library provided by the OS and may not support the zlib compression mechanism. The module will emit a warning on startup if compression support is missing. The generic deb/rpm packages are bundled with a zlib-enabled libssl library. |
- HTTPSSSLProtocol
-
This directive can be used to set the allowed SSL/TLS protocol(s). It takes a comma-separated list of values which can be any of the following:
SSLv2
,SSLv3
,TLSv1
,TLSv1.1
,TLSv1.2
andTLSv1.3
. By default, theTLSv1.2
andTLSv1.3
protocols are allowed. Note that the OpenSSL library shipped by Linux distributions may not supportSSLv2
andSSLv3
, and these will not work even if enabled with this directive.
- ProxyAddress
-
Specifies the IP address of the proxy server in case the module needs to connect to a Raijin Database server through a proxy.
Note
|
The om_raijin module supports only HTTP proxies, i.e. SOCKS4/SOCKS5 proxies are not supported. |
- ProxyPort
-
Specifies the port number required to connect to the proxy server.
- SNI
-
Specifies the hostname used for Server Name Indication (SNI) in HTTPS mode.
133.17.3. Examples
This configuration reads log messages from a file and forwards them to a
Raijin Database server deployed on localhost
.
This configuration sends logs to a cluster of Raijin Database servers
configured for NXLog failover mode. The cluster is comprised
of three hosts: http://raijin-1.example.com:2500/_bulk
,
http://localhost:2500/_bulk
,
and http://192.168.1.123:2500/_bulk
.