- OS Support
- Enterprise Edition Reference Manual
- 127. Man Pages
- 128. Configuration
- 129. Language
- 130. Extension Modules
- 131. Input Modules
- 132. Processor Modules
- 132.1. Blocker (pm_blocker)
- 132.2. Buffer (pm_buffer)
- 132.3. Event Correlator (pm_evcorr)
- 132.4. Filter (pm_filter)
- 132.5. HMAC Message Integrity (pm_hmac)
- 132.6. HMAC Message Integrity Checker (pm_hmac_check)
- 132.7. De-Duplicator (pm_norepeat)
- 132.8. Null (pm_null)
- 132.9. Pattern Matcher (pm_pattern)
- 132.10. Format Converter (pm_transformer)
- 132.11. Timestamping (pm_ts)
- 133. Output Modules
- NXLog Manager
- NXLog Add-Ons
Messages received over UDP may be dropped by the operating system if packets are not read from the message buffer fast enough. Some logging subsystems using a small circular buffer can overwrite old logs in the buffer if it is not read, also resulting in loss of log data. Buffering can help in such situations.
|To examine the supported platforms, see the list of installer packages in the Available Modules chapter.|
The pm_buffer module supports disk- and memory-based log message buffering. If both are required, multiple pm_buffer instances can be used with different settings. Because a memory buffer can be faster, though its size is limited, combining memory and disk based buffering can be a good idea if buffering is frequently used.
The disk-based buffering mode stores the log message data in chunks. When all the data is successfully forwarded from a chunk, it is then deleted in order to save disk space.
|Using pm_buffer is only recommended when there is a chance of message loss. The built-in flow control in NXLog ensures that messages will not be read by the input module until the output side can send, store, or forward. When reading from files (with im_file) or the Windows EventLog (with im_mseventlog or im_msvistalog) it is rarely necessary to use the pm_buffer module unless log rotation is used. During a rotation, there is a possibility of dropping some data while the output module (im_tcp, for example) is being blocked.|
If set to TRUE, this optional boolean directive instructs the module to create the output directory before opening the file for writing if it does not exist. The default is FALSE.
This mandatory directive specifies the size of the buffer in kilobytes.
This directive can be set to either
Diskto select memory- or disk-based buffering.
This directory will be used to store the disk buffer file chunks. This is only valid if Type is set to
This directive specifies an optional limit, smaller than MaxSize, which will trigger a warning message when reached. The log message will not be generated again until the buffer size drops to half of WarnLimit and reaches it again in order to protect against a warning message flood.
The following functions are exported by pm_buffer.
Return the number of log messages held in the memory buffer.
Return the size of the memory buffer in bytes.
This configuration accepts log messages via UDP and forwards them via TCP. An intermediate memory-based buffer allows the im_udp module instance to continue accepting messages even if the om_tcp output stops working (caused by downtime of the remote host or network issues, for example).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 <Input udp> Module im_udp ListenAddr 0.0.0.0:514 </Input> <Processor buffer> Module pm_buffer # 1 MB buffer MaxSize 1024 Type Mem # warn at 512k WarnLimit 512 </Processor> <Output tcp> Module om_tcp Host 192.168.1.1:1514 </Output> <Route udp_to_tcp> Path udp => buffer => tcp </Route>