Table of Contents
- Introduction
- Deployment
- Configuration
- 23. Configuration overview
- 24. NXLog language
- 25. Reading and receiving logs
- 26. Processing logs
- 26.1. Parsing various log formats
- 26.2. Alerting
- 26.3. Using buffers
- 26.4. Character set conversion
- 26.5. Detecting an inactive agent or log source
- 26.6. Event correlation
- 26.7. Extracting data
- 26.8. Filtering logs
- 26.9. Format conversion
- 26.10. Log rotation and retention
- 26.11. Log classification
- 26.12. Parsing multi-line logs
- 26.13. Rate limiting and traffic shaping of logs
- 26.14. Rewriting and modifying logs
- 26.15. Timestamps
- 27. Forwarding and Storing Logs
- 28. Centralized Log Collection
- 29. NXLog failover mode
- 30. High Availability (HA)
- 31. Encrypted transfer
- 32. Reducing bandwidth and data size
- 33. Reliable message delivery
- 34. Compression and Encryption
- OS Support
- Integration
- Troubleshooting
- Enterprise Edition Reference Manual
- NXLog Manager
- NXLog Add-Ons
26.6. Event correlation
It is possible to write correlation rules in the NXLog language using the built-in features such as variables and statistical counters. While these features are quite powerful, some cases cannot be detected with them, especially when conditions require a sliding window correlation.
A dedicated NXLog module, pm_evcorr, is available for advanced correlation requirements. It provides features similar to those of SEC and greatly enhances the correlation capabilities of NXLog.
Example 97. Correlation rules
The following configuration provides samples for each type of rule: Absence, Pair, Simple, Suppressed, and Thresholded.
nxlog.conf ![[Download file]](images/icons/download_icon.png "Download file")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
<Processor evcorr>
Module pm_evcorr
TimeField EventTime
<Simple>
Exec if $Message =~ /^simple/ $raw_event = "got simple";
</Simple>
<Suppressed>
# Match input event and execute an action list, but ignore the following
# matching events for the next t seconds.
Condition $Message =~ /^suppressed/
Interval 30
Exec $raw_event = "suppressing..";
</Suppressed>
<Pair>
# If TriggerCondition is true, wait Interval seconds for RequiredCondition
# to be true and then do the Exec. If Interval is 0, there is no window on
# matching.
TriggerCondition $Message =~ /^pair-first/
RequiredCondition $Message =~ /^pair-second/
Interval 30
Exec $raw_event = "got pair";
</Pair>
<Absence>
# If TriggerCondition is true, wait Interval seconds for RequiredCondition
# to be true. If RequiredCondition does not become true within the specified
# interval then do the Exec.
TriggerCondition $Message =~ /^absence-trigger/
RequiredCondition $Message =~ /^absence-required/
Interval 10
Exec log_info("'absence-required' not received within 10s");
</Absence>
<Thresholded>
# If the number of events exceeds the given threshold within the interval do
# the Exec. Same as SingleWithThreshold in SEC.
Condition $Message =~ /^thresholded/
Threshold 3
Interval 60
Exec $raw_event = "got thresholded";
</Thresholded>
<Stop>
Condition $EventTime < 2010-01-02 00:00:00
Exec log_debug("got stop");
</Stop>
<Simple>
# This will be rewritten only if the previous Stop condition is FALSE.
Exec $raw_event = "rewritten";
</Simple>
</Processor>