- Introduction
- Deployment
- Configuration
- OS Support
- Integration
- Troubleshooting
- Enterprise Edition Reference Manual
- 127. Man Pages
- 128. Configuration
- 129. Language
- 130. Extension Modules
- 130.1. Remote Management (xm_admin)
- 130.2. AIX Auditing (xm_aixaudit)
- 130.3. Apple System Logs (xm_asl)
- 130.4. Basic Security Module Auditing (xm_bsm)
- 130.5. Common Event Format (xm_cef)
- 130.6. Character Set Conversion (xm_charconv)
- 130.7. Delimiter-Separated Values (xm_csv)
- 130.8. Encryption (xm_crypto)
- 130.9. External Programs (xm_exec)
- 130.10. File Lists (xm_filelist)
- 130.11. File Operations (xm_fileop)
- 130.12. GELF (xm_gelf)
- 130.13. Go (xm_go)
- 130.14. Grok (xm_grok)
- 130.15. Java (xm_java)
- 130.16. JSON (xm_json)
- 130.17. Key-Value Pairs (xm_kvp)
- 130.18. LEEF (xm_leef)
- 130.19. Microsoft DNS Server (xm_msdns)
- 130.20. Multiline Parser (xm_multiline)
- 130.21. NetFlow (xm_netflow)
- 130.22. Microsoft Network Policy Server (xm_nps)
- 130.23. Pattern Matcher (xm_pattern)
- 130.24. Perl (xm_perl)
- 130.25. Python (xm_python)
- 130.26. Resolver (xm_resolver)
- 130.27. Rewrite (xm_rewrite)
- 130.28. Ruby (xm_ruby)
- 130.29. SNMP Traps (xm_snmp)
- 130.30. Remote Management (xm_soapadmin)
- 130.31. Syslog (xm_syslog)
- 130.32. W3C (xm_w3c)
- 130.33. WTMP (xm_wtmp)
- 130.34. XML (xm_xml)
- 130.35. Compression (xm_zlib)
- 131. Input Modules
- 132. Processor Modules
- 133. Output Modules
- NXLog Manager
- NXLog Add-Ons
130.14. Grok (xm_grok)
This module supports parsing events with Grok patterns. A field is added to the event record for each pattern semantic. For more information about Grok, see the Logstash Grok filter plugin documentation.
Note
|
To examine the supported platforms, see the list of installer packages in the Available Modules chapter. |
130.14.1. Configuration
The xm_grok module accepts the following directives in addition to the common module directives.
- Pattern
-
This mandatory directive specifies a directory or file containing Grok patterns. Wildcards may be used to specify multiple directories or files. This directive may be used more than once.
130.14.2. Functions
The following functions are exported by xm_grok.
- boolean
match_grok(string pattern)
-
Execute the match_grok() procedure with the specified pattern on the
$raw_event
field. If the event is successfully matched, return TRUE, otherwise FALSE. - boolean
match_grok(string field, string pattern)
-
Execute the match_grok() procedure with the specified pattern on the specified field. If the event is successfully matched, return TRUE, otherwise FALSE.
130.14.4. Examples
This configuration reads Syslog events from file and parses them with the
parse_syslog() procedure (this sets the
$Message field). Then the
match_grok() function is used to attempt a series
of matches on the $Message
field until one is successful. If no patterns
match, an internal message is logged.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension grok>
Module xm_grok
Pattern modules/extension/grok/patterns2.txt
</Extension>
<Input in>
Module im_file
File 'test2.log'
<Exec>
parse_syslog();
if match_grok($Message, "%{SSH_AUTHFAIL_WRONGUSER}") {}
else if match_grok($Message, "%{SSH_AUTHFAIL_WRONGCREDS}") {}
else if match_grok($Message, "%{SSH_AUTH_SUCCESS}") {}
else if match_grok($Message, "%{SSH_DISCONNECT}") {}
else
{
log_info('Event did not match any pattern');
}
</Exec>
</Input>
USERNAME [a-zA-Z0-9_-]+
INT (?:[+-]?(?:[0-9]+))
BASE10NUM (?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))
NUMBER (?:%{BASE10NUM})
WORD \b\w+\b
GREEDYDATA .*
IP (?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9])
SSH_AUTHFAIL_WRONGUSER Failed %{WORD:ssh_authmethod} for invalid user %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)
SSH_AUTHFAIL_WRONGCREDS Failed %{WORD:ssh_authmethod} for %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)
SSH_AUTH_SUCCESS Accepted %{WORD:ssh_authmethod} for %{USERNAME:ssh_user} from %{IP:ssh_client_ip} port %{NUMBER:ssh_client_port} (?<ssh_protocol>\w+\d+)(?:: %{WORD:ssh_pubkey_type} %{GREEDYDATA:ssh_pubkey_fingerprint})?
SSH_DISCONNECT Received disconnect from %{IP:ssh_client_ip} port %{INT:ssh_client_port}.*?:\s+%{GREEDYDATA:ssh_disconnect_reason}