NXLog User Guide
- OS Support
- Enterprise Edition Reference Manual
- 146. Man Pages
- 147. Configuration
- 148. Language
- 149. Extension Modules
- 149.1. Remote Management (xm_admin)
- 149.2. AIX Auditing (xm_aixaudit)
- 149.3. Apple System Logs (xm_asl)
- 149.4. Basic Security Module Auditing (xm_bsm)
- 149.5. Common Event Format (xm_cef)
- 149.6. Character set conversion (xm_charconv)
- 149.7. Encryption (xm_crypto)
- 149.8. Delimiter-Separated Values (xm_csv)
- 149.9. External programs (xm_exec)
- 149.10. File lists (xm_filelist)
- 149.11. File operations (xm_fileop)
- 149.12. GELF (xm_gelf)
- 149.13. Go (xm_go)
- 149.14. Grok (xm_grok)
- 149.15. Java (xm_java)
- 149.16. JSON (xm_json)
- 149.17. Key-value pairs (xm_kvp)
- 149.18. LEEF (xm_leef)
- 149.19. Microsoft DNS Server (xm_msdns)
- 149.20. Multiline parser (xm_multiline)
- 149.21. NetFlow (xm_netflow)
- 149.22. Microsoft Network Policy Server (xm_nps)
- 149.23. Pattern matcher (xm_pattern)
- 149.24. Perl (xm_perl)
- 149.25. Python (xm_python)
- 149.26. Resolver (xm_resolver)
- 149.27. Rewrite (xm_rewrite)
- 149.28. Ruby (xm_ruby)
- 149.29. SNMP traps (xm_snmp)
- 149.30. Remote Management (xm_soapadmin)
- 149.31. Syslog (xm_syslog)
- 149.32. W3C (xm_w3c)
- 149.33. WTMP (xm_wtmp)
- 149.34. XML (xm_xml)
- 149.35. Compression (xm_zlib)
- 150. Input Modules
- 151. Processor Modules
- 152. Output Modules
- NXLog Manager
- NXLog Add-Ons
This module provides functions and procedures for working with data formatted as comma-separated values (CSV). CSV input can be parsed into fields whilst output can be generated as CSV. Delimiters other than the comma are also supported.
|To examine the supported platforms, see the list of installer packages in the Available Modules chapter.|
The pm_transformer module provides a simple interface to parse and generate CSV format, but the xm_csv module exports an API that can be used to solve more complex tasks involving CSV-formatted data.
|It is possible to use more than one xm_csv module instance with different options in order to support different CSV formats in the same configuration file. For this reason, functions and procedures exported by this module are public and must be referenced by the module instance name.|
This optional directive specifies the character to be used as the delimiter to separate fields. See the section on Specifying Delimiter Characters for more information. The default delimiter character is the comma (
,). Note that there should not be a delimiter after the last field.
This optional directive specifies the character to be used to escape special characters. See the section on Specifying Escape Characters for more information. The escape character is used to prefix the following characters: the escape character itself, the quote character, and the delimiter character. If EscapeControl is
TRUE, the newline (
\n), carriage return (
\r), tab (
\t), and backspace (
\b) control characters are also escaped. The default escape character is the backslash character (
If this optional boolean directive is set to
TRUE, control characters are also escaped. See the EscapeChar directive for details. The default is
TRUE: control characters are escaped. Note that this is necessary to allow single line CSV field lists which contain line-breaks.
This optional directive specifies a list of field types corresponding to the field names defined in the Fields directive. If specified it must match the same number of fields. If this directive is omitted, all fields will be stored as strings. This directive has no effect on the fields-to-CSV conversion.
This optional directive accepts the following values:
All fields will be quoted.
None of the fields will be quoted. This can be problematic if a field value, which is typically text that can contain any character, contains the delimiter character. When using this option, make sure that if the delimiter is found in the field value it is escaped or replaced with another character.
Note that this directive only effects CSV generation when using to_csv(). The CSV parser can automatically detect the quotation.
This directive has been deprecated in favor of QuoteMethod, which should be used instead.
If this optional boolean directive is set to
TRUE, the CSV parser will fail to parse CSV lines that do not contain the required number of fields. When this is set to
FALSEand the input contains fewer fields than specified in the Fields directive, the rest of the fields will be unset. The default value is
This optional directive specifies a string which will be treated as an undefined value. This is particularly useful when parsing the W3C format where the dash (
-) marks an omitted field.
- Unquoted single character
Any printable character can be specified as an unquoted character, except for the backslash (
- Control characters
The following non-printable characters can be specified with escape sequences:
audible alert (bell)
For example, to use TAB delimiting:
- A character in single quotes
The configuration parser strips whitespace, so it is not possible to define a space as the delimiter unless it is enclosed within quotes:
Delimiter ' '
Printable characters can also be enclosed:
The backslash can be specified when enclosed within quotes:
- A character in double quotes
Double quotes can be used like single quotes:
Delimiter " "
The backslash can be specified when enclosed within double quotes:
- A hexadecimal ASCII code
Hexadecimal ASCII character codes can be used prefixed with
0x. For example, the space can be specified as:
This is equivalent to:
Delimiter " "
The following functions are exported by xm_csv.
Convert the specified fields to a single CSV formatted string.
The following procedures are exported by xm_csv.
$raw_eventfield as CSV input.
Parse the given string as CSV format.
Format the specified fields as CSV and put this into the
This example shows that the xm_csv module can not only parse and create CSV formatted input and output, but with multiple xm_csv module instances it is also possible to reorder, add, remove, or modify fields to output data in a different CSV format.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 <Extension csv1> Module xm_csv Fields $id, $name, $number FieldTypes integer, string, integer Delimiter , </Extension> <Extension csv2> Module xm_csv Fields $id, $number, $name, $date Delimiter ; </Extension> <Input in> Module im_file File "tmp/input" <Exec> csv1->parse_csv(); $date = now(); if not defined $number $number = 0; csv2->to_csv(); </Exec> </Input> <Output out> Module om_file File "tmp/output" </Output>
1,"John K.",42 2,"Joe F.",43
1;42;"John K.";2011-01-15 23:45:20 2;43;"Joe F.";2011-01-15 23:45:20