- Introduction
- Deployment
- Configuration
- OS Support
- Integration
- Troubleshooting
- 124. Internal logs
- 124.1. Default settings
- 124.2. Enable internal logging
- 124.3. Raise the severity level of logged events
- 124.4. Send customized log messages to the internal log
- 124.5. Send all fields to the internal log
- 124.6. Send debug dump to the internal log
- 124.7. Send internal log to STDOUT
- 124.8. Send internal log to an existing route
- 124.9. Send information to an external file
- 125. Common issues
- 126. Debugging NXLog
- 124. Internal logs
- Enterprise Edition Reference Manual
- NXLog Manager
- NXLog Add-Ons
124. Internal logs
When issues arise while configuring or maintaining an NXLog instance, a stepwise troubleshooting approach (moving from the most likely and simple cases to the more complex and rare ones) generally yields favorable results. The first step is always to inspect the internal log which NXLog generates.
124.1. Default settings
By default, NXLog generates log messages about its own operations. These messages are essential for troubleshooting problems, and should be checked first if NXLog is not functioning as expected.
These internal messages are written to the file defined in the
LogFile directive in nxlog.conf
. On Windows that
file is C:\Program Files\nxlog\data\nxlog.log
; on Linux, /opt/nxlog/var/log/nxlog/nxlog.log
. If this directive is not specified, internal logging is
disabled.
124.2. Enable internal logging
Internal logging is enabled by default after installation, but may be disabled
if edits were made to nxlog.conf
, or if the LogFile
directive points to a path which is unavailable. Enable internal logging by
editing nxlog.conf to ensure the LogFile
directive is set to an available
path.
Note
|
Some Windows applications (WordPad, for example) cannot open the log file while the NXLog process is running because of exclusive file locking. Use a viewer that does not lock the file, like Notepad. |
124.3. Raise the severity level of logged events
By default, internal logs are generated with a log level of INFO
. To get
detailed information about NXLog’s operations, the log level can be
raised to DEBUG
level. This level of detail reliably produces a large amount
of log messages, and so we recommend setting this log level only for sustained
troubleshooting sessions.
To raise the log level temporarily (until NXLog is restarted).
On Linux, send SIGUSR2:
# kill -SIGUSR2 $PID
On Windows, send service control command 201:
> sc control nxlog 201
To raise the log level for an extended troubleshooting session.
-
On all systems, set the LogLevel directive to
DEBUG
, then restart NXLog.
124.4. Send customized log messages to the internal log
It may be helpful to add extra logging statements to your configuration using the log_info() procedure.
The generated messages will be visible:
-
in the file defined in the LogFile global directive
-
in the input from the im_internal module
-
on standard output when running NXLog in the foreground with the
-f
command line switch
This configuration uses the log_info() procedure to send
values to the internal log. Log messages are accepted over UDP on port 514. If
keyword
is found in the unparsed message, an INFO
level message will
be generated.
124.5. Send all fields to the internal log
In this configuration, the to_json() procedure from the xm_json module is used to send all the fields to the internal log.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<Extension _syslog>
Module xm_syslog
</Extension>
<Extension _json>
Module xm_json
</Extension>
<Input in>
Module im_tcp
Host 0.0.0.0
Port 1514
<Exec>
parse_syslog_bsd();
# Dump $raw_event
log_info("raw_event is: " + $raw_event);
# Dump fields in JSON
log_info("Other fields are: " + to_json());
</Exec>
</Input>
{
"MessageSourceAddress": "127.0.0.1",
"EventReceivedTime": "2012-05-18 13:11:35",
"SourceModuleName": "in",
"SourceModuleType": "im_tcp",
"SyslogFacilityValue": 3,
"SyslogFacility": "DAEMON",
"SyslogSeverityValue": 3,
"SyslogSeverity": "ERR",
"SeverityValue": 4,
"Severity": "ERROR",
"Hostname": "host",
"EventTime": "2010-10-12 12:49:06",
"SourceName": "app",
"ProcessID": "12345",
"Message": "test message"
}
124.6. Send debug dump to the internal log
A simple way to quickly get a more complete picture of NXLog’s current status is to dump debug info into the internal log. This information can be helpful in determining, for example, why an input module is not sending to an output module. Normally, internal events are written to the log file configured with the LogFile directive.
On Linux, send SIGUSR1 to the application:
# kill -SIGUSR1 $PID
On Windows, send the service control command "200" to the application:
> sc control nxlog 200
2017-03-29 10:05:19 INFO event queue has 2 events;jobgroup with priority 10;job
of module in/im_file, events: 0;job of module out/om_null, events: 0;non-module
job, events: 0;jobgroup with priority 99;non-module job, events: 0;[route 1]; -
in: type INPUT, status: RUNNING queuesize: 0; - out: type OUTPUT, status:
RUNNING queuesize: 0;
Note
|
The status is the most important piece of information in the dumped log
entries. A status of PAUSED means the input module is not able to send
because the output module queue is full. In such a case the queuesize for the
corresponding output(s) would be over 99. A status of STOPPED means the
module is fully stopped, usually due to an error.
|
124.7. Send internal log to STDOUT
Run NXLog in the foreground to print internal logs to the standard output and standard error streams, which are both visible in the terminal.
Use nxlog -f
to run NXLog in the foreground.
124.8. Send internal log to an existing route
NXLog can also write internal log data into a normal route using the im_internal module. Internal log messages can then be forwarded like any other log source.
Tip
|
Local logging is more fault-tolerant than routed logging, and is therefore recommended for troubleshooting. |
Note
|
It is not possible to use a log level higher than INFO with the
im_internal module. DEBUG level messages can only be written
to the local log file.
|
124.9. Send information to an external file
Sending the internal log or other information to external files can also be useful while troubleshooting.
In this example configuration, the file_write() procedure (from the xm_fileop module) is used to dump information to an external file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<Extension _fileop>
Module xm_fileop
</Extension>
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_tcp
Host 0.0.0.0
Port 1514
<Exec>
parse_syslog_bsd();
# Debug $SyslogSeverity and $Hostname fields
file_write("/tmp/debug.txt",
"Severity: " + $SyslogSeverity +
", Hostname: " + $Hostname + "\n");
</Exec>
</Input>