NXLog Docs

F5 BIG-IP

F5 BIG-IP appliances can send logs to a remote syslog destination via TCP or UDP. TCP is a more reliable protocol and is recommended for sending logs over the network. On the other hand, UDP does not guarantee data delivery; therefore, logs may get lost or truncated, especially in high-volume logging environments.

Several BIG-IP sub-systems write traffic and audit logs to different files. Below is an example of Local Traffic Management (LTM) logs reporting pool members being up or down.

Local Traffic Management (LTM) log sample

Aug 08 16:50:12 l-lb1 notice mcpd[7660]: 01070639:5: Pool /Common/q-qa-pool member /Common/q-qa1:25 session status forced disabled.
Aug 08 16:51:33 l-lb1 notice mcpd[7660]: 01070639:5: Pool /Common/q-qa-pool member /Common/q-qa1:25 session status enabled.

Furthermore, the following is a sample from an audit log file.

Audit log sample

Aug 08 16:43:41 l-lb1 notice httpd[3064]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start="Mon Aug 08 16:43:41 2022".
Aug 08 17:10:33 l-lb1 notice httpd[1181]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start="Mon Aug 08 16:43:41 2022" end="Mon Aug 08 17:10:33 2022".

Refer to K16197: Reviewing BIG-IP log files for more information on F5 logs. In addition, you can find details on configuring BIG-IP logging for your version in the F5 Knowledge Center.

Collecting BIG-IP logs via TCP

The BIG-IP web interface does not provide a way to configure an external TCP syslog destination, so you must configure it via the command line.

Configure NXLog to receive syslog messages via TCP; see Receiving BIG-IP logs via TCP below. Restart NXLog.
Ensure all your BIG-IP devices can communicate with the NXLog agent. Depending on your setup, you may need to configure a new route or default gateway.
Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.

Remove any existing BIG-IP syslog configuration.

# tmsh list sys syslog include
# tmsh modify sys syslog include none

Configure a new remote syslog destination. Replace SYSLOG_IP and PORT with the IP address and port of the NXLog agent and LEVEL with the logging level you require.
```
# tmsh modify sys syslog include "destination remote_server {tcp(\"SYSLOG_IP\" port (PORT));};filter f_alllogs { level (LEVEL...emerg);}; log { source(local); filter(f_alllogs); destination(remote_server);};"
```
For example, this command redirects informational and higher-level logs (from info to emerg) to an NXLog agent at 192.168.100.85 listening on TCP port 1514.
```
# tmsh modify sys syslog include "destination remote_server { tcp(\"192.168.100.85\" port (1514));}; filter f_alllogs {level (info...emerg);}; log {source(local); filter(f_alllogs); destination(remote_server);};"
```
Once you execute this command, BIG-IP will stop local logging and start forwarding all appliance logs to the remote destination.

In case of a High Availability (HA) group, synchronize the configuration changes to the other units.

# tmsh run cm config-sync to-group GROUP_NAME

Once the configuration synchronizes to all group members, each member will send logs with its hostname and IP address. In the event of a failover, logging will continue from both units regardless of which one is currently active.

Example 1. Receiving BIG-IP logs via TCP

This configuration uses the im_tcp input module to listen for logs on TCP port 1514.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input tcp_listen>
    Module        im_tcp
    ListenAddr    0.0.0.0:1514
    <Exec>
        parse_syslog(); (1)
        to_json(); (2)
    </Exec>
</Input>

1	Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module.
2	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
  "MessageSourceAddress": "192.168.100.95",
  "EventReceivedTime": "2022-08-08T10:52:21.423899+03:00",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "SyslogFacilityValue": 16,
  "SyslogFacility": "LOCAL0",
  "SyslogSeverityValue": 5,
  "SyslogSeverity": "NOTICE",
  "SeverityValue": 2,
  "Severity": "INFO",
  "Hostname": "localhost.localdomain",
  "EventTime": "2022-08-08T10:51:49.000000+03:00",
  "SourceName": "tmsh",
  "ProcessID": 20306,
  "Message": "01420002:5: AUDIT - pid=20306 user=root folder=/Common module=(tmos)# status=[Command OK] cmd_data=list sys syslog include"
}
{
  "MessageSourceAddress": "192.168.100.95",
  "EventReceivedTime": "2022-08-08T10:52:33.796531+03:00",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "SyslogFacilityValue": 3,
  "SyslogFacility": "DAEMON",
  "SyslogSeverityValue": 6,
  "SyslogSeverity": "INFO",
  "SeverityValue": 2,
  "Severity": "INFO",
  "Hostname": "localhost.localdomain",
  "EventTime": "2022-08-08T10:52:01.000000+03:00",
  "SourceName": "systemd",
  "ProcessID": 1,
  "Message": "Started Session 4277 of user root."
}

NXLog can also be configured to parse log records, such as those containing key-value pairs, into structured data.

Example 2. Parsing BIG-IP logs

This configuration uses the im_tcp input module to listen for logs on TCP port 1514. It then uses regular expressions and the xm_kvp module to parse log records into structured data.

nxlog.conf

<Extension syslog>
    Module          xm_syslog
</Extension>

<Extension json>
    Module          xm_json
</Extension>

<Extension kvp>
    Module          xm_kvp
    KVPDelimiter    " "
    KVDelimiter     =
    EscapeChar      \\
</Extension>

<Input tcp_listen>
    Module          im_tcp
    ListenAddr      0.0.0.0:1514
    <Exec>
        parse_syslog(); (1)
        if $Message =~ /^([a-z]*) ([a-zA-Z]*)(.*)$/ (2)
        {
            $F5MsgLevel = $1;
            $F5Process = $2;
            $F5Message = $3;
            if $F5Message =~ /^\[[0-9]*\]: ([0-9]*):([0-9]+): (.*)$/ (3)
            {
                $F5MsgID = $1;
                $F5MsgSev = $2;
                $F5Message = $3;
                if $F5Message =~ /RAW: ([a-z]*)\(([a-z_]*)\): ([a-zA-Z]+=.+)/ (4)
                {
                    $F5Process = $1;
                    $F5Module = $2;
                    kvp->parse_kvp($3); (5)
                }
            }
            delete($F5Message); (6)
        }
        to_json(); (7)
    </Exec>
</Input>

1	Parses log records using the parse_syslog() procedure of the xm_syslog module, which creates the `$Message` field.
2	Uses a regular expression to extract the message level and process name from the `$Message` field.
3	Parses the message further to extract the message ID and severity.
4	Parses the message further to extract the process, module, and list of key-value pairs.
5	Parses the key-value pairs into fields using the parse_kvp() procedure of the xm_kvp module.
6	Deletes the redundant `$F5Message` field with the delete() procedure.
7	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
  "MessageSourceAddress": "192.168.6.91",
  "EventReceivedTime": "2022-08-08 00:08:43",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "SyslogFacilityValue": 10,
  "SyslogFacility": "AUTHPRIV",
  "SyslogSeverityValue": 5,
  "SyslogSeverity": "NOTICE",
  "SeverityValue": 2,
  "Severity": "INFO",
  "Hostname": "l-lb1",
  "Message": "notice httpd[5320]: 01070417:5: AUDIT - user john - RAW: httpd(mod_auth_pam): user=john(john) partition=[All] level=Administrator tty=/usr/bin/tmsh host=192.168.9.78 attempts=1 start=\"Mon Aug 08 00:07:59 2022\".",
  "EventTime": "2022-08-08 00:07:59",
  "F5MsgLevel": "notice",
  "F5MsgID": "01070417",
  "F5MsgSev": "5",
  "F5Process": "httpd",
  "F5Module": "mod_auth_pam",
  "user": "john(john)",
  "partition": "[All]",
  "level": "Administrator",
  "tty": "/usr/bin/tmsh",
  "host": "192.168.9.78",
  "attempts": "1",
  "start": "Mon Aug 08 00:07:59 2022"
}

Collecting BIG-IP logs via UDP

When reliable delivery is not a concern, or you need to retain a local copy of the logs on each appliance, you can configure BIG-IP to send logs to a remote syslog destination via UDP.

Configure NXLog to receive syslog messages via UDP; see Receiving BIG-IP logs via UDP below. Restart NXLog.
Ensure all your BIG-IP devices can communicate with the NXLog. Depending on your setup, you may need to configure a new route or default gateway.
Follow the instructions to configure UDP logging via the command line or web interface.

Example 3. Receiving BIG-IP logs via UDP

This configuration uses the im_udp input module to listen for logs on UDP port 514.

nxlog.conf

<Extension json>
    Module        xm_json
</Extension>

<Extension syslog>
    Module        xm_syslog
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:514
    <Exec>
        parse_syslog(); (1)
        to_json(); (2)
    </Exec>
</Input>

1	Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module.
2	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
    "MessageSourceAddress": "192.168.100.86",
    "EventReceivedTime": "2022-08-08T16:05:02.330953+03:00",
    "SourceModuleName": "udp_listen",
    "SourceModuleType": "im_udp",
    "SyslogFacilityValue": 3,
    "SyslogFacility": "DAEMON",
    "SyslogSeverityValue": 6,
    "SyslogSeverity": "INFO",
    "SeverityValue": 2,
    "Severity": "INFO",
    "Hostname": "bigip",
    "EventTime": "2022-08-08T16:05:01.000000+03:00",
    "Message": "info systemd[1]: Started Session 2059 of user root."
}

Configuring UDP logging via the command line

Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.
Configure a new remote syslog destination. Replace SYSLOG_IP and PORT with the IP address and port of the NXLog agent.
```
# tmsh modify sys syslog remote-servers add { nxlog { host SYSLOG_IP remote-port PORT } }
```
For example, this command redirects logs to an NXLog agent at 192.168.100.85 listening on UDP port 514.
```
# tmsh modify sys syslog remote-servers add { nxlog { host 192.168.100.85 remote-port 514 } }
```

In case of a High Availability (HA) group, synchronize the configuration changes to the other units.

# tmsh run cm config-sync to-group GROUP_NAME

Configuring UDP logging via the web interface

If you haven’t already, first enable the BIG-IP web interface:

Execute the following command to open the configuration utility and follow the instructions. The utility automatically detects the current machine’s IP address and makes all the appropriate changes if required.
```
# config eth0
```
The first time you log in to the BIG-IP instance, use the default username root and password default. You will be prompted to change the root password.
Execute the following command to change the password of the admin user.
```
# tmsh modify auth user admin prompt-for-password
```
Configure the BIG-IP HTTP server to listen on port 80. Replace BIG_IP_SERVER_IP with the IP address of the BIG-IP server.
```
# tmsh modify sys httpd include "Listen BIG_IP_SERVER_IP:80"
```

Once you enable access to the web interface, follow these steps to configure remote logging:

Open https://BIG_IP_SERVER_IP and log in with your account. In case of a High Availability (HA) group, ensure you are logged in to the active unit. You should see ONLINE (ACTIVE) in the top left corner.
Navigate to System > Logs > Configuration > Remote Logging.
Type the Remote IP and Remote Port of the NXLog agent and click Add, followed by Update.

In case of a High Availability (HA) group, synchronize the configuration changes to the other units:

Click the yellow Changes Pending in the top left corner.
Select the active device marked as (Self).

Enable the Sync Device to Group option and click Sync.

Using SNMP traps

NXLog Enterprise Edition exclusive feature

BIG-IP devices are capable of sending SNMP traps and contain predefined SNMP traps. They also support creating user-defined traps. You can find additional information about SNMP support for your version in the F5 Knowledge Center, under the Monitoring BIG-IP System Traffic with SNMP section in the External Monitoring of BIG-IP Systems: Implementations guide.

BIG-IP systems include Management Information Base (MIB) files stored on the device. Refer to K13322: Overview of BIG-IP MIB files for more information.

Configure NXLog to receive SNMP traps. See Receiving SNMP traps below.
Ensure all your BIG-IP devices can communicate with the NXLog agent. Depending on your setup, you may need to configure a new route or default gateway.
Follow the instructions to configure SNMP traps via the command line or web interface.

Example 4. Receiving SNMP traps

This configuration uses the im_udp input module to listen for logs on UDP port 162 and the xm_snmp module to process SNMP traps.

nxlog.conf

<Extension json>
    Module        xm_json
</Extension>

<Extension snmp>
    Module        xm_snmp
    MIBDir        /usr/share/mibs/bigip

    # The following section is required for SNMPv3
    #<User snmp_user>
    #    AuthProto      sha1
    #    AuthPasswd     q1w2e3r4
    #    EncryptPasswd  q1w2e3r4
    #    EncryptProto   aes
    #</User>
</Extension>

<Input udp_listen>
    Module        im_udp
    ListenAddr    0.0.0.0:162
    InputType     snmp (1)
    Exec          to_json(); (2)
</Input>

1	Sets the InputType directive to the xm_snmp module instance name.
2	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
  "SNMP.CommunityString": "nxlog",
  "SNMP.RequestID": 684814253,
  "EventTime": "2022-08-08T15:06:31.765042+03:00",
  "SeverityValue": 2,
  "Severity": "INFO",
  "iso.3.6.1.2.1.1.3.0": 5111365,
  "iso.3.6.1.6.3.1.1.4.1.0": "iso.3.6.1.4.1.3375.2.4.0.3",
  "iso.3.6.1.6.3.1.1.4.3.0": "iso.3.6.1.4.1.3375.2.4",
  "MessageSourceAddress": "192.168.100.86",
  "EventReceivedTime": "2022-08-08T15:06:31.765101+03:00",
  "SourceModuleName": "udp_listen",
  "SourceModuleType": "im_udp"
}

Configuring SNMP traps via the command line

Connect to the BIG-IP device via SSH. In case of a High Availability (HA) group, ensure you log in to the active unit. You should see (Active) in the command prompt.

Enable the predefined traps as required.

# tmsh modify sys snmp bigip-traps enabled
# tmsh modify sys snmp agent-trap enabled
# tmsh modify sys snmp auth-trap enabled

Create an SNMP user (SNMPv3 only). Replace USERNAME and PASSWORD with your desired values.

# tmsh modify sys snmp users add { USERNAME { username USERNAME auth-protocol sha privacy-protocol aes auth-password PASSWORD  privacy-password PASSWORD } }

For example, this command creates a user snmp_user that uses MD5 authentication and encrypts the SNMPv3 data payload with AES. Refer to K13625: Overview of SNMPv3 agent access for more information.

# tmsh modify sys snmp users add { snmp_user { username snmp_user auth-protocol md5 privacy-protocol aes auth-password q1w2e3r4 privacy-password q1w2e3r4 } }

Configure the remote SNMP destination. Replace NAME, COMMUNITY, IP_ADDRESS, and PORT with the appropriate values. Replace NETWORK with other or management if traps go out of the management interface.

# tmsh modify sys snmp traps add { NAME { community COMMUNITY host IP_ADDRESS port PORT network NETWORK } }

For example, this command enables forwarding SNMPv2 traps to 192.168.100.85.

# tmsh modify sys snmp traps add { 192_168_6_143 { community nxlog host 192.168.100.85 port 162 network other } }

When using SNMPv3, you need to specify additional parameters, including security-level, auth-protocol, auth-password, privacy-protocol, and privacy-password. For example, this command enables forwarding SNMPv3 traps to 192.168.100.85 using SHA and AES.

# tmsh modify sys snmp traps add { nxlog { version 3 host 192.168.100.85 port 162 network other security-level auth-privacy security-name snmp_user auth-protocol sha auth-password q1w2e3r4 privacy-protocol aes privacy-password q1w2e3r4 } }

If the BIG-IP configuration has been migrated or cloned, SNMPv3 may not work because the SNMP engine ID is not unique. If you encounter this problem, reset the engine ID as described in K6821: SNMP v3 fails if the SNMP engine ID is not unique.

In case of a High Availability (HA) group, synchronize the configuration changes to the other units.
```
# tmsh run cm config-sync to-group GROUP_NAME
```

Configuring SNMP traps via the web interface

Open https://BIG_IP_SERVER_IP and log in with your account. Replace BIG_IP_SERVER_IP with the IP address of your device. In case of a High Availability (HA) group, ensure you are logged in to the active unit. You should see ONLINE (ACTIVE) in the top left corner.
Navigate to System > SNMP > Traps. Select the required SNMP events and click Update.
Create an SNMP user (SNMPv3 only). Navigate to System > SNMP > Agent > Access (v3). Click Create and specify the user name, authentication type and password, privacy protocol and password, and access type. Specify an OID value to limit access to certain OIDs or use .1 to allow full access.
Navigate to System > SNMP > Traps > Destination and click Create. Specify the SNMP version, community name, destination IP address, port, and network, then click Finished.

SNMPv3 requires additional parameters, including security, authentication, and privacy settings.

In case of a High Availability (HA) group, synchronize the configuration changes to the other units.

Click the yellow Changes Pending in the top left corner.
Select the active device marked as (Self).

Enable the Sync Device to Group option and click Sync.

F5 high-speed logging

F5 BIG-IP devices support High-Speed Logging (HSL). This protocol sends as much data as the remote destination can accept. Combined with load balancing, BIG-IP makes it possible to configure multiple NXLog agents in one of the available load balancing methods.

BIG-IP can send its internal logs via HSL in addition to logs passing through the device. Since the load balancer usually resides on the network’s edge and all web traffic passes through it, collecting logs from BIG-IP may be easier and faster than processing individual web server logs.

HSL supports forwarding logs via TCP or UDP. TCP guarantees reliable delivery. However, when load balancing is enabled, BIG-IP will reuse existing TCP connections to each node to reduce overhead. As a result, load distribution between members may be less accurate.

The steps below have been tested with BIG-IP v12.

To configure HSL, you must create a node for each NXLog agent and add it to a pool:

Connect to the BIG-IP device via SSH.

Create a node for each NXLog agent. Replace NODE_NAME and IP_ADDRESS with the relevant values.

# tmsh create ltm node NODE_NAME { address IP_ADDRESS session user-enabled }

For example, this command creates a node nxlog1.

# tmsh create ltm node nxlog1 { address 192.168.100.85 session user-enabled }

Create a pool with all the nodes.

# tmsh create ltm pool POOL_NAME { members add { NODE1:PORT { address IP_ADDRESS1 }} NODE2:PORT { address IP_ADDRESS2 }} monitor PROTOCOL

For example, this command creates a pool named nxlog-pool with the node created in the previous step.

# tmsh create ltm pool nxlog-pool { members add { nxlog1:1514 { address 192.168.100.85 }} monitor tcp }

Forwarding internal BIG-IP logs to a High-Speed Logging pool

Follow these steps to send internal BIG-IP logs to the pool created above.

Connect to the BIG-IP device via SSH.
Create a remote logging destination. Replace DESTINATION_NAME with a name for the destination, POOL_NAME with the name you specified when you created the pool, DISTRIBUTION with one of the distribution options listed below, and PROTOCOL with tcp or udp. Distribution options include:

Adaptive

Sends to one of the pool members until this member cannot process logs at the required rate or the connection is lost.

Balanced

Uses the load balancing method configured on the pool and selects a new member each time it sends data.

Replicated

Sends each log to all members of the pool.
```
# tmsh create sys log-config destination remote-high-speed-log DESTINATION_NAME pool-name POOL_NAME distribution DISTRIBUTION protocol PROTOCOL
```
Create a log publisher. Replace PUBLISHER_NAME with a name for the publisher and DESTINATION_NAME with the destination name used in the previous step.
```
# tmsh create sys log-config publisher PUBLISHER_NAME destinations add {DESTINATION_NAME}
```

Create a log filter. Replace FILTER_NAME with a name for the filter, LEVEL with the required logging level between Emergency and Debugging, PUBLISHER_NAME with the name used in the previous step, and SOURCE with a specific BIG-IP process or all to forward all logs.

# tmsh create sys log-config filter FILTER_NAME level LOGGING_LEVEL publisher PUBLISHER source SOURCE

For example, the following commands will send all internal logs to the NXLog pool via TCP.

# tmsh create sys log-config destination remote-high-speed-log nxlog-hsl pool-name nxlog distribution adaptive protocol tcp
# tmsh create sys log-config publisher bigip-local-logs destinations add {nxlog-hsl}
# tmsh create sys log-config filter bigip-all-local-logs level debug publisher bigip-local-logs source all

Forwarding network logs to a High-Speed Logging pool

Follow these steps on each virtual server to configure BIG-IP to forward network logs.

Configure NXLog to receive syslog messages via TCP; see Receiving logs from BIG-IPs below. Restart NXLog.
Create a logging profile for requests. In most cases, it is enough to log requests only. However, you can also configure a profile to log responses and errors. In the example below:

LOGGING_PROFILE_NAME

A name for the new logging profile.

PROTOCOL

mds-tcp or mds-udp, depending on whether NXLog is listening for TCP or UDP connections.

POOL

The name of the pool created above.

TEMPLATE

A list of HTTP request fields to log. See Configuring the BIG-IP System as a DHCP Relay Agent in the BIG-IP Local Traffic Manager: Implementations guide for more information.
```
# tmsh create ltm profile request-log LOGGING_PROFILE_NAME { request-log-protocol PROTOCOL request-log-pool POOL_NAME request-logging enabled request-log-template "TEMPLATE" }
```

Assign the logging profile to a virtual server. The same logging profile can be assigned to multiple virtual servers. Replace VIRTUAL_SERVER_NAME with the virtual server name and LOGGING_PROFILE_NAME with the profile name used in the previous step.

# tmsh create ltm virtual VIRTUAL_SERVER_NAME {profiles add {LOGGING_PROFILE_NAME {}}}

For example, the following commands configure logging to the NXLog pool via TCP.

# tmsh create ltm profile request-log LOGGING_PROFILE_NAME { request-log-protocol mds-tcp request-log-pool POOL_NAME request-logging enabled request-log-template "client $CLIENT_IP:$CLIENT_PORT request $HTTP_REQUEST server $SERVER_IP:$SERVER_PORT status $HTTP_STATUS" }
# tmsh modify ltm virtual VIRTUAL_SERVER_NAME {profiles add {LOGGING_PROFILE_NAME {}}}

Example 5. Receiving logs from BIG-IP High-Speed Logging

This configuration uses the im_tcp input module to listen for logs on TCP port 1514.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input tcp_listen>
    Module        im_tcp
    ListenAddr    0.0.0.0:1514
    <Exec>
        parse_syslog(); (1)
        to_json(); (2)
    </Exec>
</Input>

1	Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module.
2	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
  "MessageSourceAddress": "192.168.6.91",
  "EventReceivedTime": "2017-05-10 19:16:43",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "SyslogFacilityValue": 1,
  "SyslogFacility": "USER",
  "SyslogSeverityValue": 5,
  "SyslogSeverity": "NOTICE",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventTime": "2017-05-10 19:16:43",
  "Hostname": "192.168.6.91",
  "Message": "client 192.168.9.78:63717 request GET /cmedia/img/icons/mime/mime-unknown.png?v170509919 HTTP/1.1 server 192.168.6.101:80 status "
}

NXLog can also be configured to parse log records according to the sequence of fields specified in the template.

Example 6. Parsing HSL records

This configuration uses the im_tcp input module to listen for logs on TCP port 1514 and a regular expression to parse records into structured data.

nxlog.conf

<Extension syslog>
    Module        xm_syslog
</Extension>

<Extension json>
    Module        xm_json
</Extension>

<Input tcp_listen>
    Module        im_tcp
    ListenAddr    0.0.0.0:1514
    <Exec>
        parse_syslog(); (1)
        if $Message =~ /^client (.*) request (.*) server (.*) status (.*)$/ (2)
        {
            $HTTP_Client = $1;
            $HTTP_Request = $2;
            $HTTP_Server = $3;
            $HTTP_Status = $4;
            delete($Message); (3)
        }
        to_json(); (4)
    </Exec>
</Input>

1	Parses log records into structured data using the parse_syslog() procedure of the xm_syslog module, which creates the `$Message` field.
2	Uses a regular expression to extract the HTTP client, request, server, and status from the `$Message` field.
3	Deletes the redundant `$Message` field with the delete() procedure.
4	Converts log records to JSON format using the to_json() procedure of the xm_json module.

Output sample

{
  "MessageSourceAddress": "192.168.6.91",
  "EventReceivedTime": "2017-05-10 20:06:24",
  "SourceModuleName": "tcp_listen",
  "SourceModuleType": "im_tcp",
  "SyslogFacilityValue": 1,
  "SyslogFacility": "USER",
  "SyslogSeverityValue": 5,
  "SyslogSeverity": "NOTICE",
  "SeverityValue": 2,
  "Severity": "INFO",
  "EventTime": "2017-05-10 20:06:24",
  "Hostname": "192.168.6.91",
  "HTTP_Client": "192.168.9.78:65275",
  "HTTP_Request": "GET /?disabledcookies=true HTTP/1.1",
  "HTTP_Server": "192.168.6.100:80",
  "HTTP_Status": ""
}

Using BIG-IP load balancing for an NXLog agent pool

You can use BIG-IP as a load balancer to distribute traffic between multiple NXLog agents. To accomplish this, you must create a virtual server that accepts syslog messages and forwards them to the pool.

For example, this command creates a virtual server named nx-virtual-serv that listens on IP address 192.168.100.85 TCP port 1514 and forwards logs to the pool nxlog-pool.

# tmsh create ltm virtual nx-virtual-serv { destination 192.168.100.85:1514 mask 255.255.255.255 pool nxlog-pool profiles add { tcp{} } }

After you execute this command, configure your log sources to forward syslog messages to the specified IP address.

Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

NXLog 5.5.7535
BIG-IP v17.0.0

Last revision: 10 August 2022