Puppet Bolt is an open-source orchestration tool that automates the manual configuration and management of your infrastructure.

In this post, we will look at how you can create your Puppet Bolt project directory, your inventory YAML file, and finally, your Puppet Bolt Plan to deploy NXLog on a variety of Operating Systems.

Why use Puppet Bolt to deploy NXLog?

Apart from the usual tasks of updating software packages, configuring web servers and databases, the need for constant logging has become extremely important, and a de facto necessity nowadays. In addition, there is also the sheer number of connected hosts generating valuable logs that will require NXLog to be installed and configured, if these logs are to be captured and forwarded to a remote destination for further processing or analysis.

With that being said, the ability to simultaneously install and configure NXLog using Puppet Bolt can be of great value to system administrators. This post assumes you already have a working Puppet Bolt installation. If not, please visit the Puppet Bolt Installation Docs and refer to the detailed instructions provided for your operating system.

Creating a Puppet Bolt project

Creating your project directory is pretty simple. Just run these commands:

mkdir -p nxlog_bolt; cd nxlog_bolt; bolt project init

With these commands we are creating a project directory named nxlog_bolt, navigating to the newly created project directory, and then invoking a Bolt-specific command bolt project init which will initialize a new Puppet Bolt project.

At this point, if you execute ls -a you will notice that Bolt has created three files:

  1. bolt-project.yaml.

  2. inventory.yaml.

  3. A hidden .gitignore file.

Next, we need to create a module that will contain our NXLog directory and two sub-directories, plans, and files.

Issue the following command to create this directory structure:

mkdir -p modules/nxlog/{files,plans}

  • plans: The directory containing our plan

  • files: The directory containing the various files (scripts, configuration files, images, etc.) that we want to upload to the remote server targets

The Puppet Bolt project file

This was created when you executed the bolt project init command. Edit the bolt-project.yaml file to make it look exactly like this:

bolt-project.yaml
---
name: nxlog_bolt
modules: []

color: true

plans:
  - nxlog::nxlog_install

This file contains the plan definition. In our case it is nxlog::nxlog_install.

The Puppet Bolt inventory file

The inventory file stores information about your target servers. Specifically, it defines how Bolt connects to them and lets you assign them to groups. When target servers are assigned to groups, executing commands on a group rather than on each target server individually is more efficient and easier to manage over time.

An inventory file is part of a Bolt project and must exist alongside the bolt-project.yaml file. The following inventory file example defines two groups, The first group is named linux, which includes a Ubuntu Focal and a CentOS 7 operating system. The second group is called windows, which includes a Windows machine.

inventory.yaml
groups:
  - name: linux
    targets:
      - uri: 192.168.1.12
        name: ubuntu
      - uri: 192.168.1.11
        name: centos
    config:
      transport: ssh
      ssh:
        user: root
        password: your_linux_password
        host-key-check: false

  - name: windows
    targets:
      - name: windows10
        uri: 192.168.1.10
    config:
      transport: winrm
      winrm:
        user: Administrator
        password: your_administrator_password
        ssl: false

Replace the values of the user, password, and uri fields to match those of the servers you are targeting. For testing Bolt’s network connectivity and authentication with the targets, we can use the whoami command, common to both Windows and Linux systems.

Execute the command:

$ bolt command run whoami -t all

Normally, the expected output should be:

Started on ubuntu...
Started on centos...
Started on windows10...
Finished on ubuntu:
  root
Finished on windows10:
  hostname\Administrator
Finished on centos:
  root
Successful on 3 targets: ubuntu,centos,windows10
Ran on 3 targets in 0.36 sec

If you run into problems, check your /etc/ssh/sshd_config to ensure that your OpenSSH server is configured to accept root logins by setting PermitRootLogin yes. Furthermore, enable password authentication by uncommenting PasswordAuthentication yes.

As you can see from the output above, we managed to successfully connect to our target servers and execute the whoami command on each one of them.

Creating a Puppet Bolt plan

Before you start creating your plan, a few prerequisites have to be met:

  • Puppet Bolt installed on your local workstation

  • A set of machines (the target nodes), each having a known, accessible hostname, FQDN, or IP address

  • Root access and an operational SSH server on Linux machines

  • WinRM and Administrator access enabled on Windows machines

Tip

To enable WinRM, open PowerShell with administrative rights, then run the commands Set-NetConnectionProfile -NetworkCategory Private, WinRM quickconfig, and lastly Enable-PSRemoting.

To enable Administrator access on the Windows platform, run the command: net user Administrator /active:yes.
Note
 We are forced to use WinRM rather than SSH to connect to Windows due to a Puppet Bolt bug that prevents a required temporary file from being created during file uploads.

In case you already have your ssh keys set up you can omit the password: your_root_password from the linux group in the inventory.yaml file. Note that in our example, key-based authentication is not required, even though is recommended for production systems.

There are two options to create a Puppet Bolt plan. The first option is to create a YAML file, and the second option is to create a .pp (puppet language) file. For the sake of simplicity, we will use a YAML file in this example.

nxlog_install.yaml
---
parameters:
  targets:
    type: TargetSpec

steps:
  - name: upload_ubuntu_nxlog_executable
    upload: nxlog/files/nxlog-5.4.7313_ubuntu20_amd64.tar.bz2
    destination: /root/
    targets: ubuntu
    description: Uploaded NXLog Archive

  - name: nxlog_ubuntu_install
    command: apt install -y libapr1 libdbi1 curl openjdk-8-jdk;
      mkdir -p nxlog; tar -xvf nxlog-5.4.7313_ubuntu20_amd64.tar.bz2 -C nxlog;
      dpkg -i nxlog/nxlog-5.4.7313_ubuntu20_amd64.deb;
      dpkg -i nxlog/*.deb;
      apt install -y -f;
      #rm -rf nxlog
    catch_errors: true
    targets: ubuntu
    description: "Install NxLog On Ubuntu using the .deb package"

  - name: nxlog_ubuntu_configure
    upload: nxlog/files/ubuntu.conf
    destination: /opt/nxlog/etc/nxlog.d/ubuntu.conf
    targets: ubuntu
    description: Uploaded Ubuntu Configuration File

  - name: restart_nxlog_ubuntu
    command: systemctl restart nxlog
    targets: ubuntu
    description: Restarted Ubuntu NXLog Service

#===============================================================================================================================#
  - name: upload_centos_nxlog_executable
    upload: nxlog/files/nxlog-5.4.7313_rhel7_x86_64.tar.bz2
    destination: /root/
    targets: centos
    description: Uploaded CentOS 7 NXLog Archive

  - name: nxlog_centos_install
    command: yum install -y dnf java-1.8.0-openjdk;
      mkdir -p nxlog; tar -xvf nxlog-5.4.7313_rhel7_x86_64.tar.bz2 -C nxlog;
      dnf install -y nxlog/nxlog-5.4.7313_rhel7_x86_64.rpm;
      dnf install -y nxlog/*.rpm;
    catch_errors: true
    targets: centos
    description: "Install NxLog using On CentOS7 using the .rpm package"

  - name: nxlog_centos_configure
    upload: nxlog/files/centos.conf
    destination: /opt/nxlog/etc/nxlog.d/centos.conf
    targets: centos
    description: Uploaded CentOS 7 Configuration File

  - name: restart_nxlog_centos
    command: systemctl restart nxlog
    targets: centos
    description: Restarted CentOS 7 NXLog Service

#===============================================================================================================================#
  - name: upload_windows_nxlog_executable
    upload: nxlog/files/nxlog-5.4.7313_windows_x64.msi
    #tmpdir: C:\Users\administrator
    destination: C:\Users\administrator
    targets: windows
    description: Uploaded Windows NXLog Executable

  - name: nxlog_windows_install
    command: msiexec /i 'C:\Users\administrator\nxlog-5.4.7313_windows_x64.msi' /q;
    catch_errors: true
    targets: windows
    description: "Install NxLog On Windows Using the .msi package"

  - name: nxlog_windows_configure
    upload: nxlog/files/windows.conf
    destination: C:\Program Files\nxlog\conf\nxlog.d\windows.conf
    targets: windows
    description: Uploaded Windows 10 Configuration File

  - name: restart_nxlog_windows
    command: Stop-Service -Name "nxlog";Start-Service -Name "nxlog";
    targets: windows
    description: Restarted Windows NXLog Service

The plan schema

Even though clarifying details regarding the plan is beyond the scope of this post, we thought it is worth sharing some details about the plan schema, which helps you get better acquainted with the structure of this file.

parameters

The parameters used in the plan. In our case, the parameter is targets.

steps

Step declaration

name

The name of each step

command

The command that is executed in the remote target servers

catch_errors

Returns the error if an error is raised, or returns the result of the block if no errors are raised

targets

Target servers as described in the inventory.yaml file

description

Description of each step

Puppet plan steps

Upon execution of the plan above, Puppet Bolt will conduct three tasks for each platform. This is what exactly happens under the hood:

  1. Uploading the NXLog installation file

  2. Installing the appropriate, platform-specific NXLog installation package

  3. Copying the suitable NXLog configuration file to its paired remote server

  4. Restarting NXLog to load the newly deployed configuration

Now that we have finished writing our Puppet plan, we can configure NXLog for each operating system.

NXLog configuration files

Since we want to automate the entire process, we need to upload an NXLog configuration file for each operating system. To achieve this, we need to place each configuration file in the files directory we created earlier. This Puppet Bolt’s default behavior for uploading files to a remote target.

Note
 An in-depth explanation of these NXLog configuration examples is beyond the scope of this post. However, it should be noted that they were taken from real, working NXLog agents.

Below we will create these files and show their contents. If you have been following along, your current working directory should be nxlog_bolt.

Execute the following command to create the configuration files:

$ touch modules/nxlog/files/{windows,ubuntu,centos}.conf

The content of each file is shown below:

Example 1. Windows NXLog configuration

This configuration example collects logs from Windows Event Log using the im_msvistalog module. Each event from the log source is then converted to syslog and sent over TCP port 514 using om_tcp to a remote host.

windows.conf
<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='0'>
                <Select Path='Application'>*</Select>
                <Select Path='Security'>*[System/Level&lt;4]</Select>
                <Select Path='System'>*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.10:514
    Exec      to_syslog_bsd();
</Output>

<Route eventlog_to_tcp>
    Path      eventlog => tcp
</Route>
Example 2. Ubuntu NXLog configuration

This configuration example collects logs from im_systemd module to read log messages from the Linux systemd journal. Each event from the log source is then converted to syslog and sent over TCP port 514 using om_tcp to a remote host.

ubuntu.conf
<Input systemd>
    Module    im_systemd
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.12:514
    Exec      to_syslog_bsd();
</Output>

<Route systemd_to_tcp>
    Path      systemd => tcp
</Route>
Example 3. CentOS NXLog configuration

This configuration example collects logs using the im_file module to read log messages from the Linux file system. Each event from the log source is then converted to syslog and sent over TCP port 514 using om_tcp to a remote host.

centos.conf
<Input file>
    Module    im_file
    File      "/var/log/messages"
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.11:514
    Exec      to_syslog_bsd();
</Output>

<Route file_to_tcp>
    Path      file => tcp
</Route>

Plan execution

To begin executing all steps as described in the Puppet Bolt plan, execute the command:

$ bolt plan run nxlog::nxlog_install -t all

As you can see, it’s a very simple command, but it’s worthy of some further explanation. This command is comprised of five parts:

bolt

The executable itself located in /usr/local/bin

plan

Tells Bolt that it will be using a plan

nxlog::nxlog_install

The plan reference definition, that consists of two parts, module name::filename<br> 1. module name, the name of the directory we created earlier 2. filename, the name of the file located in the plans folder

-t

The target servers In our case, we used the all option, but you could also use linux if you wanted to target only the linux servers, or windows for targeting only Windows hosts

Conclusion

With Puppet Bolt, you can instantaneously configure and update a large number of NXLog agents thus saving an enormous amount of time while avoiding the inevitable errors associated with a manual deployment strategy. Using Puppet Bolt to install and configure NXLog agents is an excellent choice for both on-premises environments as well as cloud-based ones.

NXLog Platform is an on-premises solution for centralized log management with
versatile processing forming the backbone of security monitoring.

With our industry-leading expertise in log collection and agent management, we comprehensively
address your security log-related tasks, including collection, parsing, processing, enrichment, storage, management, and analytics.

Start free Contact us
Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

Windows 10, Ubuntu 20.04.4 LTS, RHEL 7
NXLog CE 3.0.2272
NXLog EE 5.4.7313

Last revision: 11 March 2022

Share
Related Posts