News and blog
NXLog main page
  • Products
    NXLog Platform
    Log collection
    Log management and analytics
    Log storage
    NXLog Community Edition
    Integrations
    Professional Services
  • Solutions
    Use cases
    Specific OS support
    SCADA/ICS
    Windows event log
    DNS logging
    MacOS logging
    Solutions by industry
    Financial Services
    Government & Education
    Entertainment & Gambling
    Telecommunications
    Medical & Healthcare
    Military & Defense
    Law Firms & Legal Counsel
    Industrial & Manufacturing
  • Plans
  • Partners
    Find a Reseller
    Partner Program
  • Resources
    Documentation
    Blog
    White papers
    Videos
    Webinars
    Case Studies
    Community Program
    Community Forum
  • About
    Company
    Careers
  • Support
    Support portals
    Contact us

NXLog Platform
Log collection
Log management and analytics
Log storage
NXLog Community Edition
Integrations
Professional Services

Use Cases
Specific OS support
SCADA/ICS
Windows event log
DNS logging
MacOS logging
Solutions by industry
Financial Services
Government & Education
Entertainment & Gambling
Telecommunications
Medical & Healthcare
Military & Defense
Law Firms & Legal Counsel
Industrial & Manufacturing


Find a Reseller
Partner Program

Documentation
Blog
White papers
Videos
Webinars
Case Studies
Community Program
Community Forum

Company
Careers

Support portals
Contact us
Let's Talk Start free
NXLog search
  • Loading...
Let's Talk Start free
March 1, 2022 deployment

Deploying and managing NXLog with Ansible

By Tamás Burtics

Share
ALL SIEM STRATEGY SECURITY ANNOUNCEMENT DEPLOYMENT COMPLIANCE COMPARISON RSS

Ansible has become an industry standard when it comes to configuring and managing servers. As a configuration management tool, it carries the burden of simplifying system administration tasks, such as installing and updating software packages, and infrastructure provisioning. In this post, we will create an Ansible playbook that will enable us to automate the installation and configuration of NXLog across multiple endpoints. Whether you need only a single endpoint today or thousands of endpoints next week, Ansible will do the heavy lifting for you.

Why should I use Ansible to deploy NXLog?

One of the biggest challenges that any IT department faces today is large scale workload deployment. Some organizations require the deployment of hundreds of thousands of servers to meet their computing needs, which will constantly log events for forwarding to a collection server, cluster of relay servers, or directly to a SIEM. Given such vast numbers of endpoints that will use NXLog as their logging solution, there is a clear need for an automated solution like Ansible, capable of concurrently installing and configuring nodes.

To meet this challenge, we will use an Ansible playbook that can connect to any number of server IPs we provide, and then upload the required NXLog configuration file.

To install Ansible, please refer to Ansible Installation Docs for detailed instructions based on your operating system.

Writing your Ansible playbook

An Ansible playbook is a predefined set of instructions written in the YAML programming language that are executed on the remote servers to be managed. Under the hood, an SSH connection is established with the remote servers where bash commands are run.

Before you start writing your playbook, a few prerequisites have to be met:

  • A functional Ansible installation on your local machine (the control node) with an SSH client and a public-private key pair.

  • A provisioned set of servers (the target nodes), each having a known, accessible IP address or domain name, and each running an operational SSH server with key based authentication.

  • Root access on all servers (the target nodes).

At the very first step, we need a directory and file structure to work in. The below command will create a directory named ansible with a config subdirectory, with the three configuration files: playbook.yml, vars.yml, and inventory being in the same level as the config subdirectory. In addition, the script also creates the rhel.conf, ubuntu.conf, and windows.conf files in the config subdirectory. It then changes our current working directory to the ansible directory.

$ mkdir -p ansible; touch ansible/playbook.yml; touch ansible/vars.yml; touch ansible/inventory; mkdir -p ansible/config; cd ansible; touch config/{windows,ubuntu,rhel}.conf;

Once we have the directory structure, it is time to add content to the files. There are three of them to start with. We gave them real content and the content is explained after the samples.

Example 1. inventory
[nxlog_servers]
ubuntu ansible_ssh_host=192.168.1.10
rhel ansible_ssh_host=192.168.1.11
windows ansible_ssh_host=192.168.12

inventory is merely a file that groups our servers into categories. In our case, we’ll have single category called nxlog_servers.

Example 2. vars.yaml
ubuntu_focal_url: https://nxlog.co/system/files/products/files/348/nxlog-ce_3.0.2272_ubuntu_focal_amd64.deb
rhel_8_url: https://nxlog.co/system/files/products/files/348/nxlog-ce-3.0.2272_rhel8.x86_64.rpm
windows_url: https://nxlog.co/system/files/products/files/348/nxlog-ce-3.0.2272.msi

As can be seen in the YAML file above, each file’s location expressed as a URL is assigned to a YAML key. These keys can then be used, e.g. {{ rhel_8_url }}, instead of the long URLs in playbook.yml below to make it easier to read and maintain.

Example 3. playbook.yaml
---
- hosts: nxlog_servers
  vars_files:
    - vars.yml
  become: true
  become_user: root

  tasks:
    - name: Install NXLog on Ubuntu Focal
      apt:
        deb: "{{ ubuntu_focal_url }}"
      when: "ansible_distribution == 'Ubuntu' and ansible_distribution_release == 'focal'"

    - name: Copy a new "ubuntu.conf" file into the NXLog default config directory
      copy:
        src: config/ubuntu.conf
        dest: /opt/nxlog/etc/nxlog.d/ubuntu.conf
        owner: nxlog
        group: nxlog
        mode: '0755'
      when: "ansible_distribution == 'Ubuntu' and ansible_distribution_release == 'focal'"

    - name: Start NXLog on Ubuntu Focal
      command: service nxlog start
      when: "ansible_distribution == 'Ubuntu' and ansible_distribution_release == 'focal'"

#=============================================================================================================#
    - name: Install NXLog on RHEL 8 (Centos 8)
      yum:
        name="{{ rhel_8_url }}"
      when: "ansible_distribution == 'RedHat' and ansible_distribution_major_version == '8'"

    - name: Copy a new "rhel.conf" file to the NXLog default config directory
      copy:
        src: config/rhel.conf
        dest: /opt/nxlog/etc/nxlog.d/rhel.conf
        owner: nxlog
        group: nxlog
        mode: '0755'
      when: "ansible_distribution == 'RedHat' and ansible_distribution_major_version == '8'"

    - name: Start NXLog for RHEL 8 (Centos 8)
      command: systemctl start nxlog
      when: "ansible_distribution == 'RedHat' and ansible_distribution_major_version == '8'"

#==============================================================================================================#
    - name: Install NXLog on Windows
      win_package:
        path: "{{ windows_url }}"
        state: present
      when: ansible_distribution == 'Windows'

    - name: Copy a new "windows.conf" file into the NXLog default config directory
      copy:
        src: config/windows.conf
        dest: C:\Program Files\nxlog\conf\nxlog.d\windows.conf
        owner: nxlog
        group: nxlog
        mode: '750'
      when: ansible_distribution == 'Windows'

    - name: Set nxlog service startup mode to auto and ensure it's started.
      win_service:
        name: nxlog
        state: restarted 

One very useful feature of Ansible’s copy plugin, is that it always compares the checksum of the src (local) file with the checksum of the dest (remote) file. The transfer will only be initiated if the checksums are different. These three configuration files are pretty much all that Ansible needs to deploy NXLog agents to your remote servers. The next order of business is to appropriately create a configuration file for each operating system. To achieve this, we created three configuration files, one for each type of server.

The Playbook schema

Even though it is not the goal of this post to be an Ansible User Guide, we thought it is worth sharing the explanation of the playbook schema. It could help you better understand the structure of the file.

hosts

Defines which remote servers are target nodes and will be accessed remotely for performing the tasks defined in the tasks section.

vars_files

Files containing custom-defined variables that the playbook can use. Multiple "var" files can be declared. As you may have noticed by now from our vars.yml, variables we declared in it are enclosed in double curly brackets, which are enclosed in double-quotes. This is Ansible’s syntax for expanding a variable to its assigned value within the playbook.

become

This instructs Ansible that it will be using a username other than the one that is associated with the SSH key-pair. In the playbook presented above, the SSH user is root. This is required because NXLog is not installed in the regular user’s home directory. By default, regular users typically don’t have write permissions beyond their home directories.

become_user

This is the user that was just discussed above: in our case root. Depending on the task, it might be different.

tasks

A list of the tasks that will be executed on the target nodes.

Each task has multiple keys:

  • name: The name or description of the task.

  • apt: Used to invoke the Debian/Ubuntu native package manager that installs, upgrades, or removes a program.

  • deb: Path to a .deb package. Can be a URL or a local file. If:// in the path, Ansible will attempt to download deb before installing.

  • command: The command to run on the remote machine. Alternatively, cmd: | can be used to execute multiple commands instead.

  • yum: Used to invoke the RHEL native package manager that installs, upgrades, or removes a software package.

  • win_package: Used to invoke the Windows Installer, that installs, upgrades, or removes a program.

  • when: Is used to define basic conditionals. It’s similar to if-else which is used in many programming languages.

Playbook tasks

By running the playbook above, we are taking advantage of Ansible’s automation capabilities which will perform three tasks for each operating system. This is what exactly happens:

  1. Deploying the correct, platform-appropriate NXLog installation package.

  2. Copying the appropriate NXLog configuration file to its appropriate remote node.

  3. Restarting NXLog to load the newly deployed configuration.

Now that we have finished writing our Ansible Playbook and we clear about what it will do, we can write our NXLog configuration file for each server.

NXLog configuration files

The in-depth explanation of the NXLog configuration files are out of the scope of this article, yet we are using real examples, so it is worth showing them. Do not forget, these files were created earlier in the, we are just giving content here. The are in the /ansible/config directory.

Example 4. Windows NXLog configuration

This example configuration collect logs from Windows Event Log using the im_msvistalog module, then sends them to another node using om_tcp.

windows.conf
<Extension syslog>
    Module    xm_syslog
</Extension>

<Input eventlog>
    Module    im_msvistalog
    <QueryXML>
        <QueryList>
            <Query Id='0'>
                <Select Path='Application'>*</Select>
                <Select Path='Security'>*[System/Level&lt;4]</Select>
                <Select Path='System'>*</Select>
            </Query>
        </QueryList>
    </QueryXML>
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.12:514
    Exec      to_syslog_bsd();
</Output>

<Route eventlog_to_tcp>
    Path      eventlog => tcp
</Route>
Example 5. Ubuntu NXLog configuration

This example configuration uses the im_systemd module to read log messages from the Linux systemd journal, then sends them to another node using om_tcp.

ubuntu.conf
<Input systemd>
    Module    im_systemd
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.10:514
    Exec      to_syslog_bsd();
</Output>

<Route systemd_to_tcp>
    Path      systemd => tcp
</Route>
Example 6. RHEL NXLog configuration

This example configuration uses the im_file module to read log messages from the Linux file system, then sends them to another node using om_tcp.

rhel.conf
<Input file>
    Module  im_file
    File    "/var/log/messages"
</Input>

<Output tcp>
    Module    om_tcp
    Host      192.168.1.11:514
    Exec      to_syslog_bsd();
</Output>

<Route file_to_tcp>
    Path      file => tcp
</Route>

Playbook execution

To begin executing all tasks described in the Ansible Playbook, execute the command:

$ ansible-playbook -i inventory -u your_user_name --ask-become-pass --private-key ~/.ssh/ansible_id_ed25519  playbook.yml

Let’s take a closer look at the following options that we used in the command above:

  • -i: The name of the inventory file your playbook will use for logging in to the servers

  • -u: The username the playbook will be using

  • -ask-become-pass: The user’s password to execute commands with sudo privileges.

  • -private-key: The private key location you have already created. This example is using the ed25519 algorithm, since it performs much faster and provides the same level of security with significantly smaller keys.

Conclusion

In this post, we explored a practical example of how to deploy and manage NXLog to any number of servers using Ansible. One of the biggest advantages of NXLog is that it’s a cloud-agnostic log collection software, making it an ideal choice for deployment to your current cloud infrastructure. With Ansible, any changes to your NXLog configurations can be easily and instantly applied to an unlimited number of target nodes.

NXLog Platform is an on-premises solution for centralized log management with
versatile processing forming the backbone of security monitoring.

With our industry-leading expertise in log collection and agent management, we comprehensively
address your security log-related tasks, including collection, parsing, processing, enrichment, storage, management, and analytics.

Start free Contact us
Disclaimer

While we endeavor to keep the information in this topic up to date and correct, NXLog makes no representations or warranties of any kind, express or implied about the completeness, accuracy, reliability, suitability, or availability of the content represented here. We update our screenshots and instructions on a best-effort basis.

The accurateness of the content was tested and proved to be working in our lab environment at the time of the last revision with the following software versions:

Windows 10, Ubuntu 20.04.4 LTS, RHEL 8
NXLog CE 3.0.2272
NXLog EE 5.4.7313

Last revision: 3 March 2022

  • deploying nxlog
  • ansible
  • scm
  • integration
Share

Facebook Twitter LinkedIn Reddit Mail
Related Posts

Making the most of Windows Event Forwarding for centralized log collection
6 minutes | December 17, 2018
DNS Log Collection on Windows
8 minutes | May 28, 2020
DNS Log Collection and Parsing
5 minutes | May 31, 2020

Stay connected:

Sign up

Keep up to date with our monthly digest of articles.

By clicking singing up, I agree to the use of my personal data in accordance with NXLog Privacy Policy.

Featured posts

Announcing NXLog Platform 1.6
April 22, 2025
Announcing NXLog Platform 1.5
February 27, 2025
Announcing NXLog Platform 1.4
December 20, 2024
NXLog redefines log management for the digital age
December 19, 2024
2024 and NXLog - a review
December 19, 2024
Announcing NXLog Platform 1.3
October 25, 2024
NXLog redefines the market with the launch of NXLog Platform: a new centralized log management solution
September 24, 2024
Welcome to the future of log management with NXLog Platform
August 28, 2024
Announcing NXLog Enterprise Edition 5.11
June 20, 2024
Raijin announces release of version 2.1
May 31, 2024
Ingesting log data from Debian UFW to Loki and Grafana
May 21, 2024
Announcing NXLog Enterprise Edition 6.3
May 13, 2024
Raijin announces release of version 2.0
March 14, 2024
NXLog Enterprise Edition on Submarines
March 11, 2024
The evolution of event logging: from clay tablets to Taylor Swift
February 6, 2024
Migrate to NXLog Enterprise Edition 6 for our best ever log collection experience
February 2, 2024
Raijin announces release of version 1.5
January 26, 2024
2023 and NXLog - a review
December 22, 2023
Announcing NXLog Enterprise Edition 5.10
December 21, 2023
Raijin announces release of version 1.4
December 12, 2023
Announcing NXLog Enterprise Edition 6.2
December 4, 2023
Announcing NXLog Manager 5.7
November 3, 2023
Announcing NXLog Enterprise Edition 6.1
October 20, 2023
Raijin announces release of version 1.3
October 6, 2023
Upgrading from NXLog Enterprise Edition 5 to NXLog Enterprise Edition 6
September 11, 2023
Announcing NXLog Enterprise Edition 6.0
September 11, 2023
The cybersecurity challenges of modern aviation systems
September 8, 2023
Raijin announces release of version 1.2
August 11, 2023
The Sarbanes-Oxley (SOX) Act and security observability
August 9, 2023
Log Management and PCI DSS 4.0 compliance
August 2, 2023
Detect threats using NXLog and Sigma
July 27, 2023
HIPAA compliance logging requirements
July 19, 2023
Announcing NXLog Enterprise Edition 5.9
June 20, 2023
Industrial cybersecurity - The facts
June 8, 2023
Raijin announces release of version 1.1
May 30, 2023
CISO starter pack - Security Policy
May 2, 2023
Announcing NXLog Enterprise Edition 5.8
April 24, 2023
CISO starter pack - Log collection fundamentals
April 3, 2023
Raijin announces release of version 1.0
March 9, 2023
Avoid vendor lock-in and declare SIEM independence
February 13, 2023
Announcing NXLog Enterprise Edition 5.7
January 20, 2023
NXLog - 2022 in review
December 22, 2022
Need to replace syslog-ng? Changing to NXLog is easier than you think
November 23, 2022
The EU's response to cyberwarfare
November 22, 2022
Looking beyond Cybersecurity Awareness Month
November 8, 2022
GDPR compliance and log data
September 23, 2022
NXLog in an industrial control security context
August 10, 2022
Raijin vs Elasticsearch
August 9, 2022
NXLog provides native support for Google Chronicle
May 11, 2022
Aggregating macOS logs for SIEM systems
February 17, 2022
How a centralized log collection tool can help your SIEM solutions
April 1, 2020

Categories

  • SIEM
  • STRATEGY
  • SECURITY
  • ANNOUNCEMENT
  • DEPLOYMENT
  • COMPLIANCE
  • COMPARISON
logo

Subscribe to our newsletter to get the latest updates, news, and products releases. 

© Copyright 2024 NXLog FZE.

Privacy Policy. General Terms of Use

Follow us

  • Product
  • NXLog Platform 
  • Log collection
  • Log management and analysis
  • Log storage
  • Integration
  • Professional Services
  • Plans
  • Resources
  • Documentation
  • Blog
  • White papers
  • Videos
  • Webinars
  • Case studies
  • Community Program
  • Community forum
  • Support
  • Getting started guide
  • Support portals
  • About NXLog
  • About us
  • Careers
  • Find a reseller
  • Partner program
  • Contact us