The NXLog Enterprise Edition offers both agentless and agent based log collection so it's possible to use it in both ways or even in mixed mode.
In most enterprise environments the common data sources are mainly Syslog, local log files (IIS, DNS, custom app logs) and Windows Eventlog. Let's take a look at how this event data can be collected with the agentless or the agent based method.
Structured logging is on the rise. A lot of tools and logging services are finally moving towards structured logging and JSON seems to be the format of choice for this.
But what is structured logging? Traditionally logs were generated in the form of free form text messages prepended with some basic metadata such as the time of the event, severity and the source of the event. This is what the old RFC3164 style Syslog format looks like:
<30>Nov 21 11:40:27 myhost sshd[26459]: Accepted publickey for myhost from 192.168.1.1 port 424242 ssh2
This traditional syslog fromat has a header consisting of the severity, facility, timestamp, hostname and process name followed by a free form text string optionally containing additional metadata. The advantage of this is that log data in this format can be easily parsed and stored in text files for humans to look at.
The popularity of the ELK stack is steadily rising, many NXLog users send their event data to Elasticsearch and Kibana for log monitoring and analytics.
There are many tutorials and configurations scattered around on the web, some come with configuration samples that will likely not work properly. For this reason we have written a short document introducing different options on how to use NXLog with Elasticsearch and Kibana, it's available under the documentation page.
Paul Nelson from Opsbot has written an excellent article titled Windows as a First-Class Centralized Logging Citizen discussing centralized logging using open source tools. Well worth the read!
Update: the article is no longer available.
Loggly offers cloud based storage and analytics services for log data. NXLog can be used to collect and send logs off to the Loggly service.
Below is a configuration that can be used for a start. Make sure to set the value of CUSTOMER_TOKEN properly. If you are unsure where to get this, see the article about the cusomer token in the Loggly support center.
