NXlog is not connecting to ElasticSearch server

Tags:

#1 santosh.p

I am configuring the NXLog to Elastic Search Server using om_elasticserch, but it doesnt connect

Here is the nxlog.conf file:

User root Group system Panic Soft

default values:

PidFile /opt/nxlog/var/run/nxlog/nxlog.pid CacheDir /opt/nxlog/var/spool/nxlog

ModuleDir /opt/nxlog/lib/nxlog/modules

ModuleDir /opt/nxlog/libexec/nxlog/modules SpoolDir /opt/nxlog/var/spool/nxlog

define CERTDIR /opt/nxlog/var/lib/nxlog/cert define CONFDIR /opt/nxlog/var/lib/nxlog

Note that these two lines define constants only; the log file location

is ultimately set by the LogFile directive (see below). The

MYLOGFILE define is also used to rotate the log file automatically

(see the _fileop block).

define LOGDIR /opt/nxlog/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log

<Extension _json> Module xm_json </Extension>

<Input file> Module im_file File '/<path/to/my/logfile/*log' # Parse log here if needed # $EventTime should be set here </Input>

<Output out> Module om_elasticsearch URL http://linuxServer:9200/_bulk FlushInterval 2 FlushLimit 100

# Create an index daily
Index       strftime($EventTime, &quot;nxlog-%Y%m%d&quot;)

# Use the following if you do not have $EventTime set
#Index      strftime($EventReceivedTime, &quot;nxlog-%Y%m%d&quot;)

</Output>

By default, LogFile %MYLOGFILE% is set in log4ensics.conf. This

allows the log file location to be modified via NXLog Manager. If you

are not using NXLog Manager, you can instead set LogFile below and

disable the include line.

LogFile %MYLOGFILE% #include %CONFDIR%/log4ensics.conf

<Extension _syslog> Module xm_syslog </Extension>

This block rotates %MYLOGFILE% on a schedule. Note that if LogFile

is changed in log4ensics.conf via NXLog Manager, rotation of the new

file should also be configured there.

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    &lt;Exec&gt;
        if ( file_exists('%MYLOGFILE%') and
             (file_size('%MYLOGFILE%') &gt;= 5M) )
        {
             file_cycle('%MYLOGFILE%', 8);
        }
    &lt;/Exec&gt;
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>

NXlog is starting, but it waiting to connect to ElasticSearch server, not proceeding further. Here is the log in nxlog.log. And I could not see nxlog* index patter in Kibana

2020-03-31 09:37:44 WARNING already running as gid 0 2020-03-31 09:37:44 WARNING already running as uid 0 2020-03-31 09:37:44 WARNING no routes defined! 2020-03-31 09:37:44 WARNING default route added 2020-03-31 09:37:44 INFO nxlog-4.7.4715 started 2020-03-31 09:37:44 INFO om_elasticsearch_add_reconnect_event reconnecting in 0 seconds 2020-03-31 09:37:44 INFO connecting to linuxServer:9200

Any help on this is appricaited.

Thanks in advance

Santosh

#2 MisazivDeactivated Nxlog ✓
#1 santosh.p
I am configuring the NXLog to Elastic Search Server using om_elasticserch, but it doesnt connect Here is the nxlog.conf file: User root Group system Panic Soft default values: PidFile /opt/nxlog/var/run/nxlog/nxlog.pid CacheDir /opt/nxlog/var/spool/nxlog ModuleDir /opt/nxlog/lib/nxlog/modules ModuleDir /opt/nxlog/libexec/nxlog/modules SpoolDir /opt/nxlog/var/spool/nxlog define CERTDIR /opt/nxlog/var/lib/nxlog/cert define CONFDIR /opt/nxlog/var/lib/nxlog Note that these two lines define constants only; the log file location is ultimately set by the LogFile directive (see below). The MYLOGFILE define is also used to rotate the log file automatically (see the _fileop block). define LOGDIR /opt/nxlog/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log <Extension _json> Module xm_json </Extension> <Input file> Module im_file File '/<path/to/my/logfile/*log' # Parse log here if needed # $EventTime should be set here </Input> <Output out> Module om_elasticsearch URL http://linuxServer:9200/_bulk FlushInterval 2 FlushLimit 100 # Create an index daily Index strftime($EventTime, &quot;nxlog-%Y%m%d&quot;) # Use the following if you do not have $EventTime set #Index strftime($EventReceivedTime, &quot;nxlog-%Y%m%d&quot;) </Output> By default, LogFile %MYLOGFILE% is set in log4ensics.conf. This allows the log file location to be modified via NXLog Manager. If you are not using NXLog Manager, you can instead set LogFile below and disable the include line. LogFile %MYLOGFILE% #include %CONFDIR%/log4ensics.conf <Extension _syslog> Module xm_syslog </Extension> This block rotates %MYLOGFILE% on a schedule. Note that if LogFile is changed in log4ensics.conf via NXLog Manager, rotation of the new file should also be configured there. <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour &lt;Exec&gt; if ( file_exists('%MYLOGFILE%') and (file_size('%MYLOGFILE%') &gt;= 5M) ) { file_cycle('%MYLOGFILE%', 8); } &lt;/Exec&gt; &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8); &lt;/Schedule&gt; </Extension> NXlog is starting, but it waiting to connect to ElasticSearch server, not proceeding further. Here is the log in nxlog.log. And I could not see nxlog* index patter in Kibana 2020-03-31 09:37:44 WARNING already running as gid 0 2020-03-31 09:37:44 WARNING already running as uid 0 2020-03-31 09:37:44 WARNING no routes defined! 2020-03-31 09:37:44 WARNING default route added 2020-03-31 09:37:44 INFO nxlog-4.7.4715 started 2020-03-31 09:37:44 INFO om_elasticsearch_add_reconnect_event reconnecting in 0 seconds 2020-03-31 09:37:44 INFO connecting to linuxServer:9200 Any help on this is appricaited. Thanks in advance Santosh

Hi Santosh,

It looks like you are missing a route in your config.

Please check this section for an example: https://nxlog.co/docs/nxlog-ce/nxlog-reference-manual.html#route-directives

-MišaŽ