7
responses

Hi,

Here's my case :

I'm trying to create Syslog Server Configuration, based on CentOS. Server must :
-- Receive log from tcp
-- store log into a NFS Volume automaticaly mounted by fstab on server startup
---- in case of fail of the NFS Volume, create a buffer on local disk.

Here's my configuration :

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension _xm_file>
Module xm_fileop
</Extension>

<Input in>
Module im_tcp
Host 0.0.0.0
Port 39458
<Exec>
$SERVER = hostname();
parse_syslog_ietf();
$DATE = strftime($EventTime, "%Y%m%d");
</Exec>
</Input>

<Processor buffer>
Module pm_buffer
maxSize 204800
Type Disk
Directory %LOGDIR%
WarnLimit 190000
</Processor>

<Processor blocker>
Module pm_blocker
<Exec>
$CHEMIN = "/var/partage/"+$SERVER;
$test = dir_exists($CHEMIN);
if $test == TRUE
{
blocker->block(FALSE);
}
else blocker->block(TRUE);
</Exec>
</Processor>

<Output out>
Module om_file
File "/var/partage/"+$SERVER+"/"+$Hostname+"/"+$DATE+"_"+$Hostname+".log"
CreateDir TRUE
Exec to_syslog_ietf();
</Output>

<Route tcp_to_file>
Path in => buffer => blocker => out
</Route>

My Problem :
- When NFS Volume is down, pm_blocker write " can't check condition "dir_exists(<Path>)" : input/outpur Error"
- When i try without pm_blocker, om_file fails with same error and no buffer created....

Question :
- What is the best solution to implement a disk buffer system for om_file module ?
- How can i catch NFS volume exceptions ?

Thanks in advance,
best regards,
Julien

AskedJuly 16, 2020 - 6:49pm

Comments (7)

  • julienBourdon's picture

    Hi Manuel,

    Thank you for your return.

    Here's NFS mount in fstab :
    //192.168.1.53/partage /var/partage cifs defaults,uid=998,username=jbourdon,guest 0 0

    CIFS because i'm in my lab. it will be an NFS share on production environment.
    uid correspond id of nxlog account

    It works because i see logs in CIFS share, but when i force a fail over (i stop file server where is provided the CIFS Share) , disk buffer don't start.

    For example :
    2020-07-17 13:57:12 INFO [CORE|main] nxlog-5.0.5874-trial started
    2020-07-17 13:57:12 INFO [im_tcp|in] listening on '0.0.0.0'(0.0.0.0):39458
    2020-07-17 13:57:30 INFO [im_tcp|in] connection accepted from 192.168.88.129:50614
    ------ stop CIFS Share
    2020-07-17 14:00:23 ERROR [pm_blocker|blocker] failed to check whether directory '/var/partage/nxlogserver1' exists: Input/output error

    Thank you in advance.

  • manuel.munoz's picture
    (NXLog)

    Can you do a ls -lah in the mounting point? Anything interesting in the OS log? Anything else in nxlog.log apart from that line?

  • julienBourdon's picture

    - ls -lah -> cannot access /var/partage : input/output Error
    - Anything else in nxlog.log
    - Anything interesting in the OS log.

    Habitually, how do you do to start a fail over procedure with nxlog in case of mount failure ?

  • julienBourdon's picture

    i've tried to use buffer behind tcp module on CentOS Server, and i've similar problems

    Here's configuration file :

    define SYSLOGSERVER1 192.168.88.128:39458
    define SYSLOGSERVER2 192.168.88.130:39458

    <Extension _syslog>
    Module xm_syslog
    </Extension>

    <Input in>
    Module im_uds
    UDS /opt/nxlog/var/spool/nxlog/rsyslog_sock
    Exec parse_syslog();
    </Input>

    <Processor buffer>
    Module pm_buffer
    MaxSize 204800
    Type Disk
    Directory %LOGDIR%
    WarnLimit 100000
    CreateDir TRUE
    </Processor>

    <Output out>
    Module om_tcp
    Reconnect 5
    Host %SYSLOGSERVER1%
    Host %SYSLOGSERVER2%
    Exec to_syslog_ietf();
    </Output>

    <Route r>
    Path in => buffer => out
    </Route>

    When i'm shutting down syslog server, i can see error in nxlog.log... But no buffer created.
    Log :
    [After restarting nxlog services, because it don't see lost of tcp connection.. Strange too]
    ERROR [om_tcp|out]couldn't connect to "192.168.88.128":39458; no route to host
    INFO [om_tcp| out] reconnecting in 5 sec
    ERROR[...]
    > ls -lah %LOGDIR% -> No buffer
    > After syslog server coming back -> Log lost.

    Maybe i don't understand how pm_buffer works... I'm lost.
    if exist other way to make a buffer system.. i'm listenning ! :-)

    Thank you in advance,
    best regards

  • manuel.munoz's picture
    (NXLog)

    Julien,

    I assume if you are getting this error when doing a ls from CLI, there's something wrong with the mounted share.

    - ls -lah -> cannot access /var/partage : input/output Error
    

  • julienBourdon's picture

    Manuel,

    Input/Output Error appears because i've shutdown FileServer.
    My question isn't why this error appears, but how blocker module could catch this error and enable buffer module.

    I don't know if it's clear enough but i need to build a backup solutions in case of share folder's failure.

Answers (0)