Buffer is not working

View thread

Konstantin.Grudnev

Hello,



Here is my config on Windows machine, running nxlog-ce-2.9.1504




#define ROOT C:\Program Files\nxlog

define ROOT C:\Program Files (x86)\nxlog



Moduledir %ROOT%\modules

CacheDir %ROOT%\data

Pidfile %ROOT%\data\nxlog.pid

SpoolDir %ROOT%\data

LogFile %ROOT%\data\nxlog.log



###############

# Extensions  #

###############

<Extension syslog>

    Module    xm_syslog

</Extension>



<Extension json>

    Module    xm_json

</Extension>





###########

# Inputs  #

###########



<Input some_input>

    Module    im_file

    File    'C:\Logs\input.log'

    SavePos    TRUE

</Input>



###############

# Processors  #

###############



<Processor buffer>

    Module      pm_buffer

    # 1Gb disk buffer 1048576 kilo-bytes

    MaxSize    1048576

    Type    Disk

    Directory  C:\Logs\buffer    

</Processor>





############

# Outputs  #

############



<Output tcpout>

    Module    om_tcp

    Port    5170

    Host    fluentd.company.lan

</Output>



############

# Routes   #

############



<Route file>

    Path   some_input => buffer => tcpout

</Route>




Here's testing case initials:



1. Service ' fluentd.company.lan' is up and running and listens on 5170

2. nxlog up and running with given config

3. Data coming to input.log is successfully routed to output via buffer and is seen in Kibana


Then

1. I change 'C:\Windows\system32\drivers\etc\hosts' file and add '127.0.0.1 fluentd.company.lan' line, saving file

2. Using TCPView tool from SysInternals close current TCP connection with 'fluentd.company.lan:5170'

3. See in nxlog.log, that it tries to connect to 'fluentd.company.lan:5170' and fails to connect

4. Wait for some new data in input.log

5. New data arrived and I see buffer file created 'buffer.1.q' in C:\Logs\buffer and see relevant data in it

6. Wait for some time (2-3 minutes)

7. Again I change 'C:\Windows\system32\drivers\etc\hosts' file and comment '127.0.0.1 fluentd.company.lan' line, saving file

8. nxlog successfully connects to fluentd.company.lan:5170


And here's interesting part, nxlog writes new data found in input file, but I don't see logs in Kibana from buffer file with timestamps from intervals in point #6



Please check this case and make sure buffer is not working on Windows and fix this bug