NXLog 4.3.4308 failed to open files

Post a different question
1
answer

Hi everybody!

Today I found in the logs of NXLog 4.3.4308 Server very odd error:

2019-06-12 11:22:04 ERROR failed to open file <FILE> when trying to truncate: Too many open files

The service was not working at this time until I has restarted it.

Could you be please so kind tell me what is the limit of opened files? What's the number simultaneous connections can the service hold?

Thanks!

AskedJune 12, 2019 - 10:46am

Answer (1)

This will depend on your Operating System and distribution. i.e. this is handled differently in Linux between SysV Init and SystemD processes.

At the base level though, this is normally a configuration on your system to limit how many files a user can have open and as such is system configuration outside of NXLog.

AnsweredJune 12, 2019 - 5:55pm

Comments (2)

  • hatula's picture

    Thank you, Zhengshi!

    I type lsof | grep <LOG FILES PATH> and I see only about 100 opened files. I think it is not maximum.

    June 13, 2019 - 8:12am
  • Zhengshi's picture
    (NXLog)

    Hatula,

    The process will have more files open than just the files you are pulling events from and writing to.
    Sockets, networking ports, pipes, shared libraries, etc can count towards the process' open file count.

    These commands assume bash and that your version of grep supports regex patterns. Insert the replacement portion with the process ID of your running NXLog instance.
    List open files based on PID
    lsof -p $(ps -aux | grep -e '[n]xlog'|awk '{print $2}')
    Count of all open files based on PID
    lsof -p $(ps -aux | grep -e '[n]xlog'|awk '{print $2}')| wc -l
    Limits for the same process
    cat /proc/$(ps -aux | grep -e '[n]xlog'|awk '{print $2}')/limits

    If you are running systemd, you will find that it will casually ignore system level file limits for services that it launches. In this case you can perform the following:
    Create the file and directory /etc/systemd/system/nxlog.d/override.conf. Add to this file the following lines. Note that the value can be changed to whatever you need in your environment. The typical default soft limit is 1024 and hard limit is 4096.

    [Service]
LimitNOFILE=32000

    At this point you need to systemctl daemon-reload to enable these changes. Restarting the nxlog service as well.

    June 13, 2019 - 4:36pm