NXLog does not forward logs

Post a different question
3
responses

Hi Team,

Recently I started testing NXLog and was tryingto simulate log forwarding to other syslog servers. My logs are stored in *.log files and I want to forward them to another syslog destination. But after so many attempts, I still fail, and my logs are not forwarded. I also tried writing to another file using om_file but that does not help me as well. The NXLog's logs are of not much help, as It is stuck with just "Connecting to X.X.X.X:514 and never does anything ahead of it. IT does not show any warning / error as well.
How do I investigate, what went wrong.

I am on Ubuntu 16.04 with NXLog CE 2.10.2150 downloaded from this portal.

Below is my configuration,

<Input infile1>
Module im_file
File "/opt/logs/pix.log"
InputType LineBased
</Input>

<Output outfile1>
Module om_file
CreateDir TRUE
File "/opt/logs/output.log"
</Output>

<Output outtcp1>
Module om_tcp
Host X.X.X.X
Port 514
</Output>

<Route r1>
path infile1 => outtcp1, outfile1
</Route>

I have checked on the network side, did Telnet (for TCP) and NC (for UDP) everything works fine, even rsyslog is able to forward data but NXLog fails.

AskedSeptember 13, 2019 - 2:19am

Comments (2)

  • Zhengshi's picture
    (NXLog)

    I would take the outtcp1 out of your Route to troubleshoot. Make sure you are reading the files to begin with.

    Another thing I would do is stop the service and run the binary in the foreground nxlog -f. This will print errors to stdout if there are any, and you can take advantage of the next tip easier this way as well...

    Use log_info() to print out values during execution. Exec log_info($raw_event); inside of your infile1 instance could show you events as they come in.

    One important thing to note is NXLog will only read events that have come in after it has started by default. The solution is to create an event, echo data to the file in append mode (>>) or to tell NXLog to read from the beginning and not save the position, which is really useful for testing.

    ReadFromLast False
SavePos False

    Once you see that you are getting events locally then you can make sure it is written to your Output file.

    The next step is to add your outtcp1 back to the Route and verify it is sending. Check the remote side, run tcpdump against the server NXLog is installed on.

    This should give you a solid troubleshooting path to work from.

    September 13, 2019 - 8:26pm
  • ashutosh's picture

    Thanks a lot for taking out time and answering my question in details.

    Before I received your answer, I have figured out way to get solve the issue.

    The outfile1 was added there only to check whether actually outtcp1 was working or not, but outfile1 did not work as well. I enabled the DEBUG logs and run the nxlog -f to check if anything fails, unlucky me, it was not throwing any error either. The Debug logs mentioned starting of imfile omfile omtcp but after the start, there were only imfile logs been thrown and no operation was performed on omfile or omtcp. I tried creating new log files just to over come the case where NXLog will only read the logs which are new. Nothing worked.

    Instead of monitoring a file, I created a intcp1 module to monitor traffic on a TCP port with the same outfile1 and outtcp1. And it worked. As soon as I Telnet a data, it was reflecting in my syslog server as well as in the output file.

    Post this, I found an Extension for File Manipulation xm_fileop I added it before rest of the module, and Everything worked perfect.

    The solution was to add

    <Extension fileop>   Module xm_fileop </Extension>

    at the beginning.

    Thanks a lot Zhengshi, It is a helpful information you provided.

    September 13, 2019 - 9:36pm

Answer (1)