responses
Hi Team,
Recently I started testing NXLog and was tryingto simulate log forwarding to other syslog servers. My logs are stored in *.log files and I want to forward them to another syslog destination. But after so many attempts, I still fail, and my logs are not forwarded. I also tried writing to another file using om_file but that does not help me as well. The NXLog's logs are of not much help, as It is stuck with just "Connecting to X.X.X.X:514 and never does anything ahead of it. IT does not show any warning / error as well.
How do I investigate, what went wrong.
I am on Ubuntu 16.04 with NXLog CE 2.10.2150 downloaded from this portal.
Below is my configuration,
<Input infile1>
Module im_file
File "/opt/logs/pix.log"
InputType LineBased
</Input>
<Output outfile1>
Module om_file
CreateDir TRUE
File "/opt/logs/output.log"
</Output>
<Output outtcp1>
Module om_tcp
Host X.X.X.X
Port 514
</Output>
<Route r1>
path infile1 => outtcp1, outfile1
</Route>
I have checked on the network side, did Telnet (for TCP) and NC (for UDP) everything works fine, even rsyslog is able to forward data but NXLog fails.
Comments (2)
I would take the
outtcp1
out of your Route to troubleshoot. Make sure you are reading the files to begin with.Another thing I would do is stop the service and run the binary in the foreground
nxlog -f
. This will print errors to stdout if there are any, and you can take advantage of the next tip easier this way as well...Use
log_info()
to print out values during execution.Exec log_info($raw_event);
inside of yourinfile1
instance could show you events as they come in.One important thing to note is NXLog will only read events that have come in after it has started by default. The solution is to create an event, echo data to the file in append mode (
>>
) or to tell NXLog to read from the beginning and not save the position, which is really useful for testing.Once you see that you are getting events locally then you can make sure it is written to your Output file.
The next step is to add your
outtcp1
back to the Route and verify it is sending. Check the remote side, run tcpdump against the server NXLog is installed on.This should give you a solid troubleshooting path to work from.
Thanks a lot for taking out time and answering my question in details.
Before I received your answer, I have figured out way to get solve the issue.
The
outfile1
was added there only to check whether actuallyouttcp1
was working or not, butoutfile1
did not work as well. I enabled the DEBUG logs and run thenxlog -f
to check if anything fails, unlucky me, it was not throwing any error either. The Debug logs mentioned starting ofimfile
omfile
omtcp
but after the start, there were onlyimfile
logs been thrown and no operation was performed onomfile
oromtcp
. I tried creating new log files just to over come the case where NXLog will only read the logs which are new. Nothing worked.Instead of monitoring a file, I created a
intcp1
module to monitor traffic on a TCP port with the sameoutfile1
andouttcp1
. And it worked. As soon as I Telnet a data, it was reflecting in my syslog server as well as in the output file.Post this, I found an Extension for File Manipulation
xm_fileop
I added it before rest of the module, and Everything worked perfect.The solution was to add
<Extension fileop> Module xm_fileop </Extension>
at the beginning.
Thanks a lot Zhengshi, It is a helpful information you provided.