Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
NXLog does not forward logs
ashutosh created
Hi Team,
Recently I started testing NXLog and was tryingto simulate log forwarding to other syslog servers. My logs are stored in *.log files and I want to forward them to another syslog destination. But after so many attempts, I still fail, and my logs are not forwarded. I also tried writing to another file using om_file but that does not help me as well. The NXLog's logs are of not much help, as It is stuck with just "Connecting to X.X.X.X:514 and never does anything ahead of it. IT does not show any warning / error as well.
How do I investigate, what went wrong.
I am on Ubuntu 16.04 with NXLog CE 2.10.2150 downloaded from this portal.
Below is my configuration,
<Input infile1>
Module im_file
File "/opt/logs/pix.log"
InputType LineBased
</Input>
<Output outfile1>
Module om_file
CreateDir TRUE
File "/opt/logs/output.log"
</Output>
<Output outtcp1>
Module om_tcp
Host X.X.X.X
Port 514
</Output>
<Route r1>
path infile1 => outtcp1, outfile1
</Route>
I have checked on the network side, did Telnet (for TCP) and NC (for UDP) everything works fine, even rsyslog is able to forward data but NXLog fails.
ashutosh created
Issues With "Multi-line message parser (xm_multiline)"
gmelasecca created
My company is looking to setup NxLog. We are having issues reading in multiline exception logs from applications such as Tomcat, Java, Apache etc. I am able to read in the files but unfortunately the output in our GrayLog application is showing every event as one line. I tried to implement the xm_multiline module but i seem to be having issues getting it to work.
installed NxLog and checked my configuration to the following below. restarted the services, let the service run all night and still the output is the same as shown below.
Sample Input Log:
07/07/2016 13:35:11.654 [tomcat-http--43] [ERROR] [4114723 ms] Warning - unprocessed rows in esolutions.care.assess.WeAssessment
esolutions.EsolutionsException: There were 83 unprocessed rows out of 84
at esolutions.base.WeObject.sleep(WeObject.java:2767)
at esolutions.base.WeObject.clear(WeObject.java:3250)
at esolutions.care.assess.WeAssessment.clear(WeAssessment.java:7699)
at esolutions.base.WeObject.close(WeObject.java:2815)
at esolutions.util.WeHTMLTable.getTableHTML(WeHTMLTable.java:541)
at esolutions.util.WeHTMLTable.toHTML(WeHTMLTable.java:508)
at org.apache.jsp.admin.client.cp_005fassessment_jsp._jspService(cp_005fassessment_jsp.java:4412)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
07/07/2016 13:36:21.828 [tomcat-http--26] [ERROR] [4184897 ms] Warning - unprocessed rows in esolutions.care.assess.WeAssessment
esolutions.EsolutionsException: There were 82 unprocessed rows out of 83
at esolutions.base.WeObject.sleep(WeObject.java:2767)
at esolutions.base.WeObject.clear(WeObject.java:3250)
at esolutions.care.assess.WeAssessment.clear(WeAssessment.java:7699)
at esolutions.base.WeObject.close(WeObject.java:2815)
at esolutions.util.WeHTMLTable.getTableHTML(WeHTMLTable.java:541)
at esolutions.util.WeHTMLTable.toHTML(WeHTMLTable.java:508)
at org.apache.jsp.admin.client.cp_005fassessment_jsp._jspService(cp_005fassessment_jsp.java:4412)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
Sample Output From GrayLog in CSV format. The output in the webui is each event as it shows in the "message" column.
timestamp
source
EventReceivedTime
level
message
SourceModuleName
SourceModuleType
2016-07-19T21:27:08.000Z
GDPCCA02
07/19/16 17:27
6
2016/07/19 17:27:08.032 | srvmain | INFO | 07/19/2016 17:27:08
pcc-wrapper-log
im_file
2016-07-19T21:27:08.000Z
GDPCCA02
07/19/16 17:27
6
2016/07/19 17:27:08.032 | srvmain | INFO | java.lang.NumberFor
pcc-wrapper-log
im_file
2016-07-19T21:27:08.000Z
GDPCCA02
07/19/16 17:27
6
2016/07/19 17:27:08.032 | srvmain | INFO | at com.pointclickc
pcc-wrapper-log
im_file
2016-07-19T21:27:08.000Z
GDPCCA02
07/19/16 17:27
6
2016/07/19 17:27:08.032 | srvmain | INFO | at org.apache.cata
pcc-wrapper-log
im_file
Configuration File. I tried multiple regular expressions with no success.
## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension gelf>
Module xm_gelf
</Extension>
<Extension fileop>
Module xm_fileop
</Extension>
<Extension multiline>
Module xm_multiline
HeaderLine /^\d{0,2}\/\d{0,2}\/\d{0,4}/
# HeaderLine '^\d{0,2}\/\d{0,2}\/\d{0,4}\ \d{0,3}\:\d{0,3}\:\d{0,3}\.\d{0,4}\ \['
</Extension>
<Input pcc-wrapper-log>
Module im_file
File "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\wrapper.log"
SavePos TRUE
InputType multiline
</Input>
<Input pcc-mdstrace-log>
Module im_file
File "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\mdstrace.log"
SavePos TRUE
InputType multiline
</Input>
<Input pcc-exceptionHidingUtil-log>
Module im_file
File "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\exceptionHidingUtil.log"
SavePos TRUE
InputType multiline
</Input>
<Input pcc-esolutions-log>
Module im_file
File "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\esolutions.log"
SavePos TRUE
InputType multiline
</Input>
#<Input pcc-localHostAccess-log>
# Module im_file
# File "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\localhost_access_log.*"
# SavePos TRUE
# InputType multiline
#</Input>
<Output graylog>
Module om_udp
Host graylog.genesishcc.com
Port 12201
OutputType GELF
</Output>
<Route PCC>
Path pcc-wrapper-log => pcc-mdstrace-log => pcc-exceptionHidingUtil-log => pcc-esolutions-log => graylog
## Path pcc-wrapper-log => pcc-mdstrace-log => pcc-exceptionHidingUtil-log => pcc-esolutions-log => pcc-localHostAccess-log => graylog
</Route>
gmelasecca created