Hi Team,
Do anyone know what nxlog file should be installed on Amazon Linux AMI?
I was going through the below forum and it's pretty confusing: https://forums.aws.amazon.com/thread.jspa?threadID=51647
Any help would be appreciated.
andy_cognizant created
On one server:
C:\Windows\System32\dns\dns.txt - Error
C:\dns.txt - No error
C:\Windows\Temp\dns.txt - No error
On a different server:
It produced error regardless which directory the log file is in.
====
nxlog.conf snip below:
define DNSLOGFILE 'c:\Windows\System32\dns\dns.txt'
<Input dnslog>
Module im_file
File %DNSLOGFILE%
InputType LineBased
SavePos TRUE
ReadFromLast TRUE
...
</Input>
====
Note, I have already turned on Loglevel DEBUG, which shows the file was blacklist'ed for retry, but only after it failed to read the file. I have also already read this thread (https://nxlog.co/question/920/input-file-does-not-exist) which was close but did not resolve my issue.
Thanks in advance.
tlam.nt created
This is less a question and more of an observation.
I am currently running nxlog 4.1.4016 on Ubuntu 18.04.1 LTS in a vmware environment. Say I boot the VM up and the nxlog service kicks off correctly and works as intended ultimately writing to a network share that I have mounted. If I do a "sudo systemctl restart nxlog.service" or even "./nxlog -r" in order to reload nxlog with a slightly modified config file, UDP packet receive errors and UDP receive buffer errors start climbing from 0 like crazy (netstat -suna). A reboot of the VM from this state does not even fix the issue, the errors immediately appear. In order to fix the issue, I had to purge the nxlog install and do a reinstall in order to prevent any further issues. My config consists of listening for UDP on 2 ports, going through a memory buffer, and writing to the mounted share.
manoj.muthukumaran created
Hi everybody, To protect the logs from loss, I turned on the recommended settings in client's config:
PersistLogqueue TRUE SyncLogqueue TRUE CacheFlushInterval always CacheSync TRUE
But now I see that parts of log are duplicated much more often than before these setting was enabled. Why cache in memory works better? In case with cache on disk I hope to see best result.
Thank you so much!
hatula created
Hello all,
I am having an issue that I believe is specific to the Windows Server 2003 R2 systems I am trying to install Nxlog CE on. I am getting the following error from the nxlog.log file.
ERROR Failed to load module from C:\Program Files\nxlog\modules\input\im_msvistalog.dll, The specified module could not be found. ; The specified module could not be found
Because its a 32bit OS we installed nxlog in C:\Program Files. I checked and the file im_msvistalog.dll is indeed where it is supposed to be. This error has occurred on most but not all 32bit Windows Server 2003 R2. Any help you can give me would be greatly appreciated.
Thanks
S
Shelock created
Hello,
I have an application that logs some API requests and responses. Each request is logged in a different file, as a single line. In the system there are thousands of files, and Nxlog seems to have issues sending the logs to Elasticsearch. It reads the file, I can see im_file_add_file command in logs, but it takes a long time to actually send the message.
How does Nxlog process multiple files in a single directory?
alexandru.enciu created
Hello,
Do you known if it's possible to send DHCP and DC log to a SYSLOG-NG server
I don't know how specify the facilities for the windows logs
Thanks in advance.
Ezein created
Hello,
I am running a trial version of EE, but when I try to start NxLog, I get errors saying it cannot find the modules.
Here is my conf file. I have verified that nxlog is installed at C:\Program Files\nxlog.
configuration options. It should be installed locally and is also available
online at http://nxlog.org/docs/
Please set the ROOT to the folder your nxlog was installed into,
otherwise it will not start.
define ROOT C:\Program Files\nxlog #define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log #LogLevel DEBUG
<Extension json> Module xm_json </Extension>
<Input in>
For Windows 2008 and later
Module im_msvistalog
For Windows 2003 and earlier
#Module im_mseventlog
File "c:\\documents and settings\\administrator\\desktop\\events\\app.evtx"
Exec to_json();
</Input>
<Output out>
Module om_tcp
Host localhost
Port 5013
</Output>
<Route 1> Path in => out </Route>
Error logs
2018-10-12 13:51:24 ERROR Failed to load module from C:\Program Files\nxlog\modules\input\im_msvistalog.dll, The specified module could not be found. ; The specified module could not be found.
2018-10-12 13:51:24 WARNING no functional input modules!
2018-10-12 13:51:24 ERROR module 'in' is not declared at C:\Program Files\nxlog\conf\nxlog.conf:42
2018-10-12 13:51:24 ERROR route 1 is not functional without input modules, ignored at C:\Program Files\nxlog\conf\nxlog.conf:42
2018-10-12 13:51:24 INFO nxlog-4.1.4046-trial started
2018-10-12 13:51:24 WARNING not starting unused module out
Deleted user created
Hello,
i am setting up SSL connection between rsyslog over linux box and nxlog endpoint. While win boxes connect like a charm linux boxes issue the following:
2018-10-12 11:51:26 ERROR remote ssl socket was reset? (SSL_ERROR_SSL with errno=9); End of file found
I then found on your forum this post https://nxlog.co/question/1926/nxlog-ce-v291716-certificate-built-ecdsa-key where they talk about rebuild certificate without Digital Signature KeyUsage flag.
I assumed to rebuild client.crs since my rootCA.crt does not report any Digital Signature :
X509v3 extensions:
X509v3 Subject Key Identifier:
AB:E6:E4:61:11:89:43:21:87:FB:91:08:44:C0:15:A7:41:3B:A3:53
X509v3 Authority Key Identifier:
keyid:AB:E6:E4:61:11:89:43:21:87:FB:91:08:44:C0:15:A7:41:3B:A3:53
DirName:/C=US/ST=Some-State/L=Somecity/O=CompanyName/OU=Organizational Unit Name (eg, section)/CN=Common Name (e.g. server FQDN or YOUR name)/emailAddress=Email Address
serial:AF:06:5F:4B:97:ED:81:90
X509v3 Basic Constraints:
CA:TRUE
X509v3 Key Usage:
Certificate Sign, CRL Sign
I built a new client.csr without any trace of X509v3 extensions, but i always get the same error message.
Any help is well appreciated. Thanks
comoalt created
Hi guys! I really someone can help because I think I have tested all the things I could think of to make it work...
Ok, so we have those logs:
'[2018-10-11T12:06:47,434][DEBUG][o.e.a.s.TransportSearchAction] [master01] [245674] Failed to execute fetch phase org.elasticsearch.transport.RemoteTransportException: [hot08][10.10.30.168:9300][indices:data/read/search[phase/fetch/id]] Caused by: org.elasticsearch.search.SearchContextMissingException: No search context found for id [245674] at org.elasticsearch.search.SearchService.findContext(SearchService.java:520) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:487) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.action.search.SearchTransportService$11.messageReceived(SearchTransportService.java:440) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.action.search.SearchTransportService$11.messageReceived(SearchTransportService.java:437) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:258) ~[?:?] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:135) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.lambda$messageReceived$0(SecurityServerTransportInterceptor.java:307) ~[?:?] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$2(ServerTransportFilter.java:166) ~[?:?] at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.maybeRun(AuthorizationUtils.java:183) ~[?:?] at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.setRunAsRoles(AuthorizationUtils.java:177) ~[?:?] at org.elasticsearch.xpack.security.authz.AuthorizationUtils$AsyncAuthorizer.authorize(AuthorizationUtils.java:165) ~[?:?] at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.lambda$inbound$3(ServerTransportFilter.java:168) ~[?:?] at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:60) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$2(AuthenticationService.java:184) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$4(AuthenticationService.java:217) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:228) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:182) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:143) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:113) ~[?:?] at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:142) ~[?:?] at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:314) ~[?:?] at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1555) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.2.4.jar:6.2.4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]'
I really need only the first 3 lines, and is possible, make only one at the end.
I have tried the following config:
<Extension multi> Module xm_multiline HeaderLine /^[\d{0,4}-\d{0,2}-\d{0,2}\D\d\d{0,2}:\d{0,2}:\d{0,2}\D\d{0,4}]*/ EndLine /^.+(at)/ </Extension>
<Input elastic-log> InputType multi Module im_file File "/var/log/elasticsearch/mega.log.test" </Input>
<Output file> Module om_file File '/tmp/output' </Output>
The the output file keep giving me all the lines instead of the first 3 that I expect...
I have tested my regular expressions and I know they are working so.. why I can't have my first 3 lines!!!!???? :)
Any help will be very appreciated. R.
romainp created
I'm attempting to use NXLog (community edition atm) to read in active directory logs into NXLog and output to syslog/json. I have a text file (one username per line) that I need to be able to compare to the username in the Windows event logs from AD. I need to be able to drop messages that the username in the Windows AD Event logs if it matches a username in the text file of usernames.
I've spent quite a bit of time googling and reading documentation and haven't found a method to achieve this. Can anyone assit?
habrosec created
comoalt created
hi everyone
I've been searching this forum and the web, but can't find if there is a way to make nxlog include its version in the syslogs it forwards out
Thanks!
aleksandrc created
Hi everybody,
I have a very mystical case and I need your help, please:
- I have nxlog server nxlog-3.2.2016-1.x86_64
- I installed a update to latest version of nxlog server (nxlog-4.1.4016-1.x86_64)
- Config for clients was not change.
- In the logfile of nxlog server I see many errors and nxlog do not save data:
2018-09-25 10:50:32 ERROR zlib decompression error, data error (Z_DATA_ERROR) incorrect header check
- If I did downgrade version, this problem is go, everything is ok.
What's happened?! Please, help me.
hatula created
navdeepsingh83 created
Hi all,
I've set up nxlog (4.1.4016) to monitor a logfile that is been written to constantly. For some reason, when i start nxlog, the programm that creates the loglines no longer adds info to the existing log. A simple commandline 'echo logline >> thelog.txt' does add the line to the logfile (and processed by nxlog), but other logging is not added. As soon as I stop the NXLog service, the log is modified again.
NXLog is running as system, the programm is running as a normal user.
Any suggestions for troubleshooting would be welcome, as I have no clue what is happening.
JanVerhaag created
Hi all,
I am new here, so hello.
I am trying to work out a solution to collect IIS Access Log data from Azure App Services and then forward to a SIEM such as Splunk, Loggly or ElasticSearch for Security analysis, Anomoly identification and alerting.
As far I can see NXLog may provide the link between the Azure Access Logs and my chosen SIEM. Am I right? I would prefer to not get into rewriting of code, hence my interest in NXLog. In addition it appears that NXL:og seems to be widely supported by SIEM tools.
With regards to implementation, would I be correct in thinking that one needs to setup a Linux or WIndows VM on Azure with NXLog running on it. I am trying to avoid onprem installs.
In conclusion, I would welcome your advice on how I could use NXLog as a simple collector and forwarder of Access data. One final word, the Access Logs are currently stored in Azure Storage Blobs, although I could go back to storing them in the File System.
Thanks.
EdB created
hi,
Is somebody has got an experience of im_dbi ?
I tried this example but /tmp/output is filled of blank char ? I checked nxlog log at starting, everything is OK. Driver mysql has been installed correcly
<Input dbi>
Module im_dbi
Driver mysql
Option host 127.0.0.1
Option username mysql
Option password mysql
Option dbname logdb
SQL SELECT id, facility, severity, hostname,
timestamp, application, message
FROM log
</Input>
<Output file> Module om_file File "tmp/output" </Output>
<Route dbi_to_file> Path dbi => file </Route>
iCirco created
DDGH created
Hello! I'm trying to deploy nxlog with GPO on windows, but sims like MSI package from https://nxlog.co/products/nxlog-community-edition/download not working properly. After creating GPO nothing happens, I have tried install as well with scrip (cmd /c Msiexec /I \file server\share\nxlog-ce-2.9.1716.msi /qn) nothing.
When I run the script on local PC getting the error "This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer package."
We using AD 2012 and windows 10/8 machines. Is there any way to deploy nxlog massive on all PC's?
Thx
Maksimsk created