Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
om_file Module problem
GavinChen created
Sorry,my english is not good.
I occur a problem with om_file module. my config as following:
#define Log receive from Network
<Input LogNetIn>
Module im_tcp
host 0.0.0.0
port 1514
<Exec>
if $raw_event =~ /^"([^"]+)"/ {
$ModuleName=$1 ;
} else if $raw_event =~ /ModuleName=([^\d]+)/{
$ModuleName=$1 ;
} else {
file_write('%ROOT%\data\debug.txt',$raw_event +"\r\n");
}
if ($ModuleName=="HB") {
HB_IL_CSV->parse_csv();
HB_CSV->to_csv();
$StorePathFileName='%ROOT%\data\heartbeat.log';
}else if ($ModuleName=="IL") {
HB_IL_CSV->parse_csv();
IL_CSV->to_csv();
$StorePathFileName='%ROOT%\data\nxlog_agents.log';
}else if ($ModuleName=="WinEventIn") {
WinEventNetIn_CSV->parse_csv();
WinEventFileOut_CSV->to_csv();
$StorePathFileName='%ServersLogStorePath%\'+$Department+'\'+$Section+'\'+ $SourceHostName + '\WinEventLog\'+ $Channel +'\' + strftime($EventTime,"%Y%m%d")+'_'+ $SourceHostName + '_' + $Channel+ '.csv';
}else if ($ModuleName=="WebLog" or $ModuleName=="TransLog" or $ModuleName=="OthersLog") {
LogKvp->parse_kvp();
$raw_event=$Message;
$StorePathFileName='%ServersLogStorePath%\'+$Department+'\'+$Section+'\'+ $Hostname + '\' + $ModuleName +'\' + $ParentDir + '\' + $FileName;
}else {
file_write('%ROOT%\data\debugModuleName.txt',$ModuleName +"\r\n");
$StorePathFileName='%ROOT%\data\UnknownLog.log';
}
create_stat("stat", "RATE", 1); add_stat("stat", 1);
</Exec>
<Schedule>
Every 1 sec
Exec log_info("EPS: " + get_stat("stat"));
</Schedule>
</Input>
#define Log Network Receive Buffer
<Processor LogNetInBuffer>
Module pm_buffer
# 4096MB buffer
MaxSize 4194304
Type Mem
# warn at 10MB
WarnLimit 10240
#<Schedule>
# Every 10 sec
# Exec log_info("BufferSize: " + buffer_size() + ",BufferCount: " + buffer_count());
#</Schedule>
</Processor>
#define Log Output File
<Output LogStoreFile>
Module om_file
CreateDir TRUE
File $StorePathFileName
</Output>
<Route LogRoute>
Path LogNetIn => LogNetInBuffer => LogStoreFile
</Route>
When I use ultraedit text editor to open the file "20160126_VMNXLOGCLT_Security.csv". I found nxlog.log of ServerSide has as following error.
2016-01-27 17:23:29 ERROR failed to open \\apstor1\Eventlog\Servers\WinEventLog\Security\20160126_VMNXLOGCLT_Security.csv; The process cannot access the file because it is being used by another process.
2016-01-27 17:23:30 INFO EPS: 129
2016-01-27 17:23:31 INFO EPS: 124
2016-01-27 17:23:31 INFO EPS: 124
2016-01-27 17:23:33 INFO EPS: 129
2016-01-27 17:23:33 INFO EPS: 129
2016-01-27 17:23:34 ERROR failed to open \\apstor1\Eventlog\Servers\WinEventLog\Security\20160126_VMNXLOGCLT_Security.csv; The process cannot access the file because it is being used by another process.
2016-01-27 17:23:35 ERROR last message repeated 2 times
2016-01-27 17:23:35 INFO EPS: 126
The Server Side nxlog process will not output any file and it continuous consume memory, I must restart nxlog services that recover work.
So, how do I solve this problem ?
Regards
GavinChen created
Selective logging of Windows Event Log fields when forwarding to SIEM - exclude information text from the end of the log message
tsigidibam created
Here is a sample event when using to_syslog_snare() in the nxlog.conf:
<14>Jan 27 10:03:39 event_computer MSWinEventLog 1 Security 32630749 Wed Jan 27 10:03:39 2016 4624 Microsoft-Windows-Security-Auditing N/A N/A Success Audit event_computer Logon An account was successfully logged on. Subject: Security ID: S-1-0-0 Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 Impersonation Level: Impersonation New Logon: Security ID: S-1-5-21-2705889813-1605608894-1661845433-43745 Account Name: account_name Account Domain: account_domain Logon ID: 0x23820B882 Logon GUID: {00000000-0000-0000-0000-000000000000} Process Information: Process ID: 0x0 Process Name: - Network Information: Workstation Name: workstation_name Source Network Address: source_address Source Port: 54241 Detailed Authentication Information: Logon Process: NtLmSsp Authentication Package: NTLM Transited Services: - Package Name (NTLM only): NTLM V2 Key Length: 0 This event is generated when a logon session is created. It is generated on the computer that was accessed. The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe. The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network). The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on. The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases. The impersonation level field indicates the extent to which a process in the logon session can impersonate. The authentication information fields provide detailed information about this specific logon request. - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event. - Transited services indicate which intermediate services have participated in this logon request. - Package name indicates which sub-protocol was used among the NTLM protocols. - Key length indicates the length of the generated session key. This will be 0 if no session key was requested. 35284558
My issue is that I would NOT want to collect the "informational text" representing the event - in this case everything starting from the string "This event is generated---" all the way up until "--was requested."
Before I go any deeper into this, let me state that in the logs of this format I call the "<14>Jan 27 10:03:39 event_computer MSWinEventLog 1 Security 32630749 Wed Jan 27 10:03:39 2016 4624 Microsoft-Windows-Security-Auditing N/A N/A Success Audit event_computer Logon" portion of the whole log message the HEADER, and the rest is called MESSAGE.
Putting it another way, I would like to forward the message using syslog in a format constructed according to the pseudocode below:
parse fields from windows event /* e.g. SubjectUserName, LogonType, IpAddress, etc. */
/* print the header "as is" already in the to_syslog_snare() format, i.e. from "<14>---" until and including "---Logon"
print HEADER /* e.g. event_time,event_computer,event_type,event_id,... */
for all fields parsed
print "'field_name=field_value'" /* e.g. SubjectUserName=value,LogonType=value,IpAddress=value,... /*
The reason I would like to do this is that the informational text, which gets appended to some Windows events (not all, it seems), takes a lot of space, and we do not really need this information text for anything.
Another way to do this would be to statically list all the fields POSSIBLY found in an Windows event and construct the message that way, but this would often leave me with a lot of empty key-value pairs. THUS I would only like to print out those fields that were found in that specific log message while leaving out the informational message.
I do acknowledge, though, that especially Application and System events might not contain most or any of the fields that are present in a Security log event. Take for example the following System log event:
<14>Jan 27 11:09:21 event_computer MSWinEventLog 1 System 32633951 Wed Jan 27 11:09:21 2016 7036 Service Control Manager N/A N/A Information event_computer N/A The Remote Registry service entered the stopped state. 319889
In the example above, the "header" portion of the whole message only contains the string "The Remote Registry service entered the stopped state." I do hope, though, that the variable where this string is stored is actually the same that hosted the string "An account was successfully logged on.", which would mean that my approach in the pseudocode would still work (i.e. the array or list of fields that is iterated and printed would only contain one field. The HEADER portion of the field is exactly the same in all messages.
The description of to_syslog_snare() in the nxlog documentation states:
"Create a SNARE Syslog formatted log message in $raw_event. Uses the following fields to construct $raw_event: $EventTime, $Hostname, $SeverityValue, $FileName, $EventID, $SourceName, $AccountName, $AccountType, $EventType, $Category, $Message."
Thus when reflecting back to what I said, it seems that what I call the HEADER includes all the fields from $EventTime to (and including) $Category - this I would like to keep as it is. But according to the documentation, the $Message variable actually then holds all the other information in the log, or what I call the MESSAGE portion. So I guess the question is that can the contents of the $Message variable be further filtered, as it obviously is constructed from e.g. EventData's Data fields listed below. I would like to only change the $Message contents so that it would never contain the informational text if there exists such a message in a given log message, and that preferably the Data fields inside $Message would be formatted using key-value pairs instead of the to_syslog_snare format seen in the first example (one or more whitespace as delimiter).
tsigidibam created
NxLog to QRadar with TLS
Corentin created
Hello Everybody,
We are currently using nxlog to send Windows logs to QRadar SIEM utsing TLS.
It works fine, but I receive extra lines in QRadar. I receive empty logs (containing Cg== on base 64, which seems to be a carriage return or a line break).
The problem appears only when using module om_ssl, not whith om_tcp or om_udp. We tried to remove line break or carriage return using nxlog configuration, but we still have the behaviour.
Has anybody seen the same problem ? How could I correct this behaviour ?
You'll find below our current configuration
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension syslog>
Module xm_syslog
</Extension>
<Input internal>
Module im_msvistalog
ReadFromLast TRUE
</Input>
<Output ssl_out>
Module om_ssl
CertFile C:\CERTDIR\syslog-tls.cert
CertKeyFile C:\CERTDIR\syslog-tls.key
AllowUntrusted TRUE
Host 10.0.0.1
Port 6514
Exec to_syslog_snare();
</Output>
<Route 1>
Path internal => ssl_out
</Route>
Thank you !
Corentin created
Snare Lite Open source v.s. NXlog CE and PCI 3.0 compliance
hilbertd21 created
We have been using Snare Lite Open source for years, and understand it no longer meets PCI 3.0 compliance as it does not log enough event information. Snare Enterprise logs significantly more information, and would be PCI 3.0 compliant, but is expensive.
I wondered if anyone here would know if the NXlog CE edition, which is free, logs MORE event info than the Snare Lite Open source version that we could use it to replace Snare Lite Open source, and be confident it would comply with minimum PCI 3.0 requirements.
I see the comparison of Snare lite open source with enterprise here to see the differences, but haven't found a similar chart for NXlog CE to compare.
https://www.intersectalliance.com/why-snare-enterprise/
hilbertd21 created
How to capture print logs stored in Windows Eventlog
suresh created
Hi,
I am suresh, i have installed nxlog agent on one of my Windows 2008 R2 print server and it transfer Application, Security & System Eventlogs to LogRhythm log collector.
My question is, will it also transfer the print logs (stored in Event Viewer -> Applications and Service Logs -> Microsoft -> Windows -> Print Service -> Operational) to my LogRhythm log collector?
or not?
suresh created
KISS: beginner's problems with im_file and om_file
djontra created
Hello nxlog world,
Shamed to say, I've spent entire yesterday trying to figure out how to read Windows DHCP log files and ship the events to ElasticSearch.
Problem was with using direct path for folder C:\Windows\System32\dhcp\. Managed to get nxlog to read by sharing the folders (read-only permissions) to the user account used for nxlog service account logon.
As the events were not showing in ES, I'm stuck with trying to write the events into another file, in order to confirm that the source files are being read correctly.
OS: Win Srv 2008 R2 Ent
nxlog: v 2.9.1347
Here is the nxlog.conf:
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
# debugging only:
LogLevel DEBUG
NoCache TRUE
<Input msdhcp>
Module im_file
File '\\DC5\dhcp\DhcpSrvLog-*.log'
SavePos TRUE
InputType LineBased
Exec if $raw_event =~ /^#/ drop();
Exec $message = $raw_event;
</Input>
<Input dns>
Module im_file
File '\\DC5\dns\dns.log'
SavePos TRUE
InputType LineBased
Exec if $raw_event =~ /^#/ drop();
Exec $message = $raw_event;
</Input>
<Output file_test>
Module om_file
File 'C:\Program Files (x86)\nxlog\data\test_file_output.txt'
# Sync TRUE
OutputType LineBased
</Output>
<Route test>
Path msdhcp,dns => file_test
</Route>
As a result, only DNS events are written in the output file:
21.1.2016. 11:34:00 A6A8 PACKET 0000000003B27E90 UDP Snd 192.168.105.12 3f0d R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0)
21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Rcv 192.168.105.12 3c32 Q [0001 D NOERROR] A (8)PLANKING(3)lab(5)rador(0)
21.1.2016. 11:34:00 A6A8 PACKET 0000000003EDA2C0 UDP Snd 192.168.105.12 3c32 R Q [8085 A DR NOERROR] A (8)PLANKING(3)lab(5)rador(0)
..but only the new ones, i.e. as the source DNS log file is being appended.
I have tried modifying the SavePos parameter to FALSE of both input modules, but to no avail - same result.
Questions:
1. What would be the correct configuration of global NoCache and module specific SavePos parameters, in order to read and output the complete source file, regardless of prior attempts?
2. What is the reason DHCP logs (using wildcard) are not being read (or at least written in the output), as opposed to the same configuration for DNS logs?
I will provide nxlog debug level log if needed. No visible errors there.
Any help greatly appreciated!
djontra created
NXlog om_ssl infinite reconnection after handshake fail with a SaaS Log Centralizer
Pierre.Guceski created
Hi,
This question follow an old one: https://nxlog.co/question/1252/nxlog-omssl-infinite-reconnection-saas-log-centralizer
We are trying to connect NXLog with Logmatic.io with a SSL connection. The certificate provided by Logmatic is still valid and works with Rsyslog, Syslog-NG or Open-SSL.. (you can find the certificate here : https://doc.logmatic.io/docs/logging-from-windows)
However, no matter how hard we tried to connect NXLog we had no succes, and we are now stuck without any ideas of what can we do/test to make it work.
Our config hasn't changed:
<Output out>
Module om_ssl
Host api.logmatic.io
Port 10515
CAFile <path_to_your_.crt_file>
####Add the API key before the event
Exec $raw_event="<your_api_key> "+$raw_event;
</Output>
We tried a lot more configuration with AllowUntrusted TRUE/FALSE, CertFile, etc... But the result is always the same:
2016-01-06 16:19:56 INFO nxlog-ce-2.9.1504 started 2016-01-06 16:19:56 INFO connecting to api.logmatic.io:10515 2016-01-06 16:19:57 INFO successfully connected to api.logmatic.io:10515
2016-01-06 16:19:57 INFO reconnecting in 1 seconds 2016-01-06 16:19:57 ERROR SSL certificate verification failed: unable to get local issuer certificate (err: 20) 2016-01-06 16:19:58 INFO connecting to api.logmatic.io:10515 2016-01-06 16:19:58 INFO successfully connected to api.logmatic.io:10515 2016-01-06 16:19:58 INFO reconnecting in 1 seconds 2016-01-06 16:19:58 ERROR SSL certificate verification failed: unable to get local issuer certificate (err: 20)
We wiresharked the connection to logmatic (which doesn't work) and the connection to Loggly (which worked) both with the same configuration file.
https://drive.google.com/folderview?id=0B7GCHhnklusNcjgxM3ozeTZseVU&usp=sharing
The only difference were in the configuration settings (host, port, certificate..)
It seems that the Handshake between Nxlog and logmatic.io server is the root of this bug. Do you know why ?
Since it's Nxlog which refuse to extablish the connection, do you know why the option "AllowUntrusted false" in Nxlog.conf doesn't work ?
Sorry to bother you again with that but we tried everything and are now short of ideas :/
Many Thanks in advance !
Pierre.Guceski created
ProcessID is always 4
achung21 created
Hi, I recently started using Nxlog with Windows event log.
I'm able to get events off of the Security log and onto my server, however, I've noticed something peculiar.
The value of the 'ProcessID' field is always 4, no matter what process is generating the event.
My nxlog.conf has the following sections:
<Extension json>
Module xm_json
</Extension>
<Input security_log>
Module im_msvistalog
SavePos TRUE
ReadFromLast TRUE
Channel Security
Query *
</Input>
<Output out>
Module om_tcp
Host 10.1.11.50
Port 9000
Exec to_json();
</Output>
<Route 1>
Path security_log => out
</Route>
And the resulting output looks like:
{
"EventTime":"2016-01-18 10:32:13",
"Hostname":"dev-collector.dev.local",
"Keywords":-9214364837600034816,
"EventType":"AUDIT_SUCCESS",
"SeverityValue":2,
"Severity":"INFO",
"EventID":4689,
"SourceName":"Microsoft-Windows-Security-Auditing",
"ProviderGuid":"{54849625-5478-4994-A5BA-3E3B0328C30D}",
"Version":0,
"Task":13313,
"OpcodeValue":0,
"RecordNumber":355,
"ProcessID":4,
"ThreadID":48,
"Channel":"Security",
"Message":"A process has exited.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-5-21-3503561835-834928460-339998367-1000\r\n\tAccount Name:\t\tachung\r\n\tAccount Domain:\t\tDEV-COLLECTOR\r\n\tLogon ID:\t\t0x727518c\r\n\r\nProcess Information:\r\n\tProcess ID:\t0xb64\r\n\tProcess Name:\tC:\\Windows\\System32\\calc.exe\r\n\tExit Status:\t0x0",
"Category":"Process Termination",
"Opcode":"Info",
"SubjectUserSid":"S-1-5-21-3503561835-834928460-339998367-1000",
"SubjectUserName":"achung",
"SubjectDomainName":"DEV-COLLECTOR",
"SubjectLogonId":"0x727518c",
"Status":"0x0",
"ProcessName":"C:\\Windows\\System32\\calc.exe",
"EventReceivedTime":"2016-01-18 10:32:14",
"SourceModuleName":"security_log",
"SourceModuleType":"im_msvistalog"
}
The value of the "ProcessID" field is 4, when it should be 2916 (0xb64), as shown in the "Message" field. Is this a configuration issue or possibly a bug with one of the modules?
Thanks,
Allen
achung21 created
Clear Windows Eventlogs
Runsten created
Hi!
I'm new to nxlog and try to find a way to clear the Windows eventlogs during or after read/export with nxlog.
I've browsed through the manual, but can't find out how, is there a way to do this with a nxlog module/command?
Regards,
Mikael
Runsten created
Testing nxlog to papertrail - Log header entries are logging with (date, external ip address, "logger:") Would like these headers to be (date, HOSTNAME of originator, Process originator) how can I change the header?
Zendog created
Hello - I am testing nxlog to send windows based syslogs to papertrail. The following code seems to produce the logs, however, the log header entries are logging with (date, external ip address, "logger:") on each line. To be consistent with the other logs being collected, would like these headers to be (date, HOSTNAME of originator, Process originator) how can I change the header that nxlog is pushing out to papertrail? The information that I would like to populate the header is within the message body but am not clear how to populate the fields to the headers output?
Here is an example of the current output line:
Jan 11 10:25:30 159.88.217.201 logger: good dog <14>1 2016-01-11T10:25:28.998751-05:00 Win8MacVM1 Microsoft-Windows-Kernel-General 3116 - [NXLOG@14506 Keywords="-9223372036854775792" EventType="INFO" EventID="1" ProviderGuid="{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}" Version="1" Task="0" OpcodeValue="0" RecordNumber="33715" ThreadID="3480" Channel="System" Domain="NT AUTHORITY" AccountName="SYSTEM" UserID="SYSTEM" AccountType="User" Opcode="Info" NewTime="2016-01-11T15:25:28.998234400Z" OldTime="2016-01-11T15:25:28.998234400Z" Reason="3" EventReceivedTime="2016-01-11 10:25:30" SourceModuleName="in" SourceModuleType="im_msvistalog"] The system time has changed to 2016-01-11T15:25:28.998234400Z from 2016-01-11T15:25:28.998234400Z. Change Reason: System time adjusted to the new time zone.
Here is how I would like it to be:
Jan 11 10:25:30 Win8MacVM1 Microsoft-Windows-Kernel-General good dog <14>1 2016-01-11T10:25:28.998751-05:00 Win8MacVM1 Microsoft-Windows-Kernel-General 3116 - [NXLOG@14506 Keywords="-9223372036854775792" EventType="INFO" EventID="1" ProviderGuid="{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}" Version="1" Task="0" OpcodeValue="0" RecordNumber="33715" ThreadID="3480" Channel="System" Domain="NT AUTHORITY" AccountName="SYSTEM" UserID="SYSTEM" AccountType="User" Opcode="Info" NewTime="2016-01-11T15:25:28.998234400Z" OldTime="2016-01-11T15:25:28.998234400Z" Reason="3" EventReceivedTime="2016-01-11 10:25:30" SourceModuleName="in" SourceModuleType="im_msvistalog"] The system time has changed to 2016-01-11T15:25:28.998234400Z from 2016-01-11T15:25:28.998234400Z. Change Reason: System time adjusted to the new time zone.
Any clues from the community on how to accomplish this?
*** Code snippet to follow ***
## UDP Papertrail Test
define ROOT C:\Program Files (x86)\nxlog
define ROOT_STRING C:\Program Files (x86)\\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
# Include fileop while debugging, also enable in the output module below
<Extension fileop>
Module xm_fileop
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
# Enable json extension
<Extension json>
Module xm_json
</Extension>
<Input in>
Module im_msvistalog
Exec to_json();
ReadFromLast TRUE
SavePos TRUE
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
</Input>
<Output out>
Module om_udp
Host xxxx.papertrailapp.com
Port NNNNN
#Test to write unique entry in front of each message but did not change the actual header
Exec to_syslog_ietf();\
$raw_event = " good dog "+$raw_event;
#Use the following line for debugging (uncomment the fileop extension above as well)
Exec file_write("C:\\Program Files (x86)\\nxlog\data\\nxlog_output.log", $raw_event);
</Output>
<Route 1>
Path in => out
</Route>
*** Code snippet end ***
Zendog created
nxlog agent package for CentOS 5 !
kmg created
Hi, Where I can get nxlog agent package for CentOS 5 ?. Only packages available for verion 6 and 7 on download page. Could you please share the download link for the package that support centos 5.x ?
kmg created
Dropped message count
milank created
Sorry for the activity, I hope this to be the last question - I've been playing with nxlog-ce for a week already.
How can I get a count of (unintentionally) dropped records? Something syslog-ng reports in "mark" messages... Do I have to create a buffer processor, drop the records "intentionally" in it and implement my own counter in script? Or does nxlog exhibit somehow its internal flow control stats?
Milan
milank created
Route path branching
milank created
Hi,
please advise me how to configure the following setup:
several inputs (UDP, TCP, internal) joined into common path (i_udp, i_tcp, i_internal => p_pattern ...),
one global processor (pattern, filter) module,
several outputs (file, UDP, TCP),
every message is delivered to one of outputs according to processor's decision.
I call it "route branching". The solutions I have come to so far:
"broadcast": p_pattern => o_file, o_udp, o_tcp -- every output having an Exec filter to drop() messages that should be sent via another output,
"reroute": i_null => o_file; i_null => o_udp; i_null => o_tcp and use Exec reroute() in p_pattern.
The (1) is very inefficient. Otoh, (2) introduces problems with flow control, as the flow control is disabled when using reroute() (a new "feature" of 2.9.1504). So is there an efficient way with flow control enabled? ;-)
Milan
milank created
Source code of community edition for community
milank created
Hello,
will you, please, publish recent source code, the one the latest CE packages were built from? I made quite a few tests with 2.9.1504 on Debian and now would like to patch some minor problems of pm_pattern (and contributing back), but last published sources are 2.8.1248. And I can not get a changelog between theese two versions either (the changelog packaged in the binary package, as mentioned in the previous post, is too brief):
nxlog-ce (2.9.1504) unstable; urgency=low
SVN snapshot release.
-- Botond Botyanszki <boti@nxlog.org> Tue, 08 Dec 2015 14:02:37 +0100
nxlog-ce (0.1-1) unstable; urgency=low
Initial Release.
-- Botond Botyanszki <boti@nxlog.org> Sat, 06 Nov 2010 18:24:10 +0200
Local variables:
mode: debian-changelog
End:
milank created
NXLOG 2.9.1504 New Features, Bug Fixes, Etc
bluelotus created
Where can I find the NXLOG code history for the various releases? I am currently on NXLOG 2.9.1347 and need to know what's new and fixed in 2.9.1504 but cannot locate where this information is.
Thank You,
bluelotus created
NXLog performance
leshqo created
Hi
I'm recently tried to use NXLog community edition to maintain log files. Basically the idea is to rewrite log files one to one from many services on many hosts to one centralized server.
In my test scenario I've created 3 services on one host and started propagate logs. Logs were rewriten succesfully but what I observed:
- It took 40 minutes to rewrite 1GB of logs (1GB connection with 60MB transfer between hosts)
- NXLog CPU usage on host where the logs were stored was about 10% (Intel Core i7)
- In Sysinternals ProcessMonitor I've observed that for one data read (65000 bytes)
16:02:26,4218297 nxlog.exe 12884 ReadFile D:\app\Logs\service1.service\Logs_151130_12.log SUCCESS Offset: 490 945 000, Length: 65 000
there is about 450 partial data writes by 30/90/120 bytes which makes write process high ineffective
16:02:26,4219115 nxlog.exe 12884 LockFile \\192.168.199.46\Shared\logs\service1\Logs_151130_12.log SUCCESS Exclusive: True, Offset: 0, Length: 4 294 967 295, Fail Immediately: False
16:02:26,4219594 nxlog.exe 12884 QueryStandardInformationFile \\192.168.199.46\Shared\logs\service1\Logs_151130_12.log SUCCESS AllocationSize: 490 930 673, EndOfFile: 490 930 673, NumberOfLinks: 1, DeletePending: False, Directory: False
16:02:26,4219996 nxlog.exe 12884 WriteFile \\192.168.199.46\Shared\logs\service1\Logs_151130_12.log SUCCESS Offset: 490 930 673, Length: 84, Priority: Normal
16:02:26,4228112 nxlog.exe 12884 UnlockFileSingle \\192.168.199.46\Shared\logs\service1\Logs_151130_12.log SUCCESS Offset: 0, Length: 4 294 967 295
Is that lack of performance or do I have missed something. Usage of NXLog with such results is unacceptable for me.
My config file:
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
define LOG_SERVER \\\\192.168.199.46
define LOGS_PATH d:\\App\\Logs\\\\*.log
<Extension _syslog>
Module xm_fileop
Module xm_exec
</Extension>
<Input logs>
Module im_file
File "%LOGS_PATH%"
InputType LineBased
PollInterval 15
SavePos TRUE
CloseWhenIdle TRUE
Recursive TRUE
Exec $fileName = file_basename(file_name());
Exec if file_name() =~ /^.*Logs\\(.*?)\.service/ \
{ \
$serviceName = $1; \
}
</Input>
<Output out>
Module om_file
CreateDir TRUE
OutputType LineBased
File "%LOG_SERVER%\\Shared\\Logs\\" + $serviceName + "\\" + $fileName
</Output>
<Route 1>
Path logs => out
</Route>
leshqo created
Gelf - TCP doesn't work with comunity edition
mniehues created
Hi all,
I tried to use the gelf extension with the tcp module, but it doesn't work.
I saw tht the commercial version includes a special tcp gelf enxtension. Supports the community edition the usage of GELF over TCP?
Kind regards
Michael
mniehues created
No connection could be made because the target machine actively refused it.
jvaira created
Hello,
nxlog was sending to nagios log server just fine for a few days and then all of a sudden we started seeing the below errors in the nxlog.log file that are preventing logs from being sent.
ERROR om_tcp send failed; An existing connection was forcibly closed by the remote host.
ERROR couldn't connect to tcp socket on xx.xx.xxx.xx:3515; No connection could be made because the target machine actively refused it.
Is this something that you would support or would we need to go through Nagios support instead? We have already tried restarting the nxlog service but the issue remains.
Thank you
jvaira created
om_odbc sql_exec bind parameters
michaels created
Hello
When I call sql_exec with a bind parameter
if (sql_exec("INSERT INTO Events (Id) VALUES(?)", $ID) == TRUE) {}
I get the following error:
INFO SQLBindParam failed; HY104:1:0:[Microsoft][ODBC SQL Server Driver] Invalid precision value
Same commnad using string concatenatiion works fine:
if (sql_exec("INSERT INTO Events (Id) VALUES(" + $ID +")") == TRUE) {}
Could you help me with it?
michaels created
Issue with syslog message being sent when it shouldnt.
yman182 created
Hello,
I have the following config file. What i want it to do is not send anything that has the word error in it.
<Input watchfile_m_NAME1>
Module im_file
File 'LOGLOCATION'
Exec $Message = $raw_event;
Exec if $raw_event !~ /ERROR/ drop();
#Exec $SyslogSeverityValue = 6;
Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1;
#Exec if $raw_event =~ /ERROR/ $SyslogSeverityValue = 3;
The issue I am having is that it still sends the lines that say the word ERROR, but it sends them as an info message. I wan them to not send them at all. What am I doing wrong?
yman182 created