Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.
Possible Handle Leak in nxlog.exe?
btrash created
Hi!
I have more than 5000 Handles in nxlog.exe
I analysed with Sysinternals Process Explorer.
Most of Handles are from Type Thread.
When i look at threads i only see 10 threads in nxlog.exe but 5000 Thread Handles.
Can you verify that issue?
Version: nxlog-ce-2.9.1347
btrash created
bug??????? nxlog-ce-2.9.1347
sudy1 created
nxlog file has an error , but it's work ok, There is something wrong with the error ???
2015-12-01 13:37:21 INFO nxlog-ce-2.9.1347 started
2015-12-01 13:37:21 INFO connecting to 10.58.8.216:5000
2015-12-01 15:05:07 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
2015-12-01 15:27:39 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
2015-12-01 17:21:17 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
2015-12-01 17:22:36 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
2015-12-01 17:54:05 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
2015-12-01 19:14:30 ERROR if-else failed at line 45, character 244 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 101 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; cannot parse integer, invalid modifier: ';'
nxlog config file:----------------
## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Extension w3c>
Module xm_csv
Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-referrer, $sc-status, $sc-substatus, $sc-win32-status, $time-taken
FieldTypes string, string, string, string, string, string, integer, string, string, string, string, integer, integer, integer, integer
Delimiter ' '
QuoteChar '"'
UndefValue -
</Extension>
<Input in>
# Module im_msvistalog
# For windows 2003 and earlier use the following:
# Module im_mseventlog
Module im_file
File "C:\\inetpub\\logs\\LogFiles\\W3SVC3\\\u_ex*.log"
SavePos TRUE
Exec if $raw_event =~ /^#/ drop(); \
else \
{ \
w3c->parse_csv(); \
$EventTime = parsedate($date + " " + $time); \
$SourceName = "IIS"; \
$Message = to_json(); \
}
</Input>
<Output out>
Module om_tcp
Host 10.58.8.111
Port 5002
# Exec to_syslog_snare();
</Output>
<Route 1>
Path in => out
</Route>
sudy1 created
nxlog CE ipv6 targets
rherold created
hi,
we have here some ipv6 only networks where we tried nxlog CE agent for windows. Seems that there is no ipv6 support for log targets.
Ic there support planed for ipv6?
rherold created
NXLog on Windows server 2003 (im_mseventlog) invalid keyword: Query problem
emve created
Hi,
I have installed NXLog on Windows server 2003 with this configuration (example from NXLog reference manual)
<Input in>
# Module im_msvistalog
# For windows 2003 and earlier use the following:
Module im_mseventlog
Query <QueryList> <Query Id="0"> <Select Path="Security">*</Select> </Query> </Querylist>
</Input>
I received "ERROR invalid keyword: Query" in nxlog.log :
2015-11-27 10:57:38 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:21
This configuration is working fine with "Module im_msvistalog" on Windows Server 2008 and later.
(http://www.developpez.net/forums/d1545842/systemes/windows/windows-serveur/solution-nxlog-graylog/)
How can I fix this problem ?
Thank you,
emve created
nxlog-ce-2.9.1347 restart fail Not enough fields in CSV input
sudy1 created
2015-11-25 12:47:03 INFO nxlog-ce-2.9.1347 started
2015-11-25 12:47:03 INFO connecting to 10.1.252.239:5000
2015-11-25 12:47:03 ERROR if-else failed at line 45, character 241 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 45, character 98 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 15, got 12 in input '192.168.2.xx GET /page/v1/chengjiao/index.aspx placeholder=b1m8u8z2g7 80 - 220.181.108.81 Mozilla/5.0+(compatible;+Baiduspider/2.0;++http://www.baidu.com/search/spider.html) 200 0 0 328'
sudy1 created
Windows Eventlog - registry ref objects do not resolve
mwber1 created
I am new and I apologies in advance if this question has been asked already.
Problem:
I am us nxlog to forward windows eventlogs (json format) to central logging system.
Not all object are resolved in the message … example.
Object: Object Server:
DS Object Type: %{e0fa1e8c-9b45-11d0-afdd-00c04fd930c9}
Object Name: %{cc0985a1-b646-4957-bb95-ac8fe9ad147a}
Question:
Is that normal or is there something I can do to resolve those references?
mwber1 created
getting error "ERROR subprocess '8319' returned a non-zero exit value of 1" when calling external script
nxlogdesonim created
when i manually execute python script it works fine (cat /var/log/nxlog/pktdesign-alerts.log1|./sec-parse.py) , but whe nxlog calls keep on seeing the error messages below and scripot does not execute,
===> when nxlog config was whats listed below produced error "2015-11-23 20:52:41 ERROR subprocess '4843' returned a non-zero exit value of 127"
<Output alertout-pktdesign>
Module om_file
File "/var/log/nxlog/pktdesign-alerts.log1"
Exec exec_async("/bin/sh", "-c", 'echo "' + $raw_event + '"|./sec-parse.py' );
</Output>
===> when nxlog config was what's listed below produced error "2015-11-23 21:25:41 ERROR subprocess '8319' returned a non-zero exit value of 1"
<Output alertout-pktdesign>
Module om_file
File "/var/log/nxlog/pktdesign-alerts.log1"
Exec exec_async("/bin/sh", "-c", 'echo "' + $raw_event + '"|/etc/nxlog/sec-parse.py' );
</Output>
any help is appreciated.
thanks
nxlogdesonim created
How many folders/files nxlog can monitor?
zpp created
Hi,
I'm setting up nxlog on a window box to read files to send to elasticsearch, and I have some questions:
1) is there a limit on the number of im_file inputs that a output (om_tcp) can handle? And is nxlog reading those monitored files concurrently or in serial one after each other?
2) we can setup multiple paths in nxlog, different path means parallel processing, e.g. each path is handled by different threads??
3) can we install mutiple nxlog services in one box?
Thanks a lot!
zpp created
Forwarding logs with im_file om_file
leshqo created
Hi there,
I have several hosts where lots of logs are generated. On every of this hosts logs are placed into one directory and have different names with *.log extension. My point is to use nxlog on that hosts to forward logs into one central storage but I have problem with populating files names. I would like that file source/first.log to be populated to destination/first.log. Similarly source/second.log -> destination/second.log etc. Below is my config file.
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
define LOG_SERVER \\\\192.168.199.10
<Extension _syslog>
Module xm_syslog
</Extension>
<Input app-logs>
Module im_file
File "d:\\Logs\\\\*.log"
InputType LineBased
<Schedule>
Every 30 sec
Exec log_info("scheduled execution at " + now());
</Schedule>
Exec $fileName = file_name();
</Input>
<Output out>
Module om_file
CreateDir TRUE
File "%LOG_SERVER%\\Shared\\Logs\\" + $fileName
</Output>
<Route 1>
Path app-logs => out
</Route>
Logs are not populating and I have in nxlog logs something like that
2015-11-20 15:11:00 ERROR CreateDir is TRUE but couldn't create directory: \\192.168.199.10\Shared\Logs\d:\Logs\; The specified path is invalid.
Will be gratefull for help with my issue.
leshqo created
Declaring field types.
Grenage created
Hi everyone,
I've got a box running Kibana and Elasticsearch, with the information being handed across by fluentd. It's all working pretty well, and I've moved on to pushing Windows events through - which is where I'm struggling.
I'm using nxlog on a windows machine, and it's pushing the data directly to elasticsearch - unfortunately I cannot for love nor money get the date to be handled as a date, rather than a string; the nxlog output is:
URL http://192.0.0.10:9200
ContentType application/json
Exec set_http_request_path(strftime($EventTime, "/windowsevents-%Y.%m.%d/" + $SourceModuleName)); delete($EventReceivedTime); rename_field("timestamp","@timestamp"); to_json();
I create a new index with a pattern of *windowsevents*\*, but the *EventTime* field is stated as a string, not a date - so I can't sort data by age. *@timestamp* clearly isn't getting populated, as the *timestamp* field isn't being used. Has anyone used a similar setup? Is there a way to push the data on as a date?
Apologies if I've missed out pertinent information - it's all a bit new to me.
Grenage created
Installation of nxlog via gpo
Doppelbodenninja created
Hello,
I am trying to update the nxlog software via gpo startscript on our clients.
Powershell:
(Get-WmiObject -Class Win32_Product | where {$_.Name -like "nxlog"} | where {$_.Version -ne "2.9.1427"}).Uninstall()
Start-Process \\server0001\NETLOGON\SoftwarePackets\nxlog-2.9.msi /quiet
Batch:
WMIC /interactive:off product where 'name like "%%nxlog%%" and not version like "%%2.9.1427%%"' call uninstall
msiexec.exe /package \\server0001\NETLOGON\SoftwarePackets\NXLOG\nxlog-2.9.msi /quiet
The Scripts are working fine, if manually started on the client.
Only in the group policy the scripts run trought, but doesn´t install the software. (Other Software like Java, PDF24 etc. is working fine this way)
Kind regards
Doppelbodenninja created
MS SQL Profiler
ms created
Hi!
What about support for MS SQL Profiler trace (*.trc) files? Please, can nxlog to read from these files?
Thanks.
ms created
Collecting IIS Logs
chicagosteve created
I cannot seem to get NXLog to ship IIS Logs to LogAnalyzer. It is collecting Event logs perfectly. I have commented out all event logs in an attempt to isolate just IIS flow and there is nothing.
define ROOT C:\\Program Files (x86)\\nxlog
define ROOT_STRING C:\\Program Files (x86)\\nxlog
define CERTDIR %ROOT%\\cert
Moduledir %ROOT%\\modules
CacheDir %ROOT%\\data
Pidfile %ROOT%\\data\\nxlog.pid
SpoolDir %ROOT%\\data
LogFile %ROOT%\\data\\nxlog.log
# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
# Module xm_fileop
#</Extension>
<Extension json>
Module xm_json
</Extension>
#<Extension syslog>
# Module xm_syslog
#</Extension>
<Input internal>
Module im_internal
Exec $Message = to_json();
</Input>
<Input IIS>
Module im_file
File C:\inetpub\logs\LogFiles\W3SVC1\*
SavePos True
InputType LineBased
</Input>
# Windows Event Log
#<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
# Module im_msvistalog
#Uncomment im_mseventlog for Windows XP/2000/2003
#Module im_mseventlog
# Exec $Message = to_json();
# Exec if ($EventID == 5156) drop();
# Exec if ($EventID == 4656) drop();
# Exec if ($EventID == 4658) drop();
#</Input>
<Output out>
Module om_tcp
Host x.x.x.x
Port 514
</Output>
<Route 1>
Path internal, eventlog => out
</Route>
I am sure I am missing something simple. I have tried file path quotes (single and double), several different Input configs with varying levels of detail, I have tried variations of the wildcard to pull the log file, I even tried changing to double slashes in the file path (grasping at straws with that one).....
Any ideas would be appreicated...
chicagosteve created
om_http module with basic authentication
Konsantin created
Hello guys, i have http input in logstash configuration:
http {
port => "60114"
type => "PaaS"
user => "logs_account"
password => "password"
}
And i wanna send logs from NXLOG to logstash via this http endpoint. I think i can login/password in this format:
<Output elasticsearch>
Module om_http
URL "http://logs_account:password@logsserver:60111/"
</Output>
But NXLOG can't parse this URL and i have errors in log file
2015-10-30 16:25:55 ERROR invalid url: "http://logs_account:password@logsserver:60111/"
2015-10-30 16:25:55 ERROR Failed to parse url "logs_account:password@logsserver:60111/" at C:\Program Files (x86)\nxlog\conf\nxlog.conf:43
Konsantin created
Can NXLOG event correlator buffer/retain messages in memory
nxlogdesonim created
can event coorelation be used with a trigger such that when an event matches it collects subsequent if all come in within specifc time frame (say within 30second from first event) and write those or send via email?
we currently have such functionality out of per based "SEC" but are trying to migrate to NXLOG.
thanks.
nxlogdesonim created
PM_buffer module
j_aagaard created
Hi
Is there a garbage collector service, when using the pm_buffer to disk, so that the buffer file on disk is emptied? If yes, how often is this run, and can it be configured?
/Johan
j_aagaard created
Issue selecting specific levels of windows application logs in NXLog
pcort42 created
I'm trying to pass only Warning / Error / Critical level Application Logs through NXLog to my ELK stack. When I have this configuration
<Input EventLog_In> Module im_msvistalog
<QueryList>\ <Query Id="0">\
<Select Path="Application">*</Select>\
</Query>\ </QueryList>
Exec to_json(); </Input>
everything works fine, and I'm collecting all levels of Application logs. I tried putting in a parameter on the <Select Path> line like this
<Select Path="Application">*[Application/Level=1]</Select>\
And it craps itself and I get nothing. NXLog isn't reporting any issue, and I'm not seeing anything on the logstash side of things.
I got the information about Event Viewer querying from this thread and adapted it to my use case: https://serverfault.com/questions/543494/query-specific-logs-from-event-log-using-nxlog
pcort42 created
Is there a way to aggregate multiple messages into one email?
nxlogdesonim created
we need to separate and aggregate events per IP address during a period of time, such that, a single email is sent conteining multiple messages where the same IP is present, is this something that can be done with pm_evcorr?
i hace tried and not yet able to get this functionality, if possible pls provide a quick example.
thanks.
nxlogdesonim created
ASSERTION FAILED at line 33 in xm_gelf.c/xm_gelf_writer_udp()
logstarter created
hi , 2 days ago i started getting this error :
ERROR ### ASSERTION FAILED at line 33 in xm_gelf.c/xm_gelf_writer_udp(): "deflateInit(&strm, Z_DEFAULT_COMPRESSION) == Z_OK" ###
INFO reconnecting in 1 seconds
anu idea ? tks .
logstarter created
CSV-input: converting specific field(s) to lowercase
nomoresecrets created
Dear community,
I'm currently working on parsing MS Exchange logs and sending them via GELF to my graylog instance.
I'd like to convert the sender- and recipient-address field to lowercase. Sounds pretty easy, in fact, I need help :(
my current config looks like this (below). Any help is appreciated.
I've tried to work with "Exec $sender-address = lc($sender-address);" within the input as well as Output backet - neither did work.
define BASEDIR C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\MessageTracking
<Extension csv>
Module xm_csv
Fields $date-time, $client-ip, $client-hostname, $server-ip, $server-hostname, $source-context, $connector-id, $exchange_source, $event-id, $internal-message-id, $message-id, $recipient-address, $recipient-status, $total-bytes, $recipient-count, $related-recipient-address, $reference, $message-subject, $sender-address, $return-path, $message-info, $directionality, $tenant-id, $original-client-ip, $original-server-ip, $custom-data
FieldTypes string, string, string, string, string, string, string, string, string, integer, string, string, string, integer, integer, string, string, string, string, string, string, string, string, string, string, string
Delimiter ,
</Extension>
<Input in_exchange>
Module im_file
File '%BASEDIR%\MSGTRK????????*-*.LOG'
SavePos TRUE
Exec if $raw_event =~ /HealthMailbox/ drop();
Exec if $raw_event =~ /^#/ drop();
Exec csv->parse_csv();
</Input>
<Output out_exchange>
Module om_udp
Host graylog.local
Port 12203
OutputType GELF
Exec $SourceName = 'exchange_msgtrk_log';
</Output>
<Route exchange>
Path in_exchange => out_exchange
</Route>
nomoresecrets created