I'm attempting to compile the latest nxlog on FreeBSD and it fails with the following error. Any thoughts as to a fix or workaround would be appreciated. My environment is as follows: FreeBSD 10.1-STABLE OpenSSL 1.0.2h  3 May 2016 [/usr/local/src/nxlog-ce-2.8.1248]#make Making all in src Making all in common make  all-am /bin/sh ../../libtool --tag=CC    --mode=compile cc -DHAVE_CONFIG_H -I. -I/usr/local/include/apr-1   -I/usr/local/include  -pipe -W -Wall -Wshadow -Wno-address -Wcast-qual -Wbad-function-cast -Wsign-compare -Wconversion -funsigned-char -Werror-implicit-function-declaration -Wswitch-default -fstrict-aliasing -Wpointer-arith -Wcast-align -Wwrite-strings -Wmissing-noreturn -Wmissing-format-attribute -Wformat -Wextra -Werror=format-security -rdynamic -ggdb3 -MT ssl.lo -MD -MP -MF .deps/ssl.Tpo -c -o ssl.lo ssl.c libtool: compile:  cc -DHAVE_CONFIG_H -I. -I/usr/local/include/apr-1 -I/usr/local/include -pipe -W -Wall -Wshadow -Wno-address -Wcast-qual -Wbad-function-cast -Wsign-compare -Wconversion -funsigned-char -Werror-implicit-function-declaration -Wswitch-default -fstrict-aliasing -Wpointer-arith -Wcast-align -Wwrite-strings -Wmissing-noreturn -Wmissing-format-attribute -Wformat -Wextra -Werror=format-security -rdynamic -ggdb3 -MT ssl.lo -MD -MP -MF .deps/ssl.Tpo -c ssl.c -o ssl.o cc: warning: argument unused during compilation: '-rdynamic' ssl.c:46:27: warning: cast from function call of type 'apr_os_thread_t'       (aka 'struct pthread *') to non-matching type 'unsigned long'       [-Wbad-function-cast]     ret = (unsigned long) apr_os_thread_current();                           ^~~~~~~~~~~~~~~~~~~~~~~ ssl.c:343:52: warning: cast to 'void *' from smaller integer type 'int'       [-Wint-to-void-pointer-cast]     SSL_set_ex_data(ssl, nx_ssl_verify_result_idx, (void *) verify_result);                                                    ^ ssl.c:372:12: error: implicit declaration of function 'SSLv3_method' is invalid       in C99 [-Werror,-Wimplicit-function-declaration]     meth = SSLv3_method();            ^ ssl.c:372:10: warning: incompatible integer to pointer conversion assigning to       'const SSL_METHOD *' (aka 'const struct ssl_method_st *') from 'int'       [-Wint-conversion]     meth = SSLv3_method();          ^ ~~~~~~~~~~~~~~ 3 warnings and 1 error generated. *** Error code 1 Stop. make[3]: stopped in /usr/local/src/nxlog-ce-2.8.1248/src/common *** Error code 1 Stop. make[2]: stopped in /usr/local/src/nxlog-ce-2.8.1248/src/common *** Error code 1 Stop. make[1]: stopped in /usr/local/src/nxlog-ce-2.8.1248/src *** Error code 1 Stop. make: stopped in /usr/local/src/nxlog-ce-2.8.1248    

Hello Team, I am sending MS SQL Server Error logs from NXLog to our TCP server for processing. I need to select File encoding to Unicode, I tried but it is goving error if I set encoding to Unicode/unicode. If I set encoding to utf-8/utf-16 it is adding dot(.) after every single character in a source log line. Any suggestions/help would be really appriciated. Thanks, Mahesh mahibabar@gmail.com +91-9923622938

NXLog Redis output module  sends data to a Redis server with the RPUSH command allowing only to change the Redis key ( = database). RPUSH inserts all the specified values at the tail of the list stored at key I suggest that the Redis command has to be implemented as a parameter and to support at least the LPUSH. LPUSH inserts all the specified values at the head of the list stored at key There are clients that support only retreiving data from the head. The most usual case is to send data to Redis with LPUSH and then have Logstash poll record from Redis with LPOP.   Thank you.

I have a multiline log sample as follow: 2/22/2016 4:19:30 PM 09F8 EVENT The DNS server has finished the background loading of zones. All zones are now available for DNS updates and zone transfers, as allowed by their individual zone configuration. 2/22/2016 4:19:38 PM 082C EVENT The DNS server has started. 2/22/2016 4:19:38 PM 08E0 PACKET 0000000001564000 UDP Rcv 192.168.2.213 526c Q [0001 D NOERROR] SOA (5)_ldap(4)_tcp(4)snpl(3)com(2)np(0) UDP question info at 0000000001564000 Socket = 324 Remote addr 192.168.2.213, port 57611 Time Query=104249, Queued=0, Expire=0 Buf length = 0x0fa0 (4000) Msg length = 0x0028 (40) Message: XID 0x526c Flags 0x0100 QR 0 (QUESTION) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 0 Z 0 CD 0 AD 0 RCODE 0 (NOERROR) QCOUNT 1 ACOUNT 0 NSCOUNT 0 ARCOUNT 0 QUESTION SECTION: Offset = 0x000c, RR count = 0 Name "(5)_ldap(4)_tcp(4)snpl(3)com(2)np(0)" QTYPE SOA (6) QCLASS 1 ANSWER SECTION: empty AUTHORITY SECTION: empty ADDITIONAL SECTION: empty I wish to add the some string at end of this log say (ENDOFLOG) to the end of logfile. How can i achieve this?

Has anyone seen this with NXlog when you are able to telnet to the host.  When I set this up in a test lab it works fine.  I assume a firewall but I can telnet fine to the host IP and port. 2016-05-20 09:41:31 ERROR om_tcp detected a connection error; An existing connection was forcibly closed by the remote host. 2016-05-20 09:41:32 INFO connecting to seclog.lowes.com:514 Very basic config. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>     Module      xm_syslog </Extension> <Input in> #Module      im_msvistalog # For windows 2003 and earlier use the following: Module      im_mseventlog Exec if $EventID NOT IN (528, 529, 567, 592, 601, 602, 608, 612, 636, 7034, 7035, 7036, 7040, 4097, 64004, 2, 3005) drop(); </Input> <Output out>     Module      om_tcp     Host        192.168.x.x     Port        514     Exec        to_syslog_snare(); </Output> <Route 1>     Path        in => out </Route>  

Hi All,    I use nxlog-ce-2.9.1504 to read log and parse it,the format of log is like: 20160523 10:58:22 sda              0      0    0    0      14      2    2    9       9     1     1      0    0 20160523 10:58:22 sdb              0      3    2    0      20      4    8    4       2     5     1      7    2 I want get the purpose: if the third field is "sda",nxlog automatically add a string value "disk1" at the tail of  the same row,if "sdb",add "disk2",like 20160523 10:58:22 sda              0      0    0    0      14      2    2    9       9     1     1      0    0  disk1 20160523 10:58:22 sdb              0      3    2    0      20      4    8    4       2     5     1      7    2  disk2   I configure the nxlog.conf as following,but it can not add any fields,I don't know where the mistake,anybody could help me? thanks so so so much~ <Extension kvp>     Module      xm_kvp </Extension> <Input perf>     Module      im_file     File        "/collectl/LKNODE01*.dsk"     SavePos     FALSE     ReadFromLast FALSE     Exec    if $raw_event =~ /^(\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+) (\S+)/\                  {\                         kvp->parse_kvp();\                         if $3 == "sda" $raw_event = $raw_event + " disk1";\                         else if $3 == "sdb" $raw_event = $raw_event + " disk2";\                 } </Input> <Output file>     Module      om_file     file        '/tmp/2.txt' </Output> <Route 1>     Path        perf=> file </Route>

Hi all, I am new with NXLOG and I have a very basic question to ask you : is it possible to simply copy a file from server A (source file name : c:\osit\log\df_mon.log) to server B (target file name : c:\osit\log15\df_mon.log). I spent already hours on NXLOG config file but I was not able to make it worked because the instruction "File" seeems not to be compatible wit the "om_tcp" output modeule and Host/Port settings. Thanks a lot in advance, Sophie

Hi everyboy, I have a log rotation folder on catalina application with a filename based on date like that "catalina.2016-04-15.log". Catalina create a log file with the date of the day, only if he need to write log (not automatically a file per day). Some one has a exemple for configure NX log ?   Thanks in advance !    

I have the usual error "Unexpected data from server ..." using om_http to send Windows event logs through Nginx to Redis. The problem is that when randomly this error happens ( once every 5 days) the service unexpectently stops ! From the source code I can understand that NXLog receives a response which cannot map to a correnponding request. It has a response with an appropriate length but the corresponding request is null! Possibly is a bad logic implementation, or even a network problem that could cause request/response mismatch.I really don't know. But I also really can't understand why NxLog preferres to die instead of  just return from the function in case this happens and not execute the code following the condition that drives the service to crash at the end. Stelios

Would this be the correct placement to add the filter event ID string?  Should anything esle be commented out? #Windows Event Logging of Security,System and Application Logs   Module      im_mseventlog   Exec to_syslog_snare(); Exec if $EventID NOT IN (528, 529, 567, 592, 601, 602, 608, 612, 636, 7034, 7035, 7036, 7040, 4097, 64004, 2, 3005) drop(); </Input>      

When NXLog ships a Windows event, it appears to be changing the Windows original severity level, and replacing it with SeverityValue and Severity with different values.   What is the mapping of these values?  If Windows has severity values, with "Level" being 4 for Information, 3 for Warning, 2 for Error and 1 for Critical, what is the nxlog created SeverityValue?   Also, I couldn't find an explaination of why this value is changing.  

Hi, We are testing NXlog to ship the security logs to our security team.  We are using XP embedded and it working fine.  Security is asking us to only send specific event ID's.  I have looked at the documentation and it appears that we may not be able to do this with XP.  I was going to do the custom view but XP does not support that. We are supported on the embedded version of XP. My question.  Has anyone tried to do this with XP or is it even possible? Thanks, David Martin  

Hi,  I am doing a rule which detects me are making multiple responses of ICMP from the same IP in which there are diferent IPs.   The problem that i have is that don't write in the output with raw_event and file_write. In file_write I put the "otro" file.  Do you know it can be?   Antonio.   File nxlog.conf:   <Extension fileop>     Module      xm_fileop </Extension> <Input in4>     Module    im_file     File    "/home/antonio/Descargas/sn"     SavePos TRUE     Exec    if ($raw_event =~ /^\d\d:\d\d:\d\d.(.+)/) {     \                     $Message = $1;            \                     $raw_event = $Message;       \                 }     exec if $Message =~ /IP (\S{1,}) > \S{1,}:/ $IP=$1; </Input> <Input internal>     Module    im_internal     Exec    $raw_event = $Message; </Input> <Output out4>     Module    om_file     File    "/home/antonio/Descargas/nx" </Output> <Processor evcorr>     Module       pm_evcorr     <Thresholded>                 Condition  $Message =~ /^ICMP echo reply/             Threshold  3            Interval   120     Context $IP     Exec       $raw_event = "3 ECHO REPLY packets from host $IP";     Exec        file_write("/home/antonio/Descargas/otro", "3 ECHO REPLY packets from host $IP");     </Thresholded> </Processor> <Route 4> Path    in4, internal => evcorr => out4 </Route>   File sn:   17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 17:11:33.953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64   File nx:   inode changed for '/home/antonio/Descargas/sn': reopening possibly rotated file 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64 953173 IP mad01s25-in-f3.1e100.net > 192.168.20.153: ICMP echo reply, id 62478, seq 2, length 64   File otro:   Thanks :))

Hi all! Im new and would really appreciate any assistance/feedback on implementation... I'm trying to implement Audit Reduction in my environment which includes (4) Windows 7 boxes , (2) RHEL 7 boxes (SYSLOG) and (1) RHEL 6 box (SYSLOG)... We have all the logs writing to one of the Windows 7 workstations. The logs need to be manually pulled off and put in another location. This environment is replicated 7 times (7 systems with the same configuration with 7 appliances each) They ultimately get to a Dropbox that forwards them off to an ElasticSearch/Kibana instantiation....So 49~ devices/appliances reporting. I would like to know if the Community Edition would work or if I need to purchase Enterprise? Does NXlog need to be installed on each appliance? Thanks!

Hi Everyone, New forum user, so sorry if I do not respect some uses rules…and I’m French by the way, so sorry about my bad English level too…;-) I use Nxlog in the following context : A machine A send periodically a file on a machine B (same file name) Machine B is used like a first level of log centralization. Nxlog is installed on it Then, Nxlog send log to Machine C who host Logstash I’m looking for a process who remove input files after end of reading and processing, but I’m not sure about how process “in-out” operate, is-it after each line or after a n entire file reading ? How can I test a “end of reading” condition ? Right now, I’m going to test something like that : <Extension fileop>  Module xm_fileop </Extension> <Input in>     Module im_file     SavePos TRUE     ReadFromLast TRUE     PollInterval 900 # 15mn     File "/var/log/inpufile*.log"  Exec $InputFileName1 = file_name() </Input> <Output logstash>     Module om_tcp  Port 6002 Host 10.x.y.z Exec file_remove($InputFileName1); </Output> <Route 1>     Path in => logstash </Route> Is it correct ? Thanks in advance for your help

Hi there,   Has any of you encountered this virus warning? The scan was done using TrendMicro HouseCall. Is my nxlog infected? Thanks for your help

nxlog is crashing with the following error:   Faulting application name: nxlog.exe, version: 0.0.0.0, time stamp: 0x5666d55e Faulting module name: ntdll.dll, version: 6.3.9600.18202, time stamp: 0x569e72c5 Exception code: 0xc0000005 Fault offset: 0x000192cb Faulting process id: 0x1b60 Faulting application start time: 0x01d18540c8297bd3 Faulting application path: C:\Program Files (x86)\nxlog\nxlog.exe Faulting module path: C:\Windows\SYSTEM32\ntdll.dll Report Id: 06d89363-f134-11e5-80dd-005056a619fb Faulting package full name: Faulting package-relative application ID:   config file:   define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Input eventlog> # Use 'im_mseventlog' for Windows XP and 2003 Module im_msvistalog SavePos FALSE ReadFromLast FALSE Query <QueryList>\ <Query Id="0">\ <Select Path="Pool2PdfCreator.Produce">*</Select>\ </Query>\ </QueryList> </Input> <Output out> Module om_tcp Host 10.36.52.62 Port 12201 Exec $EventTime = integer($EventTime) / 1000000; to_json(); Exec log_info("RecordNumber: " + $RecordNumber); </Output> <Route r> Path eventlog => out </Route>   (during troubleshooting, I have narrowed down the query to one eventsource and also added Exec log_info("RecordNumber: " + $RecordNumber); to be able to pinpoint the exact entry that causes the issue. I was able to locate the entry that causes the crash. The strange thing is, it sometimes goes through, most of the times causes crash. I am not comfortable with sharing the entry here but I can send it via e-mail. This definitely looks like a bug.   nxlog version: nxlog-ce-2.9.1504

Using nxlog in front of logstash on a server.  On the same user, I want to use nxlog to replace rsyslog.  Seems pretty simple.  The only issue is the file format is slightly different than the what rsyslog outputs.  I see: <78>May 6 13:50:01 CRON[19454]: (root) CMD ( /opt/observium/discovery.php -h new >> /dev/null 2>&1) vs: Jul 16 18:00:01 monitor01 CRON[6871]: (root) CMD ( /opt/observium/poller-wrapper.py 16 >> /dev/null 2>&1) Main difference is the <NN> at the beginning of the line and the missing hostname (monitor01).  Here is my .conf: <Input in_uds> Module im_uds UDS /dev/log </Input> <Input in_kernel> Module im_kernel </Input> <Output out> Module om_file File "/var/log/syslog" </Output> <Route local_route> Path in_uds, in_kernel => out </Route> Is there a simple change I can make to get the desired format?  Thanks.  -- Bud    

I'm using NXlog to ship windows event logs to an ELK stack.  I need to preserve the datetime when the event happened <TimeCreated SystemTime=> that is stored in the event log.   However, the NXLog that is shipped doesn't preserve <TimeCreated SystemTime>, which I assume is because its invalid json.  How can I preserve this in my nxlog.conf?  Otherwise, I'm stuck with EventTime, which appears to be the datetime of when nxlog processes the event, not when the event happened.   How do I handle this?

Hello everyone, This is my first time posting in this community forum, so any help would be greatly appreciated. I've been working with NxLog and ElasticSearch for a few months now and I've had mostly no issues with it until very recently, where a new ElasticSearch index was created in order to accomodate the new structure of our logs. With that, we also had to update our existing nxlog.conf file. We have about 3 different ElasticSearch endpoints with the same setup that we currently work with, and at one point during the week we had run out of storage space. After increasing the storage size for all 3 endpoints, two environments appeared to continue sending new information up to ElasticSearch with no further problems. However, the third environment's NxLog services appear to be stuck in both existing AWS instances and newly created instances, repeating the following NxLog log entry over and over: 2016-04-29 15:33:12 INFO connecting to search-stage-logging-udf7h4lq2bsm245ciawp2stcvu.us-east-1.es.amazonaws.com:80 2016-04-29 15:33:12 INFO reconnecting in 1 seconds 2016-04-29 15:33:12 ERROR ### PANIC at line 2456 in module.c/nx_module_pollset_add_socket(): "failed to add descriptor to pollset: Not enough space ; [cannot dump backtrace on this platform]" ### This was the log entry that we initially discovered that alerted us we had run out of space in ElasticSearch. However, the ElasticSearch dashboard does not show a lack of space anymore, so it's a bit confusing why NxLog would continue to output log entries in the other two environments Basically, I have two questions: 1. Is this a type of scenario where the NxLog service's working state has been stuck unable to see there is space available? Or does the fault lie with ElasticSearch not showing storage space correctly? 2. If the Nxlog service has been stuck in this state, is there a configuration or some other automated procedure for NxLog to get the service to restart itself on multiple failures?   If anyone has gone through a similar experience, any tips would be greatly appreciated. Thank you for your time.