I have some windows event log being collected by nxlog, I need to send this to Sensu client(using UDP port 3030) on the same machine. Do you have any experience on it?

Have you done anything simliar on it? Just want to know how to configure nxlog and sensu client to make it work.

We have nxlog CE pushing to a GELF TCP input in Graylog, and the timestamp field received from nxlog appears to not have the milliseconds (i.e. it ends in ".000"), resulting in out-of-order messages in Graylog within a 1-second window.  Other sources (Graylog Collectors, apps pushing directly, etc.) include the original millisecond value as expected.  For Graylog inputs receiving nxlog messages we've had to set up an extractor to extract the timestamp from the message itself.  Are there any options to keep millisecond precision with nxlog?

Hello, I am using NXLOG on Windows 2012 to get DNS logs forwarded to my syslog server.  I have enabled DNS logging on the Windows server and see the dns.log file is getting created under C:\Windows\System32\DNS\ folder .   However my nxlog.conf is unable to browse or parse to get to this directory.   I have made sure to check the log  file is dns.log and not dns.txt in Windows.

If i mention the below in my nxlog.conf file, i get an error "   WARNING input file does not exist: C:\Windows\System32\dns\dns.log"

<Input in>
    Module  im_file
    File  "C:\\Windows\\System32\\dns\\dns.log"
    SavePos TRUE
    InputType LineBased
</Input>

If i mention the below File path in my nxlog.conf then i get an error : "ERROR failed to open directory: C:\Windows\System32\dns: The system cannot find the path specified."

<Input in>
    Module  im_file
    File  "C:\\Windows\\System32\\dns\\dns*"
    SavePos TRUE
    InputType LineBased
</Input>

Same thing, even if i use single quotes & single \  i get the same error -   "  WARNING input file does not exist: C:\Windows\System32\dns\dns.log

<Input in>
    Module  im_file
    File  'C:\Windows\System32\dns\dns.log'
    SavePos TRUE
    InputType LineBased
</Input>

Can someone plss help ? This is drving me crazy

Hello!

I want pass value a variable from NxLog in Perl script, and pass variable in nxlog after running the script.

how do I do this ?

Thank

Hello.

Help me please, i beginner in NxLog. I use NxLog on Unix and use Perl module, and script perl.pl

What this error ERROR the perl interpreter failed to parse /tmp/nxlog/Perl/perl.pl??

How do i fix this??

beginner

Hi,

RFC5424 and all transports (except obsolete non-octet-counted TCP) can handle MSG containing ANY character including newlines and carriage returns.

In violation of the above, NxLog's to_syslog_ietf() function backslash-escapes these two characters. Furthermore, the escaping scheme is broken because it doesn't also escape the escape character itself (the backslash) so there's no way to reliably un-escape the MSG on the receiving end.

The correct behaviour is to stop escaping these characters altogether. In the rare case that someone needs to send multiline messages over non-octet-counted TCP, they can escape/unescape the $Message themselves using NxLog's replace() function.

Patch below.

RFC References: https://tools.ietf.org/html/rfc5424#section-6.4 https://tools.ietf.org/html/rfc6587#section-3.4

Regards, Ron MacNeil

--- src/modules/extension/syslog/syslog.c.orig 2014-07-19 23:52:06.000000000 +1000 +++ src/modules/extension/syslog/syslog.c 2016-07-26 14:01:57.296175500 +1000 @@ -1321,16 +1321,8 @@ nx_syslog_add_structured_data(logdata);

 // Append message

• i = (int) logdata->raw_event->len; nx_string_append(logdata->raw_event, " ", 1); nx_string_append(logdata->raw_event, msg.string->buf, (int) msg.string->len);

• for ( ; i < (int) logdata->raw_event->len; i++ )

• { // replace linebreaks with space

•   if ( (logdata->raw_event->buf[i] == '\n') || (logdata->raw_event->buf[i] == '\r') )
  

•   {
  

•       logdata->raw_event->buf[i] = ' ';
  

•   }
  

• }

  if (tmpmsg != NULL) { // clean up temp copy

Hi,

I want to use buffer (disk and memory) before sending my data to a TCP syslog, for that I create 2 processors (diskBuffer and memoryBuffer) that I use in a route : IN => diskBuffer => memoryBuffer => out.

When i try to create another route with one or more different process but which also uses buffers (IN2 => P1 => P2 => diskBuffer => memoryBuffer => out), i have an error message on log :
2016-07-23 13:28:51 ERROR cannot add processor module 'diskBuffer' to route 'XXX' because it is already added to route 'YYYY'
2016-07-23 13:28:51 ERROR cannot add processor module 'memoryBuffer' to route 'XXX' because it is already added to route 'YYYY'

This concept is not really explain in the community documentation and suggest to think of the opposite with the example given in page 18 :
Example 4.14 Different routes
<Input in1>
    Module im_null
</Input>
<Input in2>
    Module im_null
</Input>
<Processor p1>
    Module pm_null
</Processor>
<Processor p2>
    Module pm_null
</Processor>
<Output out1>
    Module om_null
</Output>
<Output out2>
   Module om_null
</Output>
<Route 1>
    # no processor modules
    Path in1 => out1
</Route>
<Route 2>
    # one processor module
    Path in1 => p1 => out1
</Route>
<Route 3>
    # multiple modules
    Path in1, in2 => p1 => p2 => out1, out2
</Route>

We have the same error: 2016-07-23 13:36:09 ERROR cannot add processor module 'p1' to route '3' because it is already added to route '2'.

Why a processor is limited on one route ? Is it a bug or a mistake in documentation ?
I use the latest version of nxlog-ce : V2.9.1716.

Best regards

Hi,

I'm trying to fit output into the GELF format and I'd like to preserve their specification that user fields have underscore prefixes. From what I read in the nxlog docs, any fields with underscore prefix wouldn't be preserved by xm_json or xm_gelf

Is that true?

Is there any way around this?

My company is looking to setup NxLog. We are having issues reading in multiline exception logs from applications such as Tomcat, Java, Apache etc. I am able to read in the files but unfortunately the output in our GrayLog application is showing every event as one line. I tried to implement the xm_multiline module but i seem to be having issues getting it to work.

installed NxLog and checked my configuration to the following below. restarted the services, let the service run all night and still the output is the same as shown below.

Sample Input Log:

07/07/2016 13:35:11.654 [tomcat-http--43] [ERROR] [4114723 ms] Warning - unprocessed rows in esolutions.care.assess.WeAssessment
esolutions.EsolutionsException: There were 83 unprocessed rows out of 84
    at esolutions.base.WeObject.sleep(WeObject.java:2767)
    at esolutions.base.WeObject.clear(WeObject.java:3250)
    at esolutions.care.assess.WeAssessment.clear(WeAssessment.java:7699)
    at esolutions.base.WeObject.close(WeObject.java:2815)
    at esolutions.util.WeHTMLTable.getTableHTML(WeHTMLTable.java:541)
    at esolutions.util.WeHTMLTable.toHTML(WeHTMLTable.java:508)
    at org.apache.jsp.admin.client.cp_005fassessment_jsp._jspService(cp_005fassessment_jsp.java:4412)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)

07/07/2016 13:36:21.828 [tomcat-http--26] [ERROR] [4184897 ms] Warning - unprocessed rows in esolutions.care.assess.WeAssessment

esolutions.EsolutionsException: There were 82 unprocessed rows out of 83

    at esolutions.base.WeObject.sleep(WeObject.java:2767)
    at esolutions.base.WeObject.clear(WeObject.java:3250)
    at esolutions.care.assess.WeAssessment.clear(WeAssessment.java:7699)
    at esolutions.base.WeObject.close(WeObject.java:2815)
    at esolutions.util.WeHTMLTable.getTableHTML(WeHTMLTable.java:541)
    at esolutions.util.WeHTMLTable.toHTML(WeHTMLTable.java:508)
    at org.apache.jsp.admin.client.cp_005fassessment_jsp._jspService(cp_005fassessment_jsp.java:4412)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)

Sample Output From GrayLog in CSV format. The output in the webui is each event as it shows in the "message" column.

Configuration File. I tried multiple regular expressions with no success.

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/docs/

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module        xm_gelf
</Extension>

<Extension fileop>
    Module        xm_fileop
</Extension>

<Extension multiline>
    Module    xm_multiline
    HeaderLine    /^\d{0,2}\/\d{0,2}\/\d{0,4}/
#    HeaderLine    '^\d{0,2}\/\d{0,2}\/\d{0,4}\ \d{0,3}\:\d{0,3}\:\d{0,3}\.\d{0,4}\ \['
</Extension>

<Input pcc-wrapper-log>
    Module            im_file
    File            "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\wrapper.log"
    SavePos            TRUE
    InputType    multiline
</Input>
<Input pcc-mdstrace-log>
    Module            im_file
    File            "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\mdstrace.log"
    SavePos            TRUE
    InputType    multiline
</Input>
<Input pcc-exceptionHidingUtil-log>
    Module            im_file
    File            "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\exceptionHidingUtil.log"
    SavePos            TRUE
    InputType    multiline
</Input>
<Input pcc-esolutions-log>
    Module            im_file
    File            "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\esolutions.log"
    SavePos            TRUE
    InputType    multiline
</Input>
#<Input pcc-localHostAccess-log>
#    Module            im_file
#    File            "C:\\pivotal-tc-server-standard-3.1.0.RELEASE\\pccweb\\logs\\localhost_access_log.*"
#    SavePos            TRUE
#    InputType    multiline
#</Input>

<Output graylog>
    Module      om_udp
    Host        graylog.genesishcc.com
    Port        12201
    OutputType    GELF
</Output>

<Route PCC>
    Path        pcc-wrapper-log => pcc-mdstrace-log => pcc-exceptionHidingUtil-log => pcc-esolutions-log => graylog
##    Path        pcc-wrapper-log => pcc-mdstrace-log => pcc-exceptionHidingUtil-log => pcc-esolutions-log => pcc-localHostAccess-log => graylog
</Route>

Hello!

How me convert message HEX to ASCII using NxLog CE?

Hello

Help me please.

I want collect events from the database use Time-based (Not Id) 

What can i do?

Thank

Team,

As part of the assesment i need to provide list of open source items nxlog uses and its licensing information so our software legal team can do its assesment. Is there any link our documentation available which explains all the open source item we are currently using in nxlog.

Thanks,

Imran

Hi all

I would be glad to know, is it possible to use Nxlog community edition with ElasticSearch?

In the documentation I have read that the om_elasticsearch module is needed which exists in Enterprise Edition only.
Thanks in advanced.

How can I select some messages from a single source for 1 output and some for another based on the syslog content, I'm using community edition I have RTFMed but haven't found anything describing how to do this. I've tried using the Route block to send to multiple outputs and then using the drop() option in the output inside some <Exec> tags but it doesn't seem to work and I end up with the same stuff in both outputs.

Hi, I´m trying to add some info to my logs via xm_perl before send it to elasticsearch (using json format). As result, it would be nice to add some fileds from my perl code in nested way. Is it posible to use something like set_field_XXX($event, "myAddedfield.myAddedSubfield", "value")?

At the end, I want to create nested fields inside my json object.

Thx.

Hello,  I am testing nxlog to see if it works with sending security logs to our SIEM.  I only want to send the security Events on our servers, and have our config file as shown:

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
    Module xm_syslog
</Extension>

<Input internal>
    Module im_internal
</Input>

<Input eventlog>
    Module im_msvistalog
# Uncomment the following to collect specific event logs only
     Query <QueryList>\
               <Query Id="0">\
#                  <Select Path="Application">*</Select>\
#                  <Select Path="System">*</Select>\
                   <Select Path="Security">*</Select>\
               </Query>\
           </QueryList>  
</Input>

<Output out>
    Module      om_udp
    Host        10.250.254.19
    Port        514
    Exec        to_syslog_snare();
</Output>

<Route 1>
    Path        eventlog, internal => out
</Route>

I get some security logs, but many are missing, like logon/logoff events (4624, 4634) 

1. - why are these events missing   and

2.  Eventually, I would like to only send certain Event ID"s to our SIEM, and hope to get help with an example of what the query would look like with the specific Event ID's needed.

I will want to just send PCI Event ID's to our SEIM for retention.

I want to be able to take into account the version of the operating system (which is unknown at time of installation) in the configuration.

For example, I might want to output to a different server based on whether the installation is running on a server or on a workstation.

Thanks

I am attempimg to get nxlog installed on ubuntu. It appears that libperl5.18 is required but no longer available. Any suggestions?

sudo dpkg -i nxlog-ce_2.9.1504_ubuntu_1404_amd64.deb
[sudo] password for
(Reading database ... 59404 files and directories currently installed.)
Preparing to unpack nxlog-ce_2.9.1504_ubuntu_1404_amd64.deb ...
Unpacking nxlog-ce (2.9.1504) over (2.9.1504) ...
dpkg: dependency problems prevent configuration of nxlog-ce:
 nxlog-ce depends on libperl5.18 (>= 5.18.2); however:
  Package libperl5.18 is not installed.

sudo apt-get install libperl5.18
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package libperl5.18 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Linux ubuntunxlog 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Hello,
We're testing nxlog and at least a few times per day it's throwing this error. When this happens the error repeats itself in the logs for a few hours until either the log fills up the  partition or the kernel kills nxlog due to memory running out (Out of memory: Kill process 2422 (nxlog) score 762 or sacrifice child).

I'm not a C programmer but looking at the om_dbi driver source it seems to me as if this assertion might be checking if the length of the SQL query is longer than a predefined constant. Could this be the case? Is there a limit on the size of the SQL query nxlog can send? I think I might be wrong because I've hardcoded SQL queries I believe are longer than those triggering the assertion error and they went through just fine.

I came across a thread on the same error and the poster stated they resolved the issue by recompiling nxlog from source. I did that but unfortunately the latest available source is from an older version and couldn't get our nxlog.conf to work with it.

Any ideas what this assertion is checking?

BTW, NXLog is running on CentOS 6.8 and there's plenty of free memory (8 GB) and CPU power available.

Thank you,
Babak B. 

Hello

I need to support the gelf_tcp on my Gentoo servers, but on your website  the last source avaible is nxlog-ce-2.8.1248.tar.gz. This version does not support the gelf_tcp.
Where can I download the latest source of nxlog (at least NXLog-CE 2.9.1347) ?

Thanks so much