Hi guys,

I wanted to use NXLog to send data from a XML file into GrayLog.

My first test worked fine with a simple XML file. But with a Nessus Report (in XML) it don't work, because this file is more complexe. Do you have an example configuration nxlog.conf to help me ?

My actual nxlog.conf :

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
Module xm_gelf
</Extension>

<Extension multiline>

    Module	xm_multiline
    HeaderLine	/^<event>/
    EndLine	/^</event>/
</Extension>

<Extension xmlparser>
    Module	xm_xml
</Extension>

<Extension json>
    Module	xm_json
</Extension>

<Input in>
    Module	im_file
    File	"C:\Program Files\Graylog\collector-sidecar\generated\exempleFichier.xml"
    SavePos	FALSE
    ReadFromLast FALSE
    InputType	multiline
    <Exec>
      # Discard everything that doesn't seem to be an xml event   
      if $raw_event !~ /^<event>/ drop();

      # Parse the xml event
      parse_xml();

      # Rewrite some fields 
      $EventTime = parsedate($timestamp);
      delete($timestamp);
      delete($EventReceivedTime);

      # Convert to JSON
      to_json();
    </Exec>
</Input>

<Output out>
    Module      om_udp
    Host        xx.xx.xx.xx
    Port        12201
</Output>

<Route 1>
    Path        in => out
</Route>

And a (verry little) preview of my Nessus File...

<Report name="Scan_Nessus" xmlns:cm="http://www.nessus.org/cm">
<ReportHost name="192.168.1.1"><HostProperties>
<tag name="traceroute-hop-8">192.168.1.1</tag>
<tag name="LastUnauthenticatedResults">1490484150</tag>
<tag name="Credentialed_Scan">false</tag>
<tag name="policy-used">policies_scan</tag>
<tag name="patch-summary-total-cves">1</tag>
<tag name="os">other</tag>
<tag name="system-type">general-purpose</tag>
<tag name="operating-system">CISCO IOS</tag>
<tag name="traceroute-hop-7">?</tag>
<tag name="traceroute-hop-6">192.168.1.1</tag>
<tag name="traceroute-hop-5">192.168.1.2</tag>
<tag name="traceroute-hop-4">192.168.1.3</tag>
<tag name="HOST_END">Sun Mar 26 00:22:30 2017</tag>
<tag name="host-ip">192.168.1.1</tag>
<tag name="HOST_START">Sun Mar 26 00:20:19 2017</tag>
</HostProperties>
<ReportItem port="123" svc_name="ntp" protocol="udp" severity="0" pluginID="10884" pluginName="Network Time Protocol (NTP) Server Detection" pluginFamily="Service detection">
<cpe>cpe:/a:ntp:ntp</cpe>
<description>An NTP server with an insecure configuration is listening on port 123.
It provides information about its version, current date, current time, and possibly system information.</description>
<fname>ntp_open.nasl</fname>
<plugin_modification_date>2017/03/21</plugin_modification_date>
<plugin_name>Network Time Protocol (NTP) Server Detection</plugin_name>
<plugin_publication_date>2015/03/20</plugin_publication_date>
<plugin_type>remote</plugin_type>
<risk_factor>None</risk_factor>
<script_version>$Revision: 1.27 $</script_version>
<see_also>http://www.ntp.org</see_also>
<solution>n/a</solution>
<synopsis>An NTP server with an insecure configuration is listening on the remote host.</synopsis>
<plugin_output>
  Version : unknown
</plugin_output>
</ReportItem>

Thanks !

Seeing the following errors in the logs on a WIN 2008 R2 machine. Not 100% sure why but is seems to happen intermittently, I can't seem to corrolate it with log file rotations or anything like that. 

2017-03-31 12:46:52 ERROR if-else failed at line 10, character 39 in C:\Program Files (x86)\nxlog/conf/app_log/application_log_multiline_extensions.conf. statement execution has been aborted; binary operation failed at line 10, character 36 in C:\Program Files (x86)\nxlog/conf/app_log/application_log_multiline_extensions.conf. expression evaluation has been aborted; missing logdata, no field available in this context at line 10, character 19 in C:\Program Files (x86)\nxlog/conf/app_log/application_log_multiline_extensions.conf. operation possibly after drop()

hen it starts to appear it always happens at the exact same second of a given minute, but the number of minutes in between each erros seems somewhat random 

ie.

2017-03-31 10:37:42 ERROR if-else failed at line 16...

2017-03-31 10:40:42 ERROR if-else failed at line 16 ...

2017-03-31 10:41:42 ERROR if-else failed at line 16...

Config would be similar to below.

<Extension app_log_multiline>
    Module xm_multiline
    HeaderLine /.*\[\d\d \w\w\w \d\d\d\d/
    Exec if $raw_event =~ /^\s*$/ drop();
</Extension>

<Input in_app_log_files>
    Module im_file
    File '%APP_LOGS_DIR%\server_error.log'
    SavePos TRUE
    ReadFromLast TRUE 
    InputType app_log_multiline
    PollInterval 5
    DirCheckInterval 60
    CloseWhenIdle TRUE
    <Exec> 
        $Environment = "%ENVIRONMENT_TAG%"; 
        $Application = "%APP_TAG%"; 
        $Service = "%SERVICE_TAG%";
    </Exec>     
</Input>

OK, first cry for help. I've an install that I have inherited from a colleague - until four days ago I had never seen nxlog - and it has caused some performance issues.

The issue is that the are problems are occurring on VDIs on a cloud infrastructure. The cloud provider (a local operation, not one of the big boys like Amazon and Azure) says that they keep seeing machines suddenly taking large amounts of memory for periods of up to 5 minutes. This stops new processes being created (so you can't start task manager to ientify which process is hogging the memory, and in some cases it locks the VDI up so that the user has to be moved off it. The cloud providers are saying that this was not aproblem before nxlog was installed at the start of the week.

(However, the reason my colleague installed it is that the infrastructure has been attacked and there's at least some indicaion of compromise: so we are logging it to track any possible misuse) 

Am I right in thinking that adding a sleep statement to the conf file (which the documentation describes as a kind of poor man's rate limiting), passing the conf file out to all the instances on the individual VDIs, and then restarting services, should alleviate this? Or need I look elsewhere for the problem?

Like I said, please make allowances for n00b status..

J

Hello :),

I make a file rotation when my logs size exceed with 'file_cycle'.

I have something like this : 

    Exec    if outUDP->file_size() > 10k  \
            {  \
                log_info('size exceeded');  \
                file_cycle(%LOGDIR% + $programSource + '\\' + get_var('date') + '.log'); \
                outUDP->reopen();  \
            }

So my logs paths are :

C:\logs\programA\20170329.log
C:\logs\programA\20170329.log.1
C:\logs\programA\20170329.log.2
C:\logs\programB\20170329.log.1
C:\logs\programB\20170329.log.2

But I wanted something more like :

C:\logs\programA\20170329.log
C:\logs\programA\20170329.1.log
C:\logs\programA\20170329.2.log
C:\logs\programB\20170329.1.log
C:\logs\programB\20170329.2.log

I wanted to make it by myself using 'rotate_to' instead of 'file_cycle' .
The probleme is if nxlog restarts, I need it to continue the file rotation without overwrite the files that already exist. I can't check if C:\logs\programA\20170329.1.log exists then C:\logs\programA\20170329.2.log until I find the last...

It is possible to change it using 'file_cycle'? Or do you have another idea?

Thanks :)

I get this error:

MSI (s) (E4:9C) [19:13:39:684]: Product: NXLog-CE -- Error 1401. Could not create key: S-1-5-21-1688939787-1905098679-239004068-17022\Software\Microsoft.  System error 87.  Verify that you have sufficient access to that key, or contact your support personnel.

Error 1401. Could not create key: S-1-5-21-1688939787-1905098679-239004068-17022\Software\Microsoft.  System error 87.  Verify that you have sufficient access to that key, or contact your support personnel.

when using this method as part of a script...

$process = ([WMICLASS]"\\$server\ROOT\CIMV2:Win32_Process").Create("cmd.exe /c " + $remoteProcess1)

The command issued is...

msiexec.exe /i nxlog-ce.msi /q

This method works great for other MSI packages

Any one know why this might be happening?  I ensured that the account creating the remote process has administrative rights.

Why would this need to access the HKEY_Users hive?

I also noticed that if I load the same User Profile on the remote machine that I am using to start the remote process, the installation works fine.

Once that is done no other Administrator can remove the NXLog agent.  I try using the uninstall string and it fails.  THe only way to remove NXLog is to log on as the same user and uninstall.

Where can I find the documenation for this om_redis module?

Only resource I can find is: https://nxlog.co/question/1593/redis-module-lpush

Hi All,

I have previously got smtp logs to go into Graylog using NXlog, it was worknig fine. I then had a disk sapce issue on the graylog host so had to redo some bits, including the nxlog.conf for our SMTP server. 

The SMTP log header specifes the following

#Software: Microsoft Internet Information Services 8.5
#Version: 1.0
#Date: 2017-03-26 23:00:10
#Fields: date time c-ip cs-username s-sitename s-computername s-port cs-method cs-uri-query sc-bytes time-taken 

This is the error I am getting

ERROR if-else failed at line 44, character 436 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; procedure 'parse_csv' failed at line 44, character 224 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; Not enough fields in CSV input, expected 11, got 8 in input

I ahve checked and rechecked and their should be 11 items as per the .conf.

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module         xm_gelf
</Extension>

<Extension fileop>
    Module         xm_fileop
</Extension>

 <Extension json>
    Module      xm_json
</Extension>

# Create the parse rule for IIS logs. You can copy these from the header of the IIS log file.
<Extension w3c>
    Module             xm_csv
    Fields             $date, $time, $c-ip, $cs-username, $s-sitename, $s-computername, $s-port, $cs-method, $cs-uri-query, $sc-bytes, $time-taken
    FieldTypes         string, string, string, string, string, string, integer, string, string, integer, integer
    Delimiter         ' '
    QuoteChar         '"'
    EscapeControl     FALSE
    UndefValue         '-'
</Extension>

<Input smtp>
    Module        im_file
    File        "C:\\Logs\\SMTPSVC1\\\ex*.log"
    SavePos      TRUE

    Exec        if $raw_event =~ /^#/ drop();                    \
                else                                             \
                {                                                \
                    w3c->parse_csv();                            \
                    $EventTime = parsedate($date + " " + $time); \
                    $SourceName = "smtp";                         \
                    $Message = to_json();                         \
                }
</Input>

<Input eventlog>
    Module      im_msvistalog
</Input>

<Output graylog>
    Module      om_udp
    Host        graylog.mydomain.com
    Port        12201
    OutputType    GELF

    #Use the following line for debugging (uncomment the fileop extension above as well)
    #Exec file_write("C:\\Program Files (x86)\\nxlog\\data\\nxlog_output.log", $raw_event);
</Output>

<Route eventlog>
   Path        eventlog => graylog
</Route>

<Route smtp-to-graylog>
    Path        smtp => graylog
</Route>

Its so frustracting that I know this was working correctly.

Any help would be great.

Thanks

I'm struggling with rotating syslog files at midnight so that they are named "YYYY-MM-DD.log" (and contain log records for that date).

I wonder if anyone has an example of the best way to achieve this.  The manual could use an example like this.

Thanks

Phil

I'm attempting to use NXlog to perform a one time read of a CSV file which will then be passed to Fluent-D for processing and writing to a MySQL database.

I've read through the documentation for setting up the nxlog.config file but I'm at a loss to how to configure for my CSV file.

Has anyone has a config file that reads a CSV I'd apreciate the help.

Thanks. 

Hello.

While reading log from windows, I got string with two encodings WIN-1251 and UTF-8:

like 2017-02-21 16:40:24 IT-73.domain.name INFO 44 NT AUTHORITY\҈Ғƌ�Центр обновления Windows начал скачивать обновление.

where all message in utf-8, and part with AccountName that (usually NT AUTHORITY\SYSTEM in english version of Windows 7) but in russian version its NT AUTHORITY\СИСТЕМА and in logs it looks like NT AUTHORITY\҈Ғƌ�

If I use 

convert_fields("AUTO", "utf-8")

I got proper AccountName, but all other parts got wrong encoding.

Is where any way to replace that part with correct encoding?

Hello,

nxlog CE v2.9.1504, Windows Server 2008 Enterprise

relevant part of config file:

<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
    Module im_msvistalog
    Exec if ($Severity == 'INFO') drop();
<QueryXML>
    <QueryList>
        <Query Id="0">
            <Select Path="System">*</Select>
            <Select Path="Security">*</Select>
            <Select Path="Application">*</Select>
<!-- EventID 2137 - Shrepoint Health Analyzer - ignore -->
            <Suppress Path="Application">*[System[(EventID=2137)]]</Suppress>
<!-- EventID 2138 - Shrepoint Health Analyzer - ignore -->
            <Suppress Path="Application">*[System[(EventID=2138)]]</Suppress>
            <Select Path="Microsoft-Windows-TaskScheduler/Operational">*</Select>
        </Query>
    </QueryList>
</QueryXML>
# Uncomment im_mseventlog for Windows XP/2000/2003
#   Module im_mseventlog
</Input>

One event (EventID 1309 from Application channel) always returns an empty message field (message:null)  as you can see in debug output.

{"EventTime":"2017-03-21 09:54:13","Hostname":"HOST.DOMAIN.TLD","Keywords":36028797018963968,
"EventType":"WARNING","SeverityValue":3,"Severity":"WARNING","EventID":1309,"SourceName":"ASP.NET 2.0.50727.0",
"Task":3,"RecordNumber":1013344,"ProcessID":0,"ThreadID":0,"Channel":"Application","ERROR_EVT_UNRESOLVED":true,
"Category":"Web Event","EventReceivedTime":"2017-03-21 09:54:13","SourceModuleName":"eventlog","SourceModuleType":"im_msvistalog",
"message":null}

How can I get more informations, why those events has a null message field?
What means "ERROR_EVT_UNRESOLVED":true?

Thank you.

hello:

I find some problem under AIX6.1 system.  

the log:

2017-03-21 14:02:41 WARNING additional group memberships couldn't be set because getgrouplist()and setgroups() are not available on this platform
2017-03-21 14:02:41 INFO nxlog-ce-2.8.1248 started
2017-03-21 14:02:41 ERROR failed to open /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied
2017-03-21 14:02:43 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied
2017-03-21 14:02:49 ERROR last message repeated 2 times
2017-03-21 14:02:57 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied
2017-03-21 14:03:13 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied
2017-03-21 14:03:45 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied
2017-03-21 14:04:50 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/testdb/testdb/trace/alert_testdb.log;Permission denied

and i sccan the source code, find:

static void nxlog_set_groups(nxlog_t *nxlog, apr_uid_t uid)
{
    gid_t grplist[100];
    int ngroups = sizeof(grplist);
    char *user;
    apr_status_t rv;

    if ( (rv = apr_uid_name_get(&user, uid, nxlog->pool)) != APR_SUCCESS )
    {
    log_aprerror(rv, "couldn't resolve uid %d to name", uid);
    return;
    }
#ifdef HAVE_GETGROUPLIST 
# ifdef HAVE_SETGROUPS
    if ( getgrouplist(user, getgid(), grplist, &ngroups) == -1 )
    {
    log_error("couldn't get group membership for user %s (uid: %d), too many groups?", user, uid);
    return;
    }
    if ( setgroups((size_t) ngroups, grplist) != 0 )
    {
    log_errno("couldn't get group membership for user %s (uid: %d), setgroups() failed",
          user, uid);
    return;
    }
# else
    log_warn("additional group memberships couldn't be set because getgrouplist()"
         "and setgroups() are not available on this platform");
# endif
#else
    log_warn("additional group memberships couldn't be set because getgrouplist()"
         "and setgroups() are not available on this platform");
#endif

}

I don't what' wrong with this problem, thanks.

Hi All,

Checking to see if anyone has run into this.  I have a windows eventlog collector, with a subscription setup to move specific security audit events to the "Forwarded Events" log.  From there, I am looking to push those logs to Sumologic.  Unfortunately Sumo's collector does not handle this well due to the out of sequence EventRecordID of the various events coming from multiple desktops/servers we're collecting from.

Question:

I'm trying to take advantage of Sumo's native Windows eventlog parser, however the options for sending the eventlog data using NXlog send in the specific formats, syslog_snare, xml, json, etc.  Is there a configuration i can use which send the messages as windows eventlog format?  You'll see from my config below, I've tried several formats, to no avail.  Any suggestions would be greatly appreciated.

<Input eventlog>
    Module      im_msvistalog
<QueryXML>
   <QueryList>                    
     <Query Id="0"> 
        <Select Path="ForwardedEvents">*</Select>
     </Query>
   </QueryList>
</QueryXML>
</Input>

<Output out>
    Module      om_tcp
    Host        10.x.x.x
    Port        514
#    Exec       to_xml();
       Exec to_syslog_snare();
#    Exec $raw_event = replace($raw_event, "\r\n"," ");
#    Exec $raw_event = replace($raw_event, "\t", " ");
#    Exec    $raw_event();
</Output>

<Route 1>
# Path in => out
 Path eventlog, internal => out
</Route>

Thanks in advance,

-A

Hello All,

I am new to nxlog or rather logics in terms of regex and all. 

I am looking to drop any message which has *.*.*.255 in message field. I tried below , however it does not seem to be working for me:

Exec if $Message =~ /^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . ^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . ^([01][0-9][0-9]|2[0-4][0-9]|25[0-5]) . 255/  drop();

Please suggest

config file:

## This is a sample configuration file. See the nxlog reference manual about the

## configuration options. It should be installed locally under

## /usr/share/doc/nxlog-ce/ and is also available online at

## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

########################################

# Global directives                    #

########################################

User nxlog

Group nxlog

LogFile /var/log/nxlog/nxlog.log

LogLevel INFO

########################################

# Modules                              #

########################################

<Extension _syslog>

    Module      xm_syslog

</Extension>

<Extension multiline>

 Module xm_multiline

 HeaderLine /^\w{3}\s\w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\d{4}/

</Extension>

<Input oerrorin1>

 Module    im_file

 File     '/u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log'

 InputType multiline

 SavePos TRUE

 ReadFromLast  FALSE

</Input>

<Output udpout>

 Module    om_udp

 Host   10.1.227.45

 Port   514

</Output>s

########################################

# Routes                               #

########################################

<Route 1>

    Path        oerrorin1 =>udpout

</Route>

then i get the log:

2017-03-17 16:16:39 INFO nxlog-ce-2.8.1248 started

2017-03-17 16:16:39 ERROR failed to open /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:16:41 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:16:47 ERROR last message repeated 2 times

2017-03-17 16:16:55 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:17:11 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:17:43 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:18:47 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:20:55 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:25:11 ERROR apr_stat failed on file /u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log;Permission denied

2017-03-17 16:34:16 DEBUG reading config cache from /var/spool/nxlog/configcache.dat

2017-03-17 16:34:16 DEBUG Setting up module '_syslog' using xm_syslog

2017-03-17 16:34:16 DEBUG module _syslog has 4 exported functions

2017-03-17 16:34:16 DEBUG registering function syslog_facility_value

2017-03-17 16:34:16 DEBUG function 'syslog_facility_value' registered

2017-03-17 16:34:16 DEBUG registering function syslog_facility_string

2017-03-17 16:34:16 DEBUG function 'syslog_facility_string' registered

2017-03-17 16:34:16 DEBUG registering function syslog_severity_value

2017-03-17 16:34:16 DEBUG function 'syslog_severity_value' registered

2017-03-17 16:34:16 DEBUG registering function syslog_severity_string

2017-03-17 16:34:16 DEBUG function 'syslog_severity_string' registered

2017-03-17 16:34:16 DEBUG module _syslog has 9 exported procedures

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog' registered

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog' registered

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_bsd

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_bsd' registered

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_bsd

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_bsd' registered

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_ietf

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_ietf' registered

2017-03-17 16:34:16 DEBUG registering procedure parse_syslog_ietf

2017-03-17 16:34:16 DEBUG procedure 'parse_syslog_ietf' registered

2017-03-17 16:34:16 DEBUG registering procedure to_syslog_bsd

2017-03-17 16:34:16 DEBUG procedure 'to_syslog_bsd' registered

2017-03-17 16:34:16 DEBUG registering procedure to_syslog_ietf

2017-03-17 16:34:16 DEBUG procedure 'to_syslog_ietf' registered

2017-03-17 16:34:16 DEBUG registering procedure to_syslog_snare

2017-03-17 16:34:16 DEBUG procedure 'to_syslog_snare' registered

2017-03-17 16:34:16 DEBUG Setting up module 'multiline' using xm_multiline

2017-03-17 16:34:16 DEBUG Setting up module 'oerrorin1' using im_file

2017-03-17 16:34:16 DEBUG module oerrorin1 has 1 exported functions

2017-03-17 16:34:16 DEBUG registering function file_name

2017-03-17 16:34:16 DEBUG function 'file_name' registered

2017-03-17 16:34:16 DEBUG module oerrorin1 has 0 exported procedures

2017-03-17 16:34:16 DEBUG FlowControl enabled for oerrorin1

2017-03-17 16:34:16 DEBUG Setting up module 'udpout' using om_udp

2017-03-17 16:34:16 DEBUG Setting up module 'fileout2' using om_file

2017-03-17 16:34:16 DEBUG module fileout2 has 2 exported functions

2017-03-17 16:34:16 DEBUG registering function file_name

2017-03-17 16:34:16 DEBUG function 'file_name' registered

2017-03-17 16:34:16 DEBUG registering function file_size

2017-03-17 16:34:16 DEBUG function 'file_size' registered

2017-03-17 16:34:16 DEBUG module fileout2 has 2 exported procedures

2017-03-17 16:34:16 DEBUG registering procedure rotate_to

2017-03-17 16:34:16 DEBUG procedure 'rotate_to' registered

2017-03-17 16:34:16 DEBUG registering procedure reopen

2017-03-17 16:34:16 DEBUG procedure 'reopen' registered

2017-03-17 16:34:16 DEBUG CONFIG: _syslog

2017-03-17 16:34:16 DEBUG inputreader 'Syslog_TLS' registered

2017-03-17 16:34:16 DEBUG Inputreader 'Syslog_TLS' registered

2017-03-17 16:34:16 DEBUG outputwriter 'Syslog_TLS' registered

2017-03-17 16:34:16 DEBUG Outputwriter 'Syslog_TLS' registered

2017-03-17 16:34:16 DEBUG CONFIG: multiline

2017-03-17 16:34:16 DEBUG inputreader 'multiline' registered

2017-03-17 16:34:16 DEBUG Inputreader 'multiline' registered

2017-03-17 16:34:16 DEBUG CONFIG: oerrorin1

2017-03-17 16:34:16 DEBUG adding string [/u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log]

2017-03-17 16:34:16 DEBUG string literal declared at line 29, character 86 in /etc/nxlog.conf

2017-03-17 16:34:16 DEBUG literal

2017-03-17 16:34:16 DEBUG parsed expression

2017-03-17 16:34:16 DEBUG CONFIG: udpout

2017-03-17 16:34:16 DEBUG CONFIG: fileout2

2017-03-17 16:34:16 DEBUG adding string [/var/log/logmsg2.txt]

2017-03-17 16:34:16 DEBUG string literal declared at line 43, character 27 in /etc/nxlog.conf

2017-03-17 16:34:16 DEBUG literal

2017-03-17 16:34:16 DEBUG parsed expression

2017-03-17 16:34:16 DEBUG pidfile /var/run/nxlog/nxlog.pid removed

2017-03-17 16:34:16 DEBUG daemonizing...

2017-03-17 16:34:16 DEBUG INIT: _syslog

2017-03-17 16:34:16 DEBUG INIT: multiline

2017-03-17 16:34:16 DEBUG INIT: oerrorin1

2017-03-17 16:34:16 DEBUG INIT: udpout

2017-03-17 16:34:16 DEBUG Pollset initialized for module udpout (method: poll)

2017-03-17 16:34:16 DEBUG INIT: fileout2

2017-03-17 16:34:16 DEBUG Pollset initialized for module fileout2 (method: poll)

2017-03-17 16:34:16 DEBUG now running as group nxlog

2017-03-17 16:34:16 WARNING additional group memberships couldn't be set because getgrouplist()and setgroups() are not available on this platform

2017-03-17 16:34:16 DEBUG now running as user nxlog

2017-03-17 16:34:16 DEBUG running as uid: 203, euid: 203, gid: 204, egid: 204

2017-03-17 16:34:16 DEBUG pidfile /var/run/nxlog/nxlog.pid created

2017-03-17 16:34:16 DEBUG parsing path: oerrorin1 =>udpout

2017-03-17 16:34:16 DEBUG adding module oerrorin1 to route 1

2017-03-17 16:34:16 DEBUG adding module udpout to route 1

2017-03-17 16:34:16 DEBUG parsing path: in2 => fileout2

2017-03-17 16:34:16 ERROR [router.c:68/nx_route_add_module()] module 'in2' is not declared at /etc/nxlog.conf:54

2017-03-17 16:34:16 DEBUG adding module fileout2 to route tcproute

2017-03-17 16:34:16 ERROR [router.c:347/nx_add_route()] route tcproute is not functional without input modules, ignored at /etc/nxlog.conf:54

2017-03-17 16:34:16 DEBUG jobgroup created with priority 99

2017-03-17 16:34:16 DEBUG jobgroup created with priority 10

2017-03-17 16:34:16 DEBUG spawning 3 worker threads

2017-03-17 16:34:16 DEBUG worker thread 0 started

2017-03-17 16:34:16 DEBUG worker thread 1 started

2017-03-17 16:34:16 DEBUG worker thread 2 started

I have no idea about this problem. 

/u01/app/oracle/diag/rdbms/oraywsoudb/oraywsoudb/trace/alert_oraywsoudb.log   --> user and group is oracle:oinstall

and i add the nxlog user in oinstall group， but why continue report Permission denied?

Hi there!

I am a college student...i am working on windows event logger to collect & analyse windows logs(event & syslog).

I don't know how to configure nxlog.conf file such that ... i can get all windows generated logs locally on host machine.

Waiting for reply.

Thank You

have recently been trying the Community Edition of nxlog which does exactly what I need (and more) but have an issue with the msi. In order to be something I could deploy it has to pass uninstall and re-install tests. Unfortunatley the uninstall via Windows installer leaves a lot of file and registry entries behind that subsequently causes issues with a re-installation. Does anyone know if this is likely to be addressed within CE? Does anyone know if Enterprise edition has the same msi issue? Thanks. 

Hello All,

I am using nxlog-ce-2.9.1716 client to collect logs from my device and then forward it to logstash. I want to make sure nxlog client accept syslogs from particular source instead any. However when using specific IP in im_udp module, I am getting error as below:

"2017-03-07 16:04:51 ERROR failed to start im_udp; couldn't bind udp socket to 172.20.20.20:514; The requested address is not valid in its context. " 

my input driver is configured as below:

<Input in>
    Module    im_udp
    Host    172.20.20.20
    Port    514
</Input>

I tried by changing host as below as well:

Host 172.20.20.20/32 - No luck

Host 172.20.20.20/255.255.255.0 - No luck

Host 172.20.20.20/255.255.255.255 - No luck

Host 172.20.20.20 255.255.255.255 - no luck

Host 172.20.20.20 255.255.255.0 - No luck

However if I make Host 0.0.0.0 - it works , but this is not what I am looking for. 

Please help if I am missing anything. 

Regards,

Gaurang 

Hi!

I'm new to NXLOG so sorry if this is a dumb question.

So I have a Windows 2012 server that I'm attempting to set up to accept Syslog messages from an outsourced proxy system. I've been able to get NXLOG accept the logs and dump them to a flat file which our SIEM tool can pick up. However it ends up being a gigantic file... 

I'm trying to refine this now.

Instead of a huge file that we purge out every day, I'd like to have NXLOG limit the log dump to something like 200MB, copy the now older 200MB file to a different directory appending a time stamp to it, then start collecting in a new file in the original folder. The catch is, the live log that is in the original folder needs to keep the same file name, otherwise our SIEM tool won't know what to look for.

I've tried to have a PowerShell script do this and it works, however, if the NXLOG service is running, the log dump file is locked. We could get around this by disabling the service while copying the old log file to a new location, however we'd lose 10-30 minutes of logs in the file copy (not ideal).

Going through the reference manual I see a few ways to limit file size, starting a new file, and copying files over directly in the NXLOG config file. However it's turning out to be a problem condencing all of that into one configuration file that works on Windows. I've only been able to get a 1 of 3 pieces working at one time.

I appreciate any help the community can provide! Let me know if there are any questions or if something isn't clear.

​

What is the most efficient way to parse Microsoft DNS Server debug logs into something more tidy, say into a CSV or KVP format on the nxlog agent?

Consider the following log message:

"24/02/2017 16:37:22 09B0 PACKET  0000009657E7BA40 UDP Rcv 10.0.100.15   a490   Q [0001   D   NOERROR] A      (7)example(3)com(0)"

First of all, what would be the most efficient way performance-wise to convert this into a CSV or KVP format?

And also, is there some other way besides using Exec and replacing parenthesis and numbers in a sed-like manner to get the clean query name? We have tried to use the Exec method before, but we were hitting some serious performance issues on busy DNS servers.

I have currently switched on to using the pm_pattern module to drop invalid log lines (the beginning of the log file and empty lines) and I was wondering if there would be some easy way to perform both of the tasks (the formatting and the cleaning) using the pm_pattern module?

An example output could look something like the following:

datetime=24/02/2017 16:37:22,thread_id=09B0,context=PACKET,packet_id=0000009657E7BA40,protocol=UDP,action=Rcv,remote_ip=10.0.100.15,
xid=a490,event_type=-,opcode=Q,flags_hex=0001,is_authorative=-,is_truncated=-,recursion_desired=D,recursion_available=-,
response_code=NOERROR,question_type=A,question_name=example.com

The empty or "-" values result from fields specified in the DNS debug log format that are not present in the above message (e.g. all possible flags would be "ATDR", and event_type is "-" because "R" marks a response but an empty value (whitespace) marks a query.

And of course, if the above even is possible, would it be too resource consuming?