Hello,

I think this is an easy answer, but I am having some issues.  I am trying to read in a log file and send out a syslog message.  If the line in the file contains the word error, I want a syslog message to be sent.  If the file does not contain that word, just drop the message.  What would be the proper way to do this?  Below is what I currently have.

<Input watchfile_%service%>
  Module im_file
  File '%servicepath%'
  Exec $Message = $raw_event;
  Exec $SyslogSeverityValue = 6;
  Exec if file_name() =~ /.*\\(.*)/ $SourceName = $1; 
  Exec if $raw_event =~ /ERROR/ $SyslogSeverityValue = 3; 
  SavePos TRUE  
  Recursive TRUE
  PollInterval 10
</Input>

Thank You

Hi,

I am trying to read multiline xml files (results of ELMAH logs in ASP.NET), but for some of them I am unable to do so, as they contain lots of information. This results in errors like this:

2015-09-02 08:50:50 ERROR data size (119671) is over the limit (65000), will be truncated

I tried googling for a resolution to this problem, but I couldn't find any information anywhere about a data size limit in nxlog. Could some please advise what I could do to fix this?

Hello,

Recently, i ran into a problem with file locking. One application here is generating some log file, one per client, when nxlog try to read file, i got error :

ERROR failed to open LOGFILEPATH; Le processus ne peut pas accéder au fichier car ce fichier est utilisé par un autre processus.  

Initaly my configuration was :

<Input Orbis>
    Module im_file
    File 'LogDirectory\\*'
    SavePos TRUE
    ReadFromLast FALSE
    CloseWhenIdle TRUE
    InputType orbisLog
    Exec if $raw_event !~ /^L:.*/  drop();
    Exec $ClientNAme = replace(file_basename(file_name()),'.LOG','');

</Input>

I have changed the PollIntervall to 60 and 3600, no effect... Except that at PollInterval 3600, nxlog grab 40% of CPU.

Is there a way to insctruct nxlog to retry to read the file ?

Hi ,

I am trying to compile nxlog on solairs 11 sparc platform.

I am encoutering the following errors while compiling.

configure: error: libapr-1 not found

gcc: error: unrecognized command line option '-mt'

I used the following parameters :: ./configure APRCONFIG=/usr/apr/1.3/bin/apr-1-config

Please let me know if any one of you have successfully compiled it on Solaris 11.

Thanking you with regards.

Hello,

I'm quite new to nxlog, so forgive me if my question is  trivial but I'm having hard time to get the values I extract from my  logs using exec and a regex. I have a very large stash of old windows logs in text file (in multiline format), what I want to do is use nxlog to load them in graylog, but I want to format the log in a different way.

It works perfectly when I do not use the the exec and the regexp, but it fails with the message:

2015-08-31 12:12:42 ERROR invalid keyword: $timestamp at C:\Program Files (x86)\nxlog\conf\nxlog.conf:36

The regex works when I test it using http://www.regexr.com/

The error seems in the way I'm trying to assign/write the variables matched by the regex

My nxlog.conf is like the below:

## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally and is also available
## online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension gelf>
    Module       xm_gelf
</Extension>

<Extension multiline>
    Module      xm_multiline
    HeaderLine  /^{/
    EndLine     /^}/
</Extension>

<Input in>
        Module  im_file
        File    "C:\\tmp\\\\example-log.txt"
        SavePos  TRUE
        Recursive TRUE
        InputType       multiline
        exec if $raw_event =~ /(?:\{"([0-9]+?), ([0-9]+?), "(.+?)", "(.+?)", "(?:.+?)","(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)New\sLogon:\s*(?:.+?)\n\s*Account\sName:\s*(.+?)\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)Source\sNetwork\sAddress:\s*([0-9\.]{7,15})\n(?:[a-zA-Z0-9:,"\n\r\s\f\t\-\{\}\.\(\)]+?)"\})/g; 
     
    $timestamp = $1; 
    $event = $2; 
    $status = $3; 
    $type = $4; 
    $short = $5; 
    $user = $6; 
    $source = $7; 
    
</Input>

#<Output out>
#    Module      om_tcp
#    Host        192.168.1.15
#    Port        12201
#    OutputType  GELF_TCP
#</Output>

<Output out>
    Module    om_file
    File    'C:\\tmp\\output'
</Output>

<Route 1>
    Path        in => out
</Route>

 Could someone kindly help me out on this? I tried to read the nxlog manual but I need to confess that I could not understand what I'm doing wrong 

Basically I just would the output to be composed by the raw message and the few fields I match with the regex

Thanks very much!

Mark

I've got some data that comes in with a somewhat unusual format.  It's a set of fixed fields, followed by a variable length set of keys, followed by a set of values.  It looks something like this (but with more fields):

col1, col2, col3, description(key1; key2; ...;keyn), val1, val2, ..., valn

I'm trying to transform this into something more like:

a=col1, b=col2, c=col3, key1=val1, key2=val2, ..., keyn=valn

I've actually got this working by using Exec and a bit of perl that I wrote that tears apart $raw_event and writes the modified logline to a domain socket, where a second instance Route is listening and sends the log over the network to its destination.  My problem is that this is not terribly performant, since it starts a perl process per log line.  I've had trouble figuring out another way to do this, mostly because the number of keys/values is variable.

Any suggestions on ways this might be done that are likely to have better performance?

Hello there!

I am using nxlog for tailing specific files (im_file) and output them via GELF_TCP to Graylog which goes good except 1 thing > short_messages get truncated to 64 chars which, if I understand right, is related to this directive (or I might be wrong):

ShortMessageLength >This optional directive can be used to specify the length of the short_message field. This defaults to 64 if the directive is not explicitly specified. If the field short_message or ShortMessage is present, it will not be truncated.

The question might be fairly simple (to someone who knows): how to use this directive? could someone please write an example how to use it so short_messages do not get truncated?

Many thanks in advance!

P.S. Same problem here > https://groups.google.com/forum/#!topic/graylog2/wUQIaFdUlZs

I have a client who wants to use Loggly with NXLOG using UNC Paths to shared servers instead of absolute paths but has hit a road block.

Instead of using something like this: C:\files\logs\mylog.txt

He wants to do something like this: \\computername\windowsshare\c$\mylog.txt

When he has the nxlog service set to Local System and points to an absolute path to a local file it works fine.

But when he changes the service to be running as a Domain User and then sets his UNC path in the config file he gets an access denied error.

He has demonstrated that that Domain User has access to the file by logging in as that user and then opening the path to the file.

FYI, he has found that putting \\ in the config file causes problems but using \\\ seems to work.

Thanks.

-Bill

Greetings,

I'm trying to filter event viewer logs by the source name using the following configuration:

<Input EventLog>
    Module      im_msvistalog
    Query       <QueryList>\
                    <Query Id="0">\
                        <Select Path="Application">*</Select>\
                        <Select Path='System'>*[System[(SourceName="Service Control Manager")]]</Select>\
                    </Query>\
                </QueryList>
</Input>

However it's not working. When I try and filter by Event ID that works no problem. Any assistance would be much appreciated.

Hi,

I have several routes, outputs, and inputs and would like to have a schedule block that effects all logs. Is this possible? My current implementation is to put the schedule block in each Output. It works, but I was seeing if there was a way to clean it up and get it into one scheduler.

Thank you!

I'm having an alert every time the om_http sends a message to a web service. Has anyone ever experienced this?

LOG:
2015-08-19 16:45:18 INFO connecting to localhost:80
2015-08-19 16:45:18 WARNING http server disconnected while reading the response
2015-08-19 16:45:18 INFO reconnecting in 0 seconds

CONF:
<Output outATM>
    Module              om_http
    URL                 http://localhost:80/modules/AtmProcessorMT/index.php
</Output>

I am trying to rotate a log file in the Output module. Here is my configuration below:

Easier on the eyes (pastebin)

########################################

# Global directives                    #

########################################

User nxlog

Group nxlog



LogFile /var/log/nxlog/nxlog.log

LogLevel INFO

#SuppressRepeatingLogs FALSE



define WINLOG /logserv/collections/windows-collector/windows-collector-log.log



########################################

# Modules                              #

########################################

<Extension fileop>

        Module          xm_fileop

</Extension>



<Extension _syslog>

        Module          xm_syslog

</Extension>







<Input windows-collector-log>

        module          im_tcp

        host            0.0.0.0

        port            524

        Exec            parse_syslog();

        Exec            log_info("Severity Windows Collector: " + $SyslogSeverity + ", Hostname: " + $Hostname);

</Input>



<Output windows-collector-log-out>

        Module          om_file

        CreateDir    true

        File            '%WINLOG%'



     <Schedule>

                Every           30 sec

                Exec            if (file_size('%WINLOG%') >= 100M) file_cycle('%WINLOG%',500);

     </Schedule>

</Output>



<Route 5>

    Path            windows-collector-log => windows-collector-log-out

</Route>

I am getting an error saying that the file does not exist when the rotation is executed.

2015-08-19 13:22:23 ERROR failed to determine file size of '/logserv/collections/windows-collector/windows-collector-log.log': No such file or directory

Hi, 

I am writing here to inquire about nxLog for centralized logging implementation in production environment. 

We have tested nxLog community edition in development environment and We're very much interested in implementing nxLog in production(US/UK client) environment too. I'd appreciate, if someone provides me a detailed product terms and purchase/subscription cost incurred for purchasing nxLog community & enterprise edition.

I need to take a decision in the coming few days so it’s really very important that I receive this information as soon as possible. Awaiting reply. 

Best Regards,

Sreeram

Hi,

is it possible to configure at the om_tcp moule  a "closewhenidle". A persistent connection to my destination ist not good when no data sent. 

Hey all, 

I was wondering if rindex worked on nxlog, I’m not having luck getting it to work, and all my searches come up empty. I’m looking to parse a username after a "|" 

create_var('char'); \
set_var('char', '|'); \
create_var('index_num'); \
set_var('index_num', rindex($UserName, get_var('char'))); \

Hello,

Because of my lack of encryption knowledge, I search and found these instructions that I followed in order to create an SSL connection between an nxlog client (Windows server 2008 R2) and a graylog server.

So I transfered the "nxlog-ca.crt" to the client and indicated to the graylog server "nxlog-ca.key" as the TLS private key file and "nxlog-ca.crt" as the TLS cert file.

Here is the nxlog.conf :
define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension _syslog>
    Module      xm_gelf
</Extension>

<Input in>
    Module      im_msvistalog
        Query    <QueryList>\
            <Query Id="0">\
                <Select Path="Application">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
                <Select Path="Security">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
                <Select Path="System">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
                <Select Path="HardwareEvents">*[System[(Level=1  or Level=2 or Level=3)]]</Select>\
            </Query>\
        </QueryList>
</Input>

<Output sslout>
   Module          om_ssl
   Host            host_ip_address
   Port            12201
   CAFile          %CERTDIR%\nxlog-ca.crt
   OutputType      GELF_TCP
   AllowUntrusted  FALSE
</Output>

<Route 1>
    Path        in => sslout
</Route>

But when I launch "nxlog.exe -f" here is the error :

nxlog.exe -f

2015-08-04 12:23:05 INFO nxlog-ce-2.9.1347 started

2015-08-04 12:23:05 INFO connecting to host_ip_address:12201

2015-08-04 12:23:05 INFO successfully connect to host_ip_address:12201

2015-08-04 12:23:05 INFO remote socket was closed during SSL handshake
2015-08-04 12:23:05 INFO reconnecting in 1 seconds

And That's it. What am I missing ?

I read in the documentation that all the files about the certificates are in "pem" format but when I create it from certtool I have "crt" and "key" format files.

Thank you.

Hi,

I was wondering if anyone could help me please.

I want to use PowerShell to do a silent install of NxLog, I have tried different ways of using MSIEXEC in powershell and command line but I just can't seem to get it working.

Any advice would be great.

Thanks.

Sorry I am new to this. Where in the config file do you set it to pull from MSWinEventLog?

Thanks in advance!

We've had some clients where we install & deploy the MSI via group policy -- using the latest version 2.9.1347.

On Windows 10 -- it looks like the install succeeds (all the nx log files exist in Program Files (x86), etc.) -- but the service never gets installed. We don't see the nx log service anywhere in services.msc -- and don't see any logs saying that its install failed.

We've now seen this on multiple Windows 10 machines. Has anyone else -- and is there a workaround?

Hello

I write input module for nxlog. I have wrote function to read data but I don't know how to tranfer data further. Which function should I call? nx_logdata_set_string?