How can I select some messages from a single source for 1 output and some for another based on the syslog content, I'm using community edition I have RTFMed but haven't found anything describing how to do this. I've tried using the Route block to send to multiple outputs and then using the drop() option in the output inside some <Exec> tags but it doesn't seem to work and I end up with the same stuff in both outputs.

Hi, I´m trying to add some info to my logs via xm_perl before send it to elasticsearch (using json format). As result, it would be nice to add some fileds from my perl code in nested way. Is it posible to use something like set_field_XXX($event, "myAddedfield.myAddedSubfield", "value")?   At the end, I want to create nested fields inside my json object.   Thx.

Hello,  I am testing nxlog to see if it works with sending security logs to our SIEM.  I only want to send the security Events on our servers, and have our config file as shown: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog>     Module xm_syslog </Extension> <Input internal>     Module im_internal </Input> <Input eventlog>     Module im_msvistalog # Uncomment the following to collect specific event logs only      Query <QueryList>\                <Query Id="0">\ #                  <Select Path="Application">*</Select>\ #                  <Select Path="System">*</Select>\                    <Select Path="Security">*</Select>\                </Query>\            </QueryList>   </Input> <Output out>     Module      om_udp     Host        10.250.254.19     Port        514     Exec        to_syslog_snare(); </Output> <Route 1>     Path        eventlog, internal => out </Route>   I get some security logs, but many are missing, like logon/logoff events (4624, 4634)  1. - why are these events missing   and 2.  Eventually, I would like to only send certain Event ID"s to our SIEM, and hope to get help with an example of what the query would look like with the specific Event ID's needed. I will want to just send PCI Event ID's to our SEIM for retention.

I want to be able to take into account the version of the operating system (which is unknown at time of installation) in the configuration. For example, I might want to output to a different server based on whether the installation is running on a server or on a workstation.   Thanks

I am attempimg to get nxlog installed on ubuntu. It appears that libperl5.18 is required but no longer available. Any suggestions? sudo dpkg -i nxlog-ce_2.9.1504_ubuntu_1404_amd64.deb [sudo] password for (Reading database ... 59404 files and directories currently installed.) Preparing to unpack nxlog-ce_2.9.1504_ubuntu_1404_amd64.deb ... Unpacking nxlog-ce (2.9.1504) over (2.9.1504) ... dpkg: dependency problems prevent configuration of nxlog-ce:  nxlog-ce depends on libperl5.18 (>= 5.18.2); however:   Package libperl5.18 is not installed. sudo apt-get install libperl5.18 Reading package lists... Done Building dependency tree        Reading state information... Done Package libperl5.18 is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source Linux ubuntunxlog 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Hello, We're testing nxlog and at least a few times per day it's throwing this error. When this happens the error repeats itself in the logs for a few hours until either the log fills up the  partition or the kernel kills nxlog due to memory running out (Out of memory: Kill process 2422 (nxlog) score 762 or sacrifice child). I'm not a C programmer but looking at the om_dbi driver source it seems to me as if this assertion might be checking if the length of the SQL query is longer than a predefined constant. Could this be the case? Is there a limit on the size of the SQL query nxlog can send? I think I might be wrong because I've hardcoded SQL queries I believe are longer than those triggering the assertion error and they went through just fine. I came across a thread on the same error and the poster stated they resolved the issue by recompiling nxlog from source. I did that but unfortunately the latest available source is from an older version and couldn't get our nxlog.conf to work with it. Any ideas what this assertion is checking? BTW, NXLog is running on CentOS 6.8 and there's plenty of free memory (8 GB) and CPU power available. Thank you, Babak B. 

Hello I need to support the gelf_tcp on my Gentoo servers, but on your website  the last source avaible is nxlog-ce-2.8.1248.tar.gz. This version does not support the gelf_tcp. Where can I download the latest source of nxlog (at least NXLog-CE 2.9.1347) ? Thanks so much

Hi everyone, I’ve configured an Windows EventLog collection server and setup a handful of custom eventlog channels per the following article. https://blogs.technet.microsoft.com/russellt/2016/05/18/creating-custom-windows-event-forwarding-logs/ My custom event log channels are receiving the correct logs, and everything is working as expected as far as event collection goes. http://i133.photobucket.com/albums/q54/1point3liter/misc/WEC_zpsscp5bw2s.png I'm now trying to configure nxlog to pick up the event logs from my custom channels and forward them to a syslog server, but it doesn’t seem to be working. nxlog does forward if I query the built in "Security" channel, but not from my custom channels (or even "forwarded events). Any ideas? Bryan   Here is a copy of my NXlog configuration file: define ROOT C:\Program Files (x86)\nxlog define ROOT_STRING C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _syslog>   Module xm_syslog </Extension> <Extension json>   Module xm_json </Extension> #<Input eventlog> #  Module im_msvistalog #  SavePos TRUE #  #Query <QueryList><Query Id="0"><Select Path="_ApplicationServers">*</Select></Query></QueryList> #  #Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json(); #  Exec $Message = to_json(); #</Input> <Input eventlog>     Module  im_msvistalog     Query   <QueryList>\             <Query Id="0">\             <Select Path="WEC/AllServers">*</Select>\             </Query>\             </QueryList>     </Input> <Output out>  Module om_tcp  Host x.x.x.x  Port 514 </Output> <Route 1>  Path internal, eventlog => out </Route>  

Hello,  I'm trying to convert a date in NXlog from 06/15/16 to 2016-06-15 because NXlog is not able to parse the date (DEBUG couldn't parse date: 06/14/16). I created a regular expression ($Date =~ s/(\d+)\/(\d+)\/(\d+)/20$3-$2-$1/;) in my module to convert the date. See the module below  Exec if $raw_event =~ /^[0-9][0-9],/                        \         {                                                        \             ParseDHCP->parse_csv();                                 \             if $raw_event =~ /^00/ $IDdef = "The log was started.";    \             if $raw_event =~ /^01/ $IDdef = "The log was stopped.";    \             if $raw_event =~ /^02/ $IDdef = "The log was temporarily paused due to low disk space.";    \             if $raw_event =~ /^10/ $IDdef = "A new IP address was leased to a client.";    \             if $raw_event =~ /^11/ $IDdef = "A lease was renewed by a client.";    \             if $raw_event =~ /^12/ $IDdef = "A lease was released by a client.";    \             if $raw_event =~ /^13/ $IDdef = "An IP address was found to be in use on the network.";    \             if $raw_event =~ /^14/ $IDdef = "A lease request could not be satisfied because the scope's address pool was exhausted.";    \             if $raw_event =~ /^15/ $IDdef = "A lease was denied.";    \             if $raw_event =~ /^16/ $IDdef = "A lease was deleted.";    \             if $raw_event =~ /^17/ $IDdef = "A lease was expired and DNS records for an expired leases have not been deleted.";    \             if $raw_event =~ /^18/ $IDdef = "A lease was expired and DNS records were deleted.";    \             if $raw_event =~ /^20/ $IDdef = "A BOOTP address was leased to a client.";    \             if $raw_event =~ /^21/ $IDdef = "A dynamic BOOTP address was leased to a client.";    \             if $raw_event =~ /^22/ $IDdef = "A BOOTP request could not be satisfied because the scope's address pool for BOOTP was exhausted.";    \             if $raw_event =~ /^23/ $IDdef = "A BOOTP IP address was deleted after checking to see it was not in use.";    \             if $raw_event =~ /^24/ $IDdef = "IP address cleanup operation has began.";    \             if $raw_event =~ /^25/ $IDdef = "IP address cleanup statistics.";    \             if $raw_event =~ /^30/ $IDdef = "DNS update request to the named DNS server.";    \             if $raw_event =~ /^31/ $IDdef = "DNS update failed.";    \             if $raw_event =~ /^32/ $IDdef = "DNS update successful.";    \             if $raw_event =~ /^33/ $IDdef = "Packet dropped due to NAP policy.";    \             if $raw_event =~ /^34/ $IDdef = "DNS update request failed.as the DNS update request queue limit exceeded.";    \             if $raw_event =~ /^35/ $IDdef = "DNS update request failed.";    \             if $raw_event =~ /^36/ $IDdef = "Packet dropped because the server is in failover standby role or the hash of the client ID does not match.";    \             if $raw_event =~ /^[5-9][0-9]/ $IDdef = "Codes above 50 are used for Rogue Server Detection information.";    \             if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,0,/ $QResultDef = "NoQuarantine";    \             if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,1,/ $QResultDef = "Quarantine";    \             if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,2,/ $QResultDef = "Drop Packet";    \             if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,3,/ $QResultDef = "Probation";    \             if $raw_event =~ /^.+,.+,.+,.+,.+,.+,.+,.+,6,/ $QResultDef = "No Quarantine Information ProbationTime:Year-Month-Day Hour:Minute:Second:MilliSecond.";    \             $host            =    hostname_fqdn();                \             $Date =~ s/(\d+)\/(\d+)\/(\d+)/20$3-$2-$1/;   \             $EventTime         =     parsedate($Date + " " + $Time);    \             $SourceName     =     "DHCPEvents";                    \             $Message         =     to_json();                        \         }                                                        \         else                                                    \             drop(); However it returns 2016-06-15 17:37:29 INFO EventTime: 20$3-$2-$1  

Hi, I use multiple input files. I wish to log periodically (every minute) in nxlog.log the filename of the current input file to control all process chain, and add a Exec log_info("Current InputFile  : " + $InputFileName1); in a Schedule Block in Output section. But it seems that it doesn’t work in a Schedule block => error on nxlog.log => “…field not available in this context…” My config : <Input in>  Module  im_file  SavePos TRUE  ReadFromLast FALSE  ActiveFiles 20  CloseWhenIdle TRUE  File "/var/log/MUP10/sac/APMUZS4WBS04*.log"  Exec $InputFileName1 = file_name(); </Input> <Output logstash>  Module om_tcp  Port 6002  Host 10.x.y.z  Exec        create_stat("stat", "RATE", 60); add_stat("stat", 1);     <Schedule>         Every   60 sec         Exec    log_info("Events send to logstash for the last minute: " + get_stat("stat"));         Exec log_info("Current InputFile  : " + $InputFileName1);     </Schedule> </Output> If I move my Exec log_info("Current InputFile  : " + $InputFileName1); line in a Schedule block in my input section => same error The only way I find is to move the line in Input section without using a schedule block, like this : <Input in>  Module  im_file  SavePos TRUE  ReadFromLast FALSE  ActiveFiles 20  CloseWhenIdle TRUE  File "/var/log/MUP10/sac/APMUZS4WBS04*.log"  Exec $InputFileName1 = file_name(); Exec log_info("Current InputFile  : " + $InputFileName1); </Input> But it writes too many files in log…. Any idea ? Thanks in advance

Hi All Could you confirm please that File_remove with wildcards AND File's created date condition works ? I'm trying file_remove commands below : The two first works properly => no problem with file_remove, "simple" or with wildcards But not the last… using "now() - 18000", to remove file older than 5h never works, and no error in nxlog.log...looks like the line doesn't exist...? <Schedule>   Every 1 min   Exec    file_remove('/var/log/MUP10/sac/APMUZS4WBS03-2016061300.log');   Exec    file_remove('/var/log/MUP10/sac/APMUZS4WBS04-201606130*', now());   Exec    file_remove('/var/log/MUP10/sac/APMUZS4WBS*.log', (now() - 18000)); </Schedule> Thks In Nxlog documentation : ~file_remove(string file, datetime older);  description Remove the file ’file’ if its creation time is older than the value specified in ’older’. It is possible to specify a  wildcard in filenames (but not in the path). If you use backslash as the directory separator with wildcards, make sure  to escape this (e.g. ’C:\\test\\*.log’). This procedure will reopen the LogFile if this is removed. An error is logged if the operation fails.

Hi, I am using om_http module to send windows eventlogs to AWS API Gateway for further processing. I kept HTTPSAllowUntrusted to True. But I keep getting <cloudfront_hostname>:443 connection failure reconnecting in ## seconds. I can POST data to the URI using curl just fine. I believe it is related to SNI support, which was also limiting other tools like wrk, ab, siege https://github.com/wg/wrk/issues/149 .  Is there any workaround or fix to support SNI? Thanks, Shri

My Setup: Graylog2 server to collect logs Ubuntu machine running Zimbra sending logs from various Zimbra logfiles in GELF format My problem: Messaged received by Graylog are truncated. Here is an actual message as it appeared on my Ubuntu server's "mailbox.log" file (please note that I have X'd out the email address): 2016-06-12 08:51:17,832 INFO  [ImapSSLServer-95] [name=XXXXXXX@XXX.org;ip=10.10.48.74;ua=iPod touch Mail/13C75;] imap - ID elapsed=0 Here is the log as received by Graylog: 2016-06-12 08:51:17,832 INFO [ImapSSLServer-95] [name=XXXXXXX All message seem to be truncated after exactly the same number of characters. I cannot seem to figure this out and would love some help. Below I have pasted my nxlog.conf file: ## This is a sample configuration file. See the nxlog reference manual about the ## configuration options. It should be installed locally under ## /usr/share/doc/nxlog-ce/ and is also available online at ## http://nxlog.org/docs ######################################## # Global directives                    # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules                              # ######################################## <Extension gelf>     Module      xm_gelf </Extension> <Extension syslog2>     Module    xm_syslog </Extension> <Input mailbox.log>         Module  im_file         File    "/opt/zimbra/log/mailbox.log"     InputType LineBased     SavePos    TRUE </Input> <Input access_log>     Module    im_file     File    "/opt/zimbra/log/access_log*"     SavePos True </Input> <Input audit.log>     Module    im_file     File    "/opt/zimbra/log/audit.log"     SavePos    TRUE </Input> <Input clamd.log>     Module    im_file     File    "/opt/zimbra/log/clamd.log"     SavePos    TRUE </Input> <Input freshclam.log>     Module    im_file     File    "/opt/zimbra/log/freshclam.log"     SavePos    TRUE </Input> <Input mysql_error.log>     Module    im_file     File    "/opt/zimbra/log/mysql_error.log"     SavePos TRUE </Input> <Input mail.log>     Module    im_file     File    "/var/log/mail.log"     SavePos TRUE </Input> <Input zimbra.log>     Module im_file     File    "/var/log/zimbra.log"     SavePos TRUE </Input> <Input syslog>     Module im_file     File    "/var/log/syslog"     SavePos    TRUE </Input> <Input zimbra-stats.log>     Module im_file     File    "/var/log/zimbra-stats.log"     SavePos TRUE </Input> <Output out>     Module      om_udp     Host        10.10.90.45     Port        5407     Exec    to_syslog_snare(); </Output> <Output out2>     Module    om_udp     Host    10.10.90.45     Port    5413     OutputType    GELF </Output> ######################################## # Routes                               # ######################################## <Route 1>     Path        mailbox.log => out </Route> <Route 2>     Path    access_log => out </Route> <Route 3>     Path    audit.log => out </Route> <Route 4>     Path    clamd.log => out </Route> <Route 5>     Path    freshclam.log => out </Route> <Route 6>     Path    mysql_error.log => out </Route> <Route 7     Path    mail.log => out </Route> <Route 8>     Path    zimbra.log => out </Route> <Route 9>     Path    syslog => out2 </Route> <Route 10>     Path    zimbra-stats.log => out </Route>  

I've seen the blog posting for AIX support following the download source and compile route. Is a binary distribution of AIX in the roadmap to anyone's knowledge? Thanks,   Rob

Hello, I would like to obtain information in alert.log multiple lines for an event but without success. Below, my configuration : <Extension multiline>         Module       xm_multiline         HeaderLine   /^\w\w\w\s\w\w\w\s\d\d\s\d\d:\d\d:\d\d\s\d\d\d\d/ </Extension> <Extension syslog>         Module  xm_syslog </Extension> <Input aix_syslog>         Module  im_file         File "/var/log/syslog.out*"         Exec        to_syslog_bsd(); </Input> <Input oracle_log_DBA102XX>         Module      im_file         InputType   multiline         File        "/oradata/DBA102XX/admin/trace/bdump/alert_DBA102XX.log"         Exec        to_syslog_bsd();         Exec        $SourceName = "DBA102XX"; </Input> <Output fluent_aix>         Module om_udp         Host 172.31.12.104         Port 5139 </Output> <Output fluent_oracle>         Module om_udp         Host 172.31.12.104         Port 5127 </Output> ######################################## # Routes                               # ######################################## <Route 1>         Path    aix_syslog => fluent_aix </Route> <Route 2>     Path        oracle_log_DBA102XX => fluent_oracle </Route> Result Fluentd side : 7 Jun 2016 23:21:21     oracle.user.notice      {"host":"xmgpoc01","ident":"Tue","message":"21:21 DFT 2016"} 7 Jun 2016 23:21:21     oracle.user.notice      {"host":"xmgpoc01","ident":"Thread","message":"1 advanced to log sequence 66 (LGWR switch)"} 7 Jun 2016 23:21:21     oracle.user.notice      {"host":"xmgpoc01","ident":"","message":"/oradata/DBA102XX/oraredo01/LOG3A_DBA102XX.ORA"} 7 Jun 2016 23:21:21     oracle.user.notice      {"host":"xmgpoc01","ident":"","message":"/oradata/DBA102XX/oraredo02/LOG3B_DBA102XX.ORA"} This line "Exec        $SourceName = "DBA102XX";" doesn't works, i dont know why. Thank you for your help.    

Hi, I am a newbie and trying to configure NXLog on a windows machine to forward logs to a syslog server. With the below configuration, all events are being forwarded to the server. we need only critical and error level events and only Security and System related events to be forwarded to the server. Below is my configuration: <Extension _syslog> Module xm_syslog </Extension> <Input in> Module im_msvistalog </Input> <Output out> Module om_udp Host 192.168.1.25 Port 514 Exec to_syslog_snare(); </Output> <Route 1> Path in => out </Route> Could some one help me what changes I should make so that only Critical and Error levels should be forwarded and only Security and System events should be forwarded ?  

I would like to use nxlog on the recent Ubuntu server edition and the current package does not work on it.  A new deb for Ubuntu 16.04 would be great or recent sources for the current 2.9 version.

How would I change the syslog event message on an output module so that every message gets an additional field?  I want to add another value called "Project X" after the message portion of all syslogs events/messages as they are forwarded to another server?  Would I use $raw_event as I show in my example?  Current config: <Output out>    Module om_udp    Host 192.168.1.10    Port 514 </Output>   Would I do this? <Output out>    Module om_udp    Host 192.168.1.10    Port 514    Exec $raw_event = $raw_event + "Project X"; </Output>   OR is it more like this...? <Output Out>    Module om_udp    Host 192.168.1.10    Port 514    $Message = $Message + "Project X"    Exec to_syslog_bsd(); </Output out>      

I am sending data to Elasticsearch via port 9200 and I am also using security for Elasticsearch. Any access to the rest API requiries authentication. Is there a way to configure the om_http output to provide headers to authenticate when sending data to Elasticsearch? Thank you

What this error means that leads some output modules to stop sending logs without crashing?   ASSERTION FAILED at line 21 in writerfuncs.c/nx_module_output_fill_buffer(): "output->buflen == 0"