Hello,

I have installed NXLog community edition to collect table data from PostgreSQL database but, the table isn't contain an ID column. As I understand, NXLog required to this field to bookmark but, we don't have. I'm looking for a workaround to solve the issue. On the other hand I can see a workaround from the following link and we can configure the ID with select statements but, the article isn't about the PostgreSQL. Could you please someone help me for PostgreSQL?

https://nxlog.co/documentation/nxlog-user-guide/mssql.html

the second question: Can we define a specific column (such as eventime) for ID (bookmark) with the following sample data?

2020-02-11 15:00:00.0000 2020-02-11 15:00:01.0001 2020-02-11 15:00:02.0002 2020-02-11 15:00:03.0000

Thanks in Advance!

Best Regards

SD

Hi there, a little bit of a novice here. Hope you dont mind pointing me in the right direction. I’m having some difficulty getting the configuration for using patternDB on windows 2003 servers, the configuration works for windows 2008+ The logs I have are as follows: 2020-02-05 13:48:32 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:40 2020-02-05 13:48:32 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:76 2020-02-05 13:48:32 ERROR module 'eventlog' has configuration errors, not adding to route '1' at C:\Program Files\nxlog\conf\nxlog.conf:57 2020-02-05 13:48:32 ERROR module 'eventlog_Pattern' has configuration errors, not adding to route 'route_Pattern' at C:\Program Files\nxlog\conf\nxlog.conf:94 2020-02-05 13:48:32 WARNING not starting unused module eventlog 2020-02-05 13:48:32 WARNING not starting unused module eventlog_Pattern 2020-02-05 13:48:32 INFO nxlog-ce-2.10.2150 started The section of conf is: ############################################################################ #### NXLOG WITH PATTERNDB ##### #### Uncomment the following lines for Windows Events filtered ##### ############################################################################ Module im_internal Module im_msvistalog Query \ \ *\ *\ *\ \ Module pm_pattern PatternFile %ROOT%\conf\patterndb.xml Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec if not defined $PatternID or not defined $Message { drop(); } Exec $Message = to_json(); to_syslog_bsd(); Path eventlog_Pattern, internal_Pattern => match_events => out_Pattern ############################################################################ ##### /NXLOG WITH PATTERNDB ##### ############################################################################ Thanks for reading. Please let me know if any more information needs to be included.

Hi all,

According to the documentation found here it indicates that the generic RPM doesn't have all available modules as opposed to the version specific RPM:

The generic RPM above contains all the libraries (such as libpcre and libexpat) 
that are needed by NXLog, the only dependency is libc. 
However, some modules are not available (im_checkpoint, for example). 
The advantage of the generic RPM is that it can be installed on most RPM-based Linux distributions.

Is there documentation for what modules are not available?
Are there any issues for deploying this version that I should know about up front?

Thanks!!

How can we rename nxlog package ? while we are placing both the rpm into spacewalk channel these are updating as “nxlog-ce-2.10.2150-1.x86_64.rpm” .So it's making a duplicate,So i hope renaming the rpm name will help us. Any help will be appreciated on this.

nxlog-ce-2.10.2150-1_rhel6.x86_64.rpm nxlog-ce-2.10.2150-1_rhel7.x86_64.rpm

Thanks! Ela

Our project is planning to reissue certificates for large amount of agents. Do we have API on the certificates so we'll able to reissue on these agents at the same time without doing it manually (one by one)?

Hi,

We are planning to deploy NXLog to thousands of endpoints and need to know when an agent is no longer sending data regularly.

Is there an established method for determining NXLog is working normally at scale?

Thanks!

Hello, I resisted posting here for a while but am finally at a loss to explain what I'm observing. I'm trying to send nginx access logs to graylog, and am mostly using code adapted from the nxlog ce user guide but I haven't been able to get the fields to successfully capture and arrive in graylog. The nginx server in question logs two sorts of traffic: 1. `x.x.x.x - - [04/Feb/2020:03:23:22 +0000] "GET /" 400 271 "-" "-" "-"` - These are status checks from a load balancer which I'm wanting to drop. 2. `x.x.x.x - [04/Feb/2020:03:23:01 +0000] "POST /rest/api/endpoint HTTP/1.1" 201 508 "-" "okhttp/3.3.0" "-"` - This is legitimate traffic to the application behind nginx which I want to parse and capture. I have the following input defined in my config file. Module im_file File '/var/log/nginx/access.log' PollInterval 1 SavePos True ReadFromLast True Recursive False RenameCheck False if $raw_event =~ /(?x)^(\S+)\ \S+\ (\S+)\ \[([^\]]+)\]\ \"(\S+)\ (.+)\ HTTP\/\d\.\d\"\ (\S+)\ (\S+)\ \"([^\"]+)\"\ \"([^\"]+)\"\ \"\S+\"$/ { $Hostname = $1; if $2 != '-' $AccountName = $2; $EventTime = parsedate($3); $HTTPMethod = $4; $HTTPURL = $5; $HTTPResponseStatus = $6; if $7 != '-' $FileSize = $7; if $8 != '-' $HTTPReferer = $8; if $9 != '-' $HTTPUserAgent = $9; delete($Message); } else drop(); If I remove the `else drop();` then this config forwards only the load balancer lines that I don't want, which indicates to me that the lines I do want are matching my regexp. But as shown, this configuration forwards nothing. I also don't think anything is wrong with my outputs or routes since just straight piping this all to graylog in the message field works fine. If anyone can help me understand what is going on here that would be much appreciated. Thanks

Hey y'all. Does anyone know where to find the GPG key used to sign the NXLog Enterprise Edition (trial) RPM packages? Key ID `9354d2051da9e40e`, or just `1da9e40e` for short. If someone at NXLog reads this, maybe [this page][] would be a good site to link it from, if it's not immediately available in the download package. I figured if I post this question here, maybe someone else will find it useful in the future. ([Wisdom of the Ancients, and all that][]) Thanks :) [this page]: https://nxlog.co/documentation/nxlog-user-guide/deploy-rhel.html [Wisdom of the Ancients, and all that]: https://xkcd.com/979/

I am currently running into an issue receiving syslog over ssl/tls. I cannot figure it out for the life of me!

Version: CE-2.10.2150

Error: INFO SSL connection accepted from IP_ADDRESS:PORT ERROR SSL certificate verification failed: unsupported certificate purpose (err: 26) WARNING SSL connection closed from IP_ADDRESS:PORT

Config: <Input in> Module im_ssl Host 0.0.0.0 Port 516 AllowUntrusted TRUE CAFile %CERTDIR%%CA-PEM% CertFile %CERTDIR%%CRT% CertKeyFile %CERTDIR%%KEY% KeyPass %PASSWORD% </Input>

Hi, I am using NXLog to run a script via `im_exec` and the internal `restart true` feature which restarts my script when it ends. The thing I'm noticing is that it's filling up the nxlog.log file with error conditions because the script stopped. `ERROR Module test got EOF, process exited?` The script is restarting deliberately so I don't need to know about these specific messages from this specific module. How can I suppress error logging from a specific module? Have a wonderful day! ***NOTE:*** I was able to get it working a different way. The solution involves executing a script on startup to get the initial useful data and send it to a log file. From there NXLog searches for that file and copies it to `gateway.log`. The input module `im_file` then reads that file which initiates the heartbeat. The way that `im_file` works is that it caches the file and won’t read from it again unless the file changes(modifying last modified doesn’t appear to count as a change). To that end I added logic that would periodically delete and re-copy the file. In this way `im_file` has a new file to read from, and thus send the heartbeat. While NXLog logs when the new files are read, they are logged as WARNING, not ERROR, so you can set the `LogLevel` to `ERROR` ``` Module xm_fileop Every 25 sec if file_exists('%GATEWAY%.log') file_remove('%GATEWAY%.log'); if file_exists('%GATEWAY%') file_copy('%GATEWAY%', '%GATEWAY%.log'); Module im_file PollInterval 60 ReadFromLast false SavePos false File '%GATEWAY%.log' $gateway = $raw_event; ``` This is not meant to be some kind of groundbreaking discovery. I only post it here in case someone else ends up trying to figure out this requirement. Hope this helps!

Attempting to log to a syslog server based upon the specified filtered log ID's. When I use this config, I do not get any errors in the error log when the service starts. However I do not get anything to my syslog server. Not sure if this a problem with the in or output and would love some feedback.

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log

LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension> define HighEventIds 4618, 4649, 4719, 4765, 4766, 4794, 4897, 4964, 5124, 1102

define MediumEventIds 4621, 4675, 4692, 4693, 4706, 4713, 4714, 4715, 4716, 4724,
4727, 4735, 4737, 4739, 4754, 4755, 4764, 4764, 4780, 4816,
4865, 4866, 4867, 4868, 4870, 4882, 4885, 4890, 4892, 4896,
4906, 4907, 4908, 4912, 4960, 4961, 4962, 4963, 4965, 4976,
4977, 4978, 4983, 4984, 5027, 5028, 5029, 5030, 5035, 5037,
5038, 5120, 5121, 5122, 5123, 5376, 5377, 5453, 5480, 5483,
5484, 5485, 6145, 6273, 6274, 6275, 6276, 6277, 6278, 6279,
6280, 24586, 24592, 24593, 24594

define LowEventIds 4608, 4609, 4610, 4611, 4612, 4614, 4615, 4616, 4624, 4625,
4634, 4647, 4648, 4656, 4657, 4658, 4660, 4661, 4662, 4663,
4672, 4673, 4674, 4688, 4689, 4690, 4691, 4696, 4697, 4698,
4699, 4700, 4701, 4702, 4704, 4705, 4707, 4717, 4718, 4720,
4722, 4723, 4725, 4726, 4728, 4729, 4730, 4731, 4732, 4733,
4734, 4738, 4740, 4741, 4742, 4743, 4744, 4745, 4746, 4747,
4748, 4749, 4750, 4751, 4752, 4753, 4756, 4757, 4758, 4759,
4760, 4761, 4762, 4767, 4768, 4769, 4770, 4771, 4772, 4774,
4775, 4776, 4778, 4779, 4781, 4783, 4785, 4786, 4787, 4788,
4789, 4790, 4869, 4871, 4872, 4873, 4874, 4875, 4876, 4877,
4878, 4879, 4880, 4881, 4883, 4884, 4886, 4887, 4888, 4889,
4891, 4893, 4894, 4895, 4898, 5136, 5137

<Input events> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0" Path="Directory Service"> <Select Path="Directory Service">*[System[Provider[ @Name='Microsoft-Windows-ActiveDirectory_DomainService']]] </Select> </Query> </QueryList> </QueryXML> <Exec> if $EventID NOT IN (%HighEventIds%) and $EventID NOT IN (%MediumEventIds%) and $EventID NOT IN (%LowEventIds%) drop(); </Exec> </Input>

<Output udp> Module om_udp Host 172.17.103.13 Port 514 Exec to_syslog_snare(); </Output>

<Route uds_to_udp> Path events => udp </Route>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

I am looking for the buster Debian package. I already tried stretch version but it complained about unmet dependencies related to libssl.

The following packages have unmet dependencies:
 nxlog-ce : Depends: libperl5.24 (>= 5.24.0) but it is not installable
            Depends: libssl1.0.2 (>= 1.0.2d) but it is not installable

root@debian:~# dpkg --search libssl
libssl1.1:amd64: /usr/share/doc/libssl1.1
libssl1.1:amd64: /usr/share/doc/libssl1.1/changelog.Debian.gz
libssl-dev:amd64: /usr/lib/x86_64-linux-gnu/pkgconfig/libssl.pc
libssl1.1:amd64: /usr/share/doc/libssl1.1/NEWS.Debian.gz
libssl-dev:amd64: /usr/share/doc/libssl-dev/changelog.gz
libssl1.1:amd64: /usr/lib/x86_64-linux-gnu/libssl.so.1.1
libssl-dev:amd64: /usr/share/doc/libssl-dev
libssl-dev:amd64: /usr/lib/x86_64-linux-gnu/libssl.a
libssl-dev:amd64: /usr/share/doc/libssl-dev/copyright
libssl1.1:amd64: /usr/share/doc/libssl1.1/copyright
libssl-dev:amd64: /usr/lib/x86_64-linux-gnu/libssl.so
libssl-dev:amd64: /usr/share/doc/libssl-dev/changelog.Debian.gz
android-libboringssl: /usr/lib/x86_64-linux-gnu/android/libssl.so.0
libssl1.1:amd64: /usr/share/doc/libssl1.1/changelog.gz
root@debian:~# dpkg --search libperl
libperl5.28:amd64: /usr/share/doc/libperl5.28/changelog.Debian.gz
libperl5.28:amd64: /usr/lib/x86_64-linux-gnu/libperl.so.5.28
libperl5.28:amd64: /usr/share/doc/libperl5.28
libperl5.28:amd64: /usr/lib/x86_64-linux-gnu/libperl.so.5.28.1
libperl5.28:amd64: /usr/share/doc/libperl5.28/copyright

I think it's because of compile, so I tried to compiled it myself on a Debian buster but I stuck on ./configure which can not find libcrypto.(libssl-dev installed, libraries exist in lib path, ... )

I appreciate if anyone can share the binary package for buster release.

Thank you.

Hi, I'm trying to execute the following bash command using NXLog's im_exec input module: `/usr/sbin/route -n | grep 'UG[ \t]' | awk '{print $2}'` From the Linux terminal that command will give the local gateway. When I attempt to run the command in NXLog I get the following: ``` 2020-01-24 13:43:09 WARNING im_exec process /usr/sbin/route exited 2020-01-24 13:43:10 ERROR subprocess '8114' returned a non-zero exit value of 3 ``` Input module I'm using to execute the command: ``` Module im_exec Restart true Command /usr/sbin/route Arg -n | grep 'UG[ \t]' | awk '{print $2}' Arg grep 'UG[ \t]' Arg awk '{print $2}' Arg sleep 30 $gateway = $raw_event; ``` I've tried various combinations of configuring the `Arg`'s but so far no luck. If I try putting everything in the `Command` field I get this error: `/usr/sbin/route -n |grep 'UG[ \t]'| awk '{print $2}';sleep 30;No such file or directory` Can someone help me with the correct syntax? ***NOTE: It seems like the problem is with using the `|` character in the command. Anyone know if NXLog will accept this argument or what syntax I need to use?*** Thanks in advance!!

I am following the nxlog to splunk guide here: https://nxlog.co/documentation/nxlog-user-guide/splunk.html. Specifically, section '93.3. Sending Specific Log Types for Splunk to Parse'. When testing, even using the config from the page, I am still getting an error (see further below)

<Input eventxml> Module im_msvistalog Channel Security CaptureEventXML TRUE Exec $raw_event = $EventXML; </Input>

<Output splunk_hec> Module om_http URL https://127.0.0.1:8088/services/collector/raw AddHeader Authorization: Splunk c6580856-29e8-4abf-8bcb-ee07f06c80b3 </Output>

This generates this error: ERROR invalid keyword: CaptureEventXML at C:\Program Files (x86)\nxlog\conf\nxlog.conf

Any ideas? thanks

Hello, I'm using nxlog community to send logs from my firewalls through syslog. My output is like that: <output log_to_file> module om_file File 'c:\datas\firewall_' + $MessageSourceAddress + '.log' <output> If my firewalls 192.168.1.1 and 192.168.1.2 are correctly configured, the result will be two files: c:\datas\firewall_192.168.1.1.log and c:\datas\firewall_192.168.1.2.log My problem is now to rotate this file on a daily basis. I've tried to apply the command rotate_to but it seems that it applies only on the first file.

How can I do rotation on multiple files with names based on a variable ?

Thank you !

nxlog-ce 2.9.1716 on Windows 10/Server 2016.

The usage of om_udp seems to cause nxlog.exe to listen on ephemeral port. om_tcp does not cause this. I can't find anything in documentation that explains this behavior.

Please help.

MK

After using kvp parser i've got variables with spaces in names. For example: "$Event Time" or "$Source Name".

I'm interested in two things:

1. How i can interact with this variable names? For example i'm trying construction "$EventTime = $Event Time;" with many shield variations: ",',),], etc, but this not work.
2. it is possible to prevent this situation? Massage format example below:

"DeviceEvent: Virus found,IP Address: 10.X.X.X,Computer name: xxx-xxx,Source: Auto-Protect scan,Risk name: Infostealer.Gampass,Occurrences: 1,File path: X:\xxxx_xxx.exe,Description: ,Actual action: Moved back,Requested action: Quarantined,Secondary action: Deleted,Event time: 2020-01-21 17:24:58,Event Insert Time: 2020-01-21 17:27:06,End Time: 2020-01-21 17:59:17,Last update time: 2020-01-21 18:01:07,Domain Name: xxxx,Group Name: XXXX,Server Name: xx-xxx,User Name: SYSTEM,Source Computer Name: ,Source Computer IP: ,Disposition: Reputation was not used in this detection.........."

Hello, I am sending a message with hostname to my syslog server, my conf is as follows:

define ROOT C: \ Program Files (x86) \ nxlog

Moduledir% ROOT% \ modules CacheDir% ROOT% \ data Pidfile% ROOT% \ data \ nxlog.pid SpoolDir% ROOT% \ data LogFile% ROOT% \ data \ nxlog.log

<Extension _syslog> Module xm_syslog </Extension>

<Input in> Module im_msvistalog

<Exec> parse_syslog (); $ Message = "hostnamexxx" + $ Message; to_syslog_ietf (); </Exec>

</Input>

<Output out> Om_udp module Host xx.xxx.xx Port 514 Exec to_syslog_ietf (); </Output>

<Route 1> Path in => out </Route>

My log is coming with the message correctly:

Feb 12 23:11:34 DESKTOP-XXXXX Microsoft-Windows-Eventlog [964] hostnamexxxxINFO 1102 The audit log was cleared. Subject: Security ID: # xxxxxxxx-1001 Account Name: Admin Domain Name: DESKTOP-XXXXX Logon ID: 0xD438A

However, the message "hostnamexxxx" is coming in the middle of the log, as you can see above. This is disturbing my parser, is there any way I can put this "hostnamexxxx" message last in my log? Example:

Feb 12 23:11:34 DESKTOP-XXXXX Microsoft-Windows-Eventlog [964] INFO 1102 The audit log was cleared. Subject: Security ID: # xxxxxxxx-1001 Account Name: Admin Domain Name: DESKTOP-XXXXX Logon ID: 0xD438A hostnamexxxx

Thanks

Hello everyone,

I'm having trouble architecturing something with hmac verification, any help of yours would be welcome.

I'm trying to setup an architecture with three clients/servers and using hmac/hmac_check to guarantee the integrity of the logs. Logs 1 are created by client 1 and sent to client 2, which check their integrity, logs 2 are created by client 2, and both logs 1 and 2 are send in the end to client 3 which finally check for integrity for both of them. Here is a "beautiful" scheme to illustrate my words:

client 1 ---hmac(logs 1)---> client 2 ---hmac_check(logs 1) + hmac(logs 2)---> client 3 ---hmac_check(logs 1) + hmac_check(logs2) + hmac(logs 3)-->...

I would have multiple routes on each clients and I would use different instances of each processor on each route to avoid having errors like "processor X already used in route A and was not load in the route B". Also, I would be using batchcompress between client 1 and 2 but UDP between client 2 and 3.

I'm wondering how you would do this thing? Would you open multiple UDP ports on client 3 to receive independantly logs coming from client 1 and client 2 and check the hmac independantly or would you send those two logs on the same network port and check them with the same hmac_check processor? And would you use multiple routes to process independantly logs coming from different clients because of the hmac integrity check?

Thank you in advance, Kind Regards,

Say I wanted to run a PowerShell script on an interval using the `im_exec` module how would I do that? ``` Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "script goes here" $output = $raw_event; ``` I can make the interval work by creating pauses in the code but then it appears as though the script process would run forever which is undesirable for efficiency and stability reasons. Is there a way to run the code on a interval, say every fifteen minutes? Thoughts? Thanks in advance!!