Hi I am trying to install nxlog on XP machine but I couldn't troubleshoot the error. Following is the error I am seeing on XP machine:

2020-08-20 15:54:06 WARNING nxlog-ce received a termination request signal, exiting... 2020-08-20 15:54:10 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:37 2020-08-20 15:54:10 ERROR module 'in' has configuration errors, not adding to route '1' at C:\Program Files\nxlog\conf\nxlog.conf:52 2020-08-20 15:54:10 ERROR route 1 is not functional without input modules, ignored at C:\Program Files\nxlog\conf\nxlog.conf:52 2020-08-20 15:54:10 WARNING no routes defined! 2020-08-20 15:54:10 WARNING not starting unused module in 2020-08-20 15:54:10 WARNING not starting unused module out 2020-08-20 15:54:10 INFO nxlog-ce-2.10.2150 started

I cant seem to get my im_file input to find my files. It is from a network mount that is mapped to the Z drive.

This is the actual file: Z:\WebCom\DDIWebCm Log Thursday, August 20, 2020.txt

I have tried to point the input directly to it and even tried wildcards. For example: File "Z:\WebCom\DDIWebCm*.txt" Yields 2020-08-20 13:50:29 ERROR failed to open directory: Z:\WebCom: The system cannot find the path specified.

Any ideas for me? Thanks in advance

Hi

Requirement: Reading logs of multiple servers from a single server

Is there any way to read a network shared log file like "\RemoteServer\Remotelog.log" using nxlog.

I tired using im_file module but not succeed, kindly let me know whether it is possible or not.

BR, Ravi

How can I get the filepath from im_file and pass it as variable in the Output.. as you can see i am using Recursive TRUE, want's to know which file from which directory, current message came from

<Input tst> Module im_file File "/home/*.log" Recursive TRUE </Input>

<Output tst_testfile> Module om_file File "/home/user/nxlogtest" <Exec> $tst_filepath = ??; $Message = $raw_event; to_json(); </Exec> </Output>

<Route tst_out> Path maas => maas_testfile </Route>

Hi all,

I want to send this text-logfiles to a syslog service. Now I´m stuck with xm_kvp, because it manipualtes the data in a way I don´t expect. The data looks as follows:

"srv1","IAS",08/01/2020,10:33:39,1,"123\xyz","123\xyz","dc-aa-94-96-52-70:WLAN-XYZ","f8-aa-4e-24-bc-7c",,,"wlc01","1.1.1.1",13,0,"1.1.1.1","wlc01",,,19,,,2,5,"wlan-xyz-01",0,"311 1 4.4.4.4 03/25/2020 14:12:43 689",,,,,,,,,"5e8427b3/f8:aa:4e:24:bc:7c/133777",,,,,,,,,13,6,,,,"49",,,,,,,,,,,"Wireless-hj",1,,,,


So there is comma separated values, without fieldnames. I want to create KVP values and send it to syslog via UDP. Lets focus on the formatted data.




So this is my code:

#####################
<Extension csv1>
Module xm_csv
Fields $ComputerName, $ServiceName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Callback-Number, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-Vendor, $Client-IP-Address, $Client-Friendly-Name, $Event-Timestamp, $Port-Limit, $NAS-Port-Type, $Connect-Info, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Class, $Session-Timeout, $Idle-Timeout, $Termination-Action, $EAP-Friendly-Name, $Acct-Status-Type, $Acct-Delay-Time, $Acct-Input-Octets, $Acct-Output-Octets, $Acct-Session-Id, $Acct-Authentic, $Acct-Session-Time, $Acct-Input-Packets, $Acct-Output-Packets, $Acct-Terminate-Cause, $Acct-Multi-Ssn-ID, $Acct-Link-Count, $Acct-Interim-Interval, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Client-Endpt, $Tunnel-Server-Endpt, $Acct-Tunnel-Conn, $Tunnel-Pvt-Group-ID, $Tunnel-Assignment-ID, $Tunnel-Preference, $MS-Acct-Auth-Type, $MS-Acct-EAP-Type, $MS-RAS-Version, $MS-RAS-Vendor, $MS-CHAP-Error, $MS-CHAP-Domain, $MS-MPPE-Encryption-Types, $MS-MPPE-Encryption-Policy, $Proxy-Policy-Name, $Provider-Type, $Provider-Name, $Remote-Server-Address, $MS-RAS-Client-Name, $MS-RAS-Client-Version
#EscapeControl FALSE
Delimiter ,
</Extension>
<Extension csv2>
Module xm_csv
Fields $ComputerName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-IP-Address, $Client-Friendly-Name, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Pvt-Group-ID
Delimiter ;
#EscapeControl False
EscapeChar \n
</Extension>
<Extension kvp1>
Module xm_kvp
#Delimiter ''
#ValueQuoteChar "
QuoteMethod All
#KVDelimiter =
EscapeChar \n
KVPDelimiter ;
IncludeHiddenFields False
</Extension>
<Extension kvp2>
Module xm_kvp
KVPDelimiter ;
</Extension>
<Input in>
Module im_file
File "d:\\nxlog\\IN2004.log"
InputType LineBased
PollInterval 1
ReadFromLast FALSE
SavePos FALSE
<Exec>
csv1->parse_csv();
if not defined $number $number = 0;
csv2->to_csv();
kvp1->to_kvp();
delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);
</Exec>
</Input>

#####################

Till csv2->to_csv();, it works fine and the output is as expected. Values are now semicolon seperated and surrounded by quotation marks "". This is what I want.

But know, when kvp1->to_kvp(); is also active, quotes are removed from all values but values with spaces in it. I do not want to change the quotes surrounding the values.

The result looks like this:

EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;ComputerName=srv1;ServiceName=IAS;Record-Date=08/01/2020;Record-Time=10:33:39;Packet-Type=1;User-Name=123\\xyz;Fully-Qualified-Distinguished-Name=123\\xyz;Called-Station-ID=dc-aa-94-96-52-70:WLAN-XYZ;Calling-Station-ID=f8-aa-4e-24-bc-7c;NAS-Identifier=wlc01;NAS-IP-Address=1.1.1.1;NAS-Port=13;Client-Vendor=0;Client-IP-Address=1.1.1.1;Client-Friendly-Name=wlc01;NAS-Port-Type=19;Service-Type=2;Authentication-Type=5;Policy-Name=wlan-xyz-01;Reason-Code=0;Class='311 1 4.4.4.4 03/25/2020 14:12:43 689';Acct-Session-Id=5e8427b3/f8:aa:4e:24:bc:7c/133777;Tunnel-Type=13;Tunnel-Medium-Type=6;Tunnel-Pvt-Group-ID=49;Proxy-Policy-Name=Wireless-hj;Provider-Type=1;number=0;

So does somebody know, why the quotes are beeing removed in general, but only kept for values with spaces in it?

Additionally, I would like to remove these fields: "EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;"

by using

delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);


but it is also not working.

Any ideas?



Thanks!

We need to send Cortex Data Lake logs (IETF) to Arcsight (CEF). Will I be able to set up NXLog to do this conversion for me?

Hi All,

I wonder if someone can answer this for me.

According to the documentation, it states that for a UDP client, the localport will be a random high port as per https://nxlog.co/documentation/nxlog-user-guide/om_udp.html

I have a situation where I am sending Zeek logs via UDP through a Google Seesaw load balancer see https://github.com/google/seesaw

The issue I am facing is that each separate log packet / connection from NXLog has the same client source port i.e 41460 in my case.

Tcpdump confirms this

Packet 1 15:55:10.533740 IP (tos 0x0, ttl 64, id 57228, offset 0, flags [DF], proto UDP (17), length 506) 172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 478

Packet 2 15:55:10.534026 IP (tos 0x0, ttl 64, id 57229, offset 0, flags [DF], proto UDP (17), length 847)172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 819

Is there a way to get NXLog to use a random client port for each connection?

It looks as if it chooses a random high port when the service is started.

Cheers

Cyberkryption

2020-08-17 16:31:18 INFO nxlog-ce-2.10.2150 started 2020-08-17 16:31:18 ERROR couldn't connect to udp socket on 10.0.20.99:12201; A socket operation was attempted to an unreachable network.

why I am getting this error?How can I solve this?

NXLog-CE version- 2.10.2150

Hi, we provide a SIEM solution for our customers, using AlienVault USM appliance and we are trying to implement/test NXlog for their servers. Has anyone implemented Oracle monitoring on Windows platform in NXlog and could share his experience? Don't seem to find any documentation for it. Many thanks in advance

Internet Explorer Logs Appear to have been deleted after installing nxlog. We received an alert shortly after installing nxlog on our server. After digging, they appear to be temporary log files. We have a very basic configuration with no purging explicitly defined. Is this normal behaviour?

Hi,

in my design, I use NXlog Community Edition servers as proxy collectors in network security zones; all production servers forward their logs to their closest NXlog proxy collector node, which in turn forwards to a SIEM server Output target. My question is: On a such collector node, can I parse the incoming data and if coming from a certain production server Input module instance, e.g. <Input myInput1>, forward only this data to a secondary Output target? The challenge lies in the fact that currently I've only got one collector node per security zone. The individual production server can only forward to the collector in the same zone, otherwise I would have created a separate Output instance and a Route for the particular Input instance to the secondary server.

Good day Family

I have a problem with nxlog on Linux, I am having difficulties pulling records from an oracle DB using the agent. Has anyone done it before?

Please help

Hi again! i want to check this config its ok. I need send a .csv to graylog and the graylog server is not getting messages and I wanted to check that the nxlog configuration was well done. The nxlog log, start without problems

#define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension json> Module xm_json </Extension>

<Extension fileop> Module xm_fileop </Extension>

<Extension _syslog> Module xm_syslog </Extension>

<Extension gelf> Module xm_gelf </Extension>

<Extension jira> Module xm_csv Fields $ComputerName,$SID,$Message FieldTypes string,string,string Delimiter"," </Extension>

<Input in> Module im_file File "C:\logs\logs.csv" #ReadFromLast False #Recursive True #SavePos True

</Input>

<Output out> Module om_udp Host 172.28.36.25 Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output>

<Route 1> Path in => out </Route>

Thanks all!

Is it possible to force the agent to send the IP address of the host in the syslog message?

Either send Only the IP or at least ensure the IP is included with the hostname.

You can get microseconds of an DateTime object:

integer microsecond(datetime datetime)
    Return the microsecond part of the time value.

Since this a fraction of a second want it to log:

 second($EventTime) + "." + microsecond(($EventTime)

However this is wrong, when the microseconds fraction is lower then 100000. I need to add leading zeroes.

How do i do this?

Note: I cannot use sprintf("%06d", val) in perl module, since i am on windows.

Any roadmap to create a specific Add-On for Azure Sentinel. It looks like a great fit since they are leveraging Logstash and fluentd as recommendations and those are not the easiest items to manage at scale.

Hi all, i am using nxlog to convert log from cef to json, output the same : {"SourceModuleName":"udp","timestamp":"2020-07-30T10:23:53.433042+07:00","serverity":"Low","signature":"/Execute/Query","category":"/Success","action":"keyinst","direction":"0","host":"192.168.51.15"} i want to add fields "vendor_id":"xxxx","unit_id":"00000","sensor_id":"xxxx" to before message and change the order of fields. After message the same: {"vendor_id":"xxxx","unit_id":"00000","sensor_id","timestamp":"2020-07-30T10:23:53.433042+07:00","action":"keyinst","direction":"0","host":"192.168.51.15""SourceModuleName":"udp","serverity":"Low","signature":"/Execute/Query","category":"/Success",}

Thanks!

I have been looking for a way of aggregating disparate logs and according to the Web NXLog is what I need. I have installed NXLog Community Edition and that's it. It a running. I had hoped for some form of web interface and from that a way of collecting log files. But I cannot seem to find anything of such ilk. I suspect I am missing something obvious, which is normal as I am partially sighted and tend to miss the obvious. If anyone could please spare the time to start me going collecting and reading logs, I would be extremely grateful.

Ubuntu 18.04 VM Many thanks and kind regards,

jB

Hi,

On output Module (om_tcp or om_ssl), "Listen" directives allow to configure output as a server : it will listen for incomming connection. On input Module (in_tcp or in_ssl), i don't find any "Connect" or "Listen->False" directives to configure input as a client... Why is it possible for output and not for input ?

We have a lot of server on DMZ Network and they can't connect to LAN Network for security reason.

What is the best way to do this ?

Thanks in advance, Best regards Julien.

Hi, I have a working configuration for sending Windows DHCP server logs to a remote syslog server, where we are combining the logs with FreeRADIUS logs for auditing and troubleshooting WiFi logon events. As the Windows servers use MAC addresses without colons, and our other logs use MAC addresses with colons, in order to more easily correlate events I would like to use NXLog to take the client MAC address ($MACAddress) and create a new variable ($ColonMAC) which will be appended to the end of the messages from the DCHP server. I have tried to use some of the regex from here: https://www.perlmonks.org/?node_id=947757 Having no Perl experience, I don't know how to properly format the code or where in the config file is the most appropriate place to add it in. All attempts so far have resulted in NXLog finding syntax errors in the following line. Can anyone suggest what needs to be added to the config below? Panic Soft #NoFreeOnExit TRUE #GLOBAL CONFIG define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_syslog Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 Module xm_exec Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); Module xm_csv Fields ID, Date, Time, Description, IPAddress, ClientHostname, MACAddress, \ UserName, TransactionID, QResult, ProbationTime, CorrelationID, \ DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \ UserClassASCII, RelayAgentInformation, DnsRegError Module xm_csv Fields ID, Date, Time, Description, IPv6Address, Hostname, ErrorCode, \ DuidLength, DuidBytesHex, UserName, Dhcid, SubnetPrefix #INPUT Module im_file File "C:\\Windows\\Sysnative\\dhcp\\DhcpSrvLog-*.log" # Only process lines that begin with an event ID if $raw_event =~ /^\d+,/ { $FileName = file_name(); if $FileName =~ /DhcpSrvLog-/ { dhcp_csv_parser->parse_csv(); $QResult = integer($QResult); if $QResult == 0 $QMessage = "NoQuarantine"; else if $QResult == 1 $QMessage = "Quarantine"; else if $QResult == 2 $QMessage = "Drop Packet"; else if $QResult == 3 $QMessage = "Probation"; else if $QResult == 6 $QMessage = "No Quarantine Information"; } else { dhcpv6_csv_parser->parse_csv(); } $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S'); $ID = integer($ID); # DHCP Event IDs if $ID == 0 $Message = "The log was started."; else if $ID == 1 $Message = "The log was stopped."; else if $ID == 2 $Message = "The log was temporarily paused due to low disk space."; else if ($ID >= 10 and $ID = 20 and $ID = 50 and $ID < 1000) $Message = "Codes above 50 are used for Rogue Server Detection " + "information."; # DHCPv6 Event IDs else if $ID == 11000 $Message = "DHCPv6 Solicit."; else if $ID == 11001 $Message = "DHCPv6 Advertise."; else if $ID == 11002 $Message = "DHCPv6 Request."; else if $ID == 11003 $Message = "DHCPv6 Confirm."; else if $ID == 11004 $Message = "DHCPv6 Renew."; else if $ID == 11005 $Message = "DHCPv6 Rebind."; else if $ID == 11006 $Message = "DHCPv6 Decline."; else if $ID == 11007 $Message = "DHCPv6 Release."; else if $ID == 11008 $Message = "DHCPv6 Information Request."; else if $ID == 11009 $Message = "DHCPv6 Scope Full."; else if $ID == 11010 $Message = "DHCPv6 Started."; else if $ID == 11011 $Message = "DHCPv6 Stopped."; else if $ID == 11012 $Message = "DHCPv6 Audit log paused."; else if $ID == 11013 $Message = "DHCPv6 Log File."; else if $ID == 11014 $Message = "DHCPv6 Bad Address."; else if $ID == 11015 $Message = "DHCPv6 Address is already in use."; else if $ID == 11016 $Message = "DHCPv6 Client deleted."; else if $ID == 11017 $Message = "DHCPv6 DNS record not deleted."; else if $ID == 11018 $Message = "DHCPv6 Expired."; else if $ID == 11019 $Message = "DHCPv6 Leases Expired and Leases Deleted."; else if $ID == 11020 $Message = "DHCPv6 Database cleanup begin."; else if $ID == 11021 $Message = "DHCPv6 Database cleanup end."; else if $ID == 11022 $Message = "DNS IPv6 Update Request."; else if $ID == 11023 $Message = "DNS IPv6 Update Failed."; else if $ID == 11024 $Message = "DNS IPv6 Update Successful."; else if $ID == 11028 $Message = "DNS IPv6 update request failed as the DNS update " + "request queue limit exceeded."; else if $ID == 11029 $Message = "DNS IPv6 update request failed."; else if $ID == 11030 $Message = "DHCPv6 stateless client records purged."; else if $ID == 11031 $Message = "DHCPv6 stateless client record is purged as the " + "purge interval has expired for this client record."; else if $ID == 11032 $Message = "DHCPV6 Information Request from IPV6 Stateless Client."; else drop(); } # Discard header lines (which do not begin with an event ID) else drop(); #OUTPUT Module om_udp Host 10.0.0.200 Port 514 Exec to_syslog_bsd(); #ROUTES Path dhcp_server_audit => out_syslog