We need to send Cortex Data Lake logs (IETF) to Arcsight (CEF). Will I be able to set up NXLog to do this conversion for me?

Hi All, I wonder if someone can answer this for me. According to the documentation, it states that for a UDP client, the localport will be a random high port as per https://nxlog.co/documentation/nxlog-user-guide/om_udp.html I have a situation where I am sending Zeek logs via UDP through a Google Seesaw load balancer see https://github.com/google/seesaw The issue I am facing is that each separate log packet / connection from NXLog has the same client source port i.e 41460 in my case. Tcpdump confirms this Packet 1 15:55:10.533740 IP (tos 0x0, ttl 64, id 57228, offset 0, flags [DF], proto UDP (17), length 506) 172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 478 Packet 2 15:55:10.534026 IP (tos 0x0, ttl 64, id 57229, offset 0, flags [DF], proto UDP (17), length 847)172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 819 Is there a way to get NXLog to use a random client port for each connection? It looks as if it chooses a random high port when the service is started. Cheers Cyberkryption

2020-08-17 16:31:18 INFO nxlog-ce-2.10.2150 started 2020-08-17 16:31:18 ERROR couldn't connect to udp socket on 10.0.20.99:12201; A socket operation was attempted to an unreachable network. why I am getting this error?How can I solve this? NXLog-CE version- 2.10.2150

Hi, we provide a SIEM solution for our customers, using AlienVault USM appliance and we are trying to implement/test NXlog for their servers. Has anyone implemented Oracle monitoring on Windows platform in NXlog and could share his experience? Don't seem to find any documentation for it. Many thanks in advance

Internet Explorer Logs Appear to have been deleted after installing nxlog. We received an alert shortly after installing nxlog on our server. After digging, they appear to be temporary log files. We have a very basic configuration with no purging explicitly defined. Is this normal behaviour?

Hi, in my design, I use NXlog Community Edition servers as proxy collectors in network security zones; all production servers forward their logs to their closest NXlog proxy collector node, which in turn forwards to a SIEM server Output target. My question is: On a such collector node, can I parse the incoming data and if coming from a certain production server Input module instance, e.g. <Input myInput1>, forward only this data to a secondary Output target? The challenge lies in the fact that currently I've only got one collector node per security zone. The individual production server can only forward to the collector in the same zone, otherwise I would have created a separate Output instance and a Route for the particular Input instance to the secondary server.

Good day Family I have a problem with nxlog on Linux, I am having difficulties pulling records from an oracle DB using the agent. Has anyone done it before? Please help

Hi again! i want to check this config its ok. I need send a .csv to graylog and the graylog server is not getting messages and I wanted to check that the nxlog configuration was well done. The nxlog log, start without problems #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension fileop> Module xm_fileop </Extension> <Extension _syslog> Module xm_syslog </Extension> <Extension gelf> Module xm_gelf </Extension> <Extension jira> Module xm_csv Fields $ComputerName,$SID,$Message FieldTypes string,string,string Delimiter"," </Extension> <Input in> Module im_file File "C:\logs\logs.csv" #ReadFromLast False #Recursive True #SavePos True </Input> <Output out> Module om_udp Host 172.28.36.25 Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output> <Route 1> Path in => out </Route> Thanks all!

Is it possible to force the agent to send the IP address of the host in the syslog message? Either send Only the IP or at least ensure the IP is included with the hostname.

You can get microseconds of an DateTime object: integer microsecond(datetime datetime) Return the microsecond part of the time value. Since this a fraction of a second want it to log: second($EventTime) + "." + microsecond(($EventTime) However this is wrong, when the microseconds fraction is lower then 100000. I need to add leading zeroes. How do i do this? Note: I cannot use sprintf("%06d", val) in perl module, since i am on windows.

Any roadmap to create a specific Add-On for Azure Sentinel. It looks like a great fit since they are leveraging Logstash and fluentd as recommendations and those are not the easiest items to manage at scale.

Hi all, i am using nxlog to convert log from cef to json, output the same : {"SourceModuleName":"udp","timestamp":"2020-07-30T10:23:53.433042+07:00","serverity":"Low","signature":"/Execute/Query","category":"/Success","action":"keyinst","direction":"0","host":"192.168.51.15"} i want to add fields "vendor_id":"xxxx","unit_id":"00000","sensor_id":"xxxx" to before message and change the order of fields. After message the same: {"vendor_id":"xxxx","unit_id":"00000","sensor_id","timestamp":"2020-07-30T10:23:53.433042+07:00","action":"keyinst","direction":"0","host":"192.168.51.15""SourceModuleName":"udp","serverity":"Low","signature":"/Execute/Query","category":"/Success",} Thanks!

I have been looking for a way of aggregating disparate logs and according to the Web NXLog is what I need. I have installed NXLog Community Edition and that's it. It a running. I had hoped for some form of web interface and from that a way of collecting log files. But I cannot seem to find anything of such ilk. I suspect I am missing something obvious, which is normal as I am partially sighted and tend to miss the obvious. If anyone could please spare the time to start me going collecting and reading logs, I would be extremely grateful. Ubuntu 18.04 VM Many thanks and kind regards, jB

Hi, On output Module (om_tcp or om_ssl), "Listen" directives allow to configure output as a server : it will listen for incomming connection. On input Module (in_tcp or in_ssl), i don't find any "Connect" or "Listen->False" directives to configure input as a client... Why is it possible for output and not for input ? We have a lot of server on DMZ Network and they can't connect to LAN Network for security reason. What is the best way to do this ? Thanks in advance, Best regards Julien.

Hi, I have a working configuration for sending Windows DHCP server logs to a remote syslog server, where we are combining the logs with FreeRADIUS logs for auditing and troubleshooting WiFi logon events. As the Windows servers use MAC addresses without colons, and our other logs use MAC addresses with colons, in order to more easily correlate events I would like to use NXLog to take the client MAC address ($MACAddress) and create a new variable ($ColonMAC) which will be appended to the end of the messages from the DCHP server. I have tried to use some of the regex from here: https://www.perlmonks.org/?node_id=947757 Having no Perl experience, I don't know how to properly format the code or where in the config file is the most appropriate place to add it in. All attempts so far have resulted in NXLog finding syntax errors in the following line. Can anyone suggest what needs to be added to the config below? Panic Soft #NoFreeOnExit TRUE #GLOBAL CONFIG define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB <Schedule> Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); </Schedule> # Rotate our log file every week on Sunday at midnight <Schedule> When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); </Schedule> </Extension> <Extension dhcp_csv_parser> Module xm_csv Fields ID, Date, Time, Description, IPAddress, ClientHostname, MACAddress, \ UserName, TransactionID, QResult, ProbationTime, CorrelationID, \ DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \ UserClassASCII, RelayAgentInformation, DnsRegError </Extension> <Extension dhcpv6_csv_parser> Module xm_csv Fields ID, Date, Time, Description, IPv6Address, Hostname, ErrorCode, \ DuidLength, DuidBytesHex, UserName, Dhcid, SubnetPrefix </Extension> #INPUT <Input dhcp_server_audit> Module im_file File "C:\\Windows\\Sysnative\\dhcp\\DhcpSrvLog-*.log" <Exec> # Only process lines that begin with an event ID if $raw_event =~ /^\d+,/ { $FileName = file_name(); if $FileName =~ /DhcpSrvLog-/ { dhcp_csv_parser->parse_csv(); $QResult = integer($QResult); if $QResult == 0 $QMessage = "NoQuarantine"; else if $QResult == 1 $QMessage = "Quarantine"; else if $QResult == 2 $QMessage = "Drop Packet"; else if $QResult == 3 $QMessage = "Probation"; else if $QResult == 6 $QMessage = "No Quarantine Information"; } else { dhcpv6_csv_parser->parse_csv(); } $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S'); $ID = integer($ID); # DHCP Event IDs if $ID == 0 $Message = "The log was started."; else if $ID == 1 $Message = "The log was stopped."; else if $ID == 2 $Message = "The log was temporarily paused due to low disk space."; else if ($ID >= 10 and $ID <= 16) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName + " |" + $ColonMac; else if $ID == 17 drop(); else if $ID == 18 drop(); else if ($ID >= 20 and $ID <= 23) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 24 $Message = "IP address cleanup operation has began."; else if $ID == 25 $Message = "IP address cleanup statistics."; else if $ID == 30 drop(); else if $ID == 31 drop(); else if $ID == 32 drop(); else if $ID == 33 $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 34 drop(); else if $ID == 35 drop(); else if $ID == 36 drop(); else if ($ID >= 50 and $ID < 1000) $Message = "Codes above 50 are used for Rogue Server Detection " + "information."; # DHCPv6 Event IDs else if $ID == 11000 $Message = "DHCPv6 Solicit."; else if $ID == 11001 $Message = "DHCPv6 Advertise."; else if $ID == 11002 $Message = "DHCPv6 Request."; else if $ID == 11003 $Message = "DHCPv6 Confirm."; else if $ID == 11004 $Message = "DHCPv6 Renew."; else if $ID == 11005 $Message = "DHCPv6 Rebind."; else if $ID == 11006 $Message = "DHCPv6 Decline."; else if $ID == 11007 $Message = "DHCPv6 Release."; else if $ID == 11008 $Message = "DHCPv6 Information Request."; else if $ID == 11009 $Message = "DHCPv6 Scope Full."; else if $ID == 11010 $Message = "DHCPv6 Started."; else if $ID == 11011 $Message = "DHCPv6 Stopped."; else if $ID == 11012 $Message = "DHCPv6 Audit log paused."; else if $ID == 11013 $Message = "DHCPv6 Log File."; else if $ID == 11014 $Message = "DHCPv6 Bad Address."; else if $ID == 11015 $Message = "DHCPv6 Address is already in use."; else if $ID == 11016 $Message = "DHCPv6 Client deleted."; else if $ID == 11017 $Message = "DHCPv6 DNS record not deleted."; else if $ID == 11018 $Message = "DHCPv6 Expired."; else if $ID == 11019 $Message = "DHCPv6 Leases Expired and Leases Deleted."; else if $ID == 11020 $Message = "DHCPv6 Database cleanup begin."; else if $ID == 11021 $Message = "DHCPv6 Database cleanup end."; else if $ID == 11022 $Message = "DNS IPv6 Update Request."; else if $ID == 11023 $Message = "DNS IPv6 Update Failed."; else if $ID == 11024 $Message = "DNS IPv6 Update Successful."; else if $ID == 11028 $Message = "DNS IPv6 update request failed as the DNS update " + "request queue limit exceeded."; else if $ID == 11029 $Message = "DNS IPv6 update request failed."; else if $ID == 11030 $Message = "DHCPv6 stateless client records purged."; else if $ID == 11031 $Message = "DHCPv6 stateless client record is purged as the " + "purge interval has expired for this client record."; else if $ID == 11032 $Message = "DHCPV6 Information Request from IPV6 Stateless Client."; else drop(); } # Discard header lines (which do not begin with an event ID) else drop(); </Exec> </Input> #OUTPUT <Output out_syslog> Module om_udp Host 10.0.0.200 Port 514 Exec to_syslog_bsd(); </Output> #ROUTES <Route 1> Path dhcp_server_audit => out_syslog </Route>

I have file log, it has like 2000 lines, when i try to send it through HTTP POST, it is only receive half data. It start from the middle line. Its start from the last 4 hour updated line/ row which the 1000+ rows/line. How to setup/ config the nxlog so it will start from the first line/ the whole file. Please help me... below is my config nxlog : <Extension multiline> Module xm_multiline FixedLineCount 32 </Extension> <Extension multiline_action> Module xm_multiline FixedLineCount 32 </Extension> <Input filein> Module im_file #Wincor File &quot;C:\\DataSend\\&quot; + &quot;data_1.log&quot; InputType multiline SavePos TRUE ReadFromLast TRUE CloseWhenIdle TRUE PollInterval 300 </Input> <Output http> Module om_http URL http://localhost/log-monitor/public/index.php/receive-log #HTTPSAllowUntrusted TRUE </Output> <Route filein_to_http> Path filein => http </Route> Thankyou in advance

Hi all! Actually i have this config in a server to get windows event and send to a graylog: #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _gelf> #Module xm_syslog Module xm_gelf #Module xm_json </Extension> <Input in1> Module im_msvistalog # Uncomment the following to collect specific event logs only Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> <Exec> if $Channel == 'Security' and $ObjectName =~ /.tmp/ drop(); </Exec> </Input> <Output out> Module om_udp Host 11x.11x.11x.11x Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output> <Route 1> Path in1 => out </Route> But i want to know if exist any method to read a lot a .evtx files in a folder and send to graylog. I need help with this because i dont know what is the best "input modules" i need Thanks!

Hi, I monitor file which is overwritten (not appended) with the same numberof logs (3 lines). I am not able to force nxlog to send this change. Example: file.txt before change: 1123 3256 2546 2342 file.txt after overwrite: 8888 3256 2461 2342 There is the same number of lines so size did not changed of file. What I have checked so far: <Schedule> Every 5 sec Exec log_info("scheduled execution at " + now()); module_restart(); </Schedule> SavePos FALSE ReadFromLast FALSE I use as a output file and tcp steam - the same result for both. Any advice how to force nxlog to read file again?

Hi I am trying to post events from my logs files to gelf_tcp (Graylog). My log records are in flat json format. I can push logs to gelf, however with few issues; I appreciate if you can help. Issue 1: My log record has a field called "level". When the record is sent to Graylog, level does not match the one that I have in log file. I do parse_json() first and also I tried to explicitly set the value of level based on NXLog documentation https://nxlog.co/documentation/nxlog-user-guide/xm_gelf.html There is also something wrong with documentation. It says gelf understand field "SeverityLevel" but in the example in the same page it is using "SyslogSeverityLevel". I tried to explicitly set both fields with $level field but nothing changes in graylog. Issue 2: I have timestamp filed in my log record which is ISO 8601 format. I could not find an easy way to parse it. the parsedata() function does not help there. I end up extracting date and time part from my field and then send it to parsedate($1 + " " + $2). If you know a better way, please let me know. Thanks Kev

Hello to All! I have an issue using the nxlog agent community edition. My output part of the config file looks like this: Module om_udp Host graylog.domain.local Port 12201 OutputType GELF As failover mechanism I use the DNS record for graylog.domain.local. When the primary graylog node is unreachable, the DNS record is updated to the secondary node. For some log sources, the OS is picking up the change almost immediately (there is no cache enable) but the agent doesn't deliver logs to the secondary node until the nxlog service is restarted. Couple of questions: Does the nxlog service keep any DNS cache? If yes, is there any way to bypass it? Any advice on how to fix the issue? Many thanks in advance