Internet Explorer Logs Appear to have been deleted after installing nxlog. We received an alert shortly after installing nxlog on our server. After digging, they appear to be temporary log files. We have a very basic configuration with no purging explicitly defined. Is this normal behaviour?

Hi, in my design, I use NXlog Community Edition servers as proxy collectors in network security zones; all production servers forward their logs to their closest NXlog proxy collector node, which in turn forwards to a SIEM server Output target. My question is: On a such collector node, can I parse the incoming data and if coming from a certain production server Input module instance, e.g. <Input myInput1>, forward only this data to a secondary Output target? The challenge lies in the fact that currently I've only got one collector node per security zone. The individual production server can only forward to the collector in the same zone, otherwise I would have created a separate Output instance and a Route for the particular Input instance to the secondary server.

Good day Family I have a problem with nxlog on Linux, I am having difficulties pulling records from an oracle DB using the agent. Has anyone done it before? Please help

Hi again! i want to check this config its ok. I need send a .csv to graylog and the graylog server is not getting messages and I wanted to check that the nxlog configuration was well done. The nxlog log, start without problems #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension fileop> Module xm_fileop </Extension> <Extension _syslog> Module xm_syslog </Extension> <Extension gelf> Module xm_gelf </Extension> <Extension jira> Module xm_csv Fields $ComputerName,$SID,$Message FieldTypes string,string,string Delimiter"," </Extension> <Input in> Module im_file File "C:\logs\logs.csv" #ReadFromLast False #Recursive True #SavePos True </Input> <Output out> Module om_udp Host 172.28.36.25 Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output> <Route 1> Path in => out </Route> Thanks all!

Is it possible to force the agent to send the IP address of the host in the syslog message? Either send Only the IP or at least ensure the IP is included with the hostname.

You can get microseconds of an DateTime object: integer microsecond(datetime datetime) Return the microsecond part of the time value. Since this a fraction of a second want it to log: second($EventTime) + "." + microsecond(($EventTime) However this is wrong, when the microseconds fraction is lower then 100000. I need to add leading zeroes. How do i do this? Note: I cannot use sprintf("%06d", val) in perl module, since i am on windows.

Any roadmap to create a specific Add-On for Azure Sentinel. It looks like a great fit since they are leveraging Logstash and fluentd as recommendations and those are not the easiest items to manage at scale.

Hi all, i am using nxlog to convert log from cef to json, output the same : {"SourceModuleName":"udp","timestamp":"2020-07-30T10:23:53.433042+07:00","serverity":"Low","signature":"/Execute/Query","category":"/Success","action":"keyinst","direction":"0","host":"192.168.51.15"} i want to add fields "vendor_id":"xxxx","unit_id":"00000","sensor_id":"xxxx" to before message and change the order of fields. After message the same: {"vendor_id":"xxxx","unit_id":"00000","sensor_id","timestamp":"2020-07-30T10:23:53.433042+07:00","action":"keyinst","direction":"0","host":"192.168.51.15""SourceModuleName":"udp","serverity":"Low","signature":"/Execute/Query","category":"/Success",} Thanks!

I have been looking for a way of aggregating disparate logs and according to the Web NXLog is what I need. I have installed NXLog Community Edition and that's it. It a running. I had hoped for some form of web interface and from that a way of collecting log files. But I cannot seem to find anything of such ilk. I suspect I am missing something obvious, which is normal as I am partially sighted and tend to miss the obvious. If anyone could please spare the time to start me going collecting and reading logs, I would be extremely grateful. Ubuntu 18.04 VM Many thanks and kind regards, jB

Hi, On output Module (om_tcp or om_ssl), "Listen" directives allow to configure output as a server : it will listen for incomming connection. On input Module (in_tcp or in_ssl), i don't find any "Connect" or "Listen->False" directives to configure input as a client... Why is it possible for output and not for input ? We have a lot of server on DMZ Network and they can't connect to LAN Network for security reason. What is the best way to do this ? Thanks in advance, Best regards Julien.

Hi, I have a working configuration for sending Windows DHCP server logs to a remote syslog server, where we are combining the logs with FreeRADIUS logs for auditing and troubleshooting WiFi logon events. As the Windows servers use MAC addresses without colons, and our other logs use MAC addresses with colons, in order to more easily correlate events I would like to use NXLog to take the client MAC address ($MACAddress) and create a new variable ($ColonMAC) which will be appended to the end of the messages from the DCHP server. I have tried to use some of the regex from here: https://www.perlmonks.org/?node_id=947757 Having no Perl experience, I don't know how to properly format the code or where in the config file is the most appropriate place to add it in. All attempts so far have resulted in NXLog finding syntax errors in the following line. Can anyone suggest what needs to be added to the config below? Panic Soft #NoFreeOnExit TRUE #GLOBAL CONFIG define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB <Schedule> Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); </Schedule> # Rotate our log file every week on Sunday at midnight <Schedule> When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); </Schedule> </Extension> <Extension dhcp_csv_parser> Module xm_csv Fields ID, Date, Time, Description, IPAddress, ClientHostname, MACAddress, \ UserName, TransactionID, QResult, ProbationTime, CorrelationID, \ DHCID, VendorClassHex, VendorClassASCII, UserClassHex, \ UserClassASCII, RelayAgentInformation, DnsRegError </Extension> <Extension dhcpv6_csv_parser> Module xm_csv Fields ID, Date, Time, Description, IPv6Address, Hostname, ErrorCode, \ DuidLength, DuidBytesHex, UserName, Dhcid, SubnetPrefix </Extension> #INPUT <Input dhcp_server_audit> Module im_file File "C:\\Windows\\Sysnative\\dhcp\\DhcpSrvLog-*.log" <Exec> # Only process lines that begin with an event ID if $raw_event =~ /^\d+,/ { $FileName = file_name(); if $FileName =~ /DhcpSrvLog-/ { dhcp_csv_parser->parse_csv(); $QResult = integer($QResult); if $QResult == 0 $QMessage = "NoQuarantine"; else if $QResult == 1 $QMessage = "Quarantine"; else if $QResult == 2 $QMessage = "Drop Packet"; else if $QResult == 3 $QMessage = "Probation"; else if $QResult == 6 $QMessage = "No Quarantine Information"; } else { dhcpv6_csv_parser->parse_csv(); } $EventTime = strptime($Date + ' ' + $Time, '%m/%d/%y %H:%M:%S'); $ID = integer($ID); # DHCP Event IDs if $ID == 0 $Message = "The log was started."; else if $ID == 1 $Message = "The log was stopped."; else if $ID == 2 $Message = "The log was temporarily paused due to low disk space."; else if ($ID >= 10 and $ID <= 16) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName + " |" + $ColonMac; else if $ID == 17 drop(); else if $ID == 18 drop(); else if ($ID >= 20 and $ID <= 23) $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 24 $Message = "IP address cleanup operation has began."; else if $ID == 25 $Message = "IP address cleanup statistics."; else if $ID == 30 drop(); else if $ID == 31 drop(); else if $ID == 32 drop(); else if $ID == 33 $Message = $Description + " |" + $IPAddress + " |" + $ClientHostname + " |" + $MACAddress + " |" + $UserName; else if $ID == 34 drop(); else if $ID == 35 drop(); else if $ID == 36 drop(); else if ($ID >= 50 and $ID < 1000) $Message = "Codes above 50 are used for Rogue Server Detection " + "information."; # DHCPv6 Event IDs else if $ID == 11000 $Message = "DHCPv6 Solicit."; else if $ID == 11001 $Message = "DHCPv6 Advertise."; else if $ID == 11002 $Message = "DHCPv6 Request."; else if $ID == 11003 $Message = "DHCPv6 Confirm."; else if $ID == 11004 $Message = "DHCPv6 Renew."; else if $ID == 11005 $Message = "DHCPv6 Rebind."; else if $ID == 11006 $Message = "DHCPv6 Decline."; else if $ID == 11007 $Message = "DHCPv6 Release."; else if $ID == 11008 $Message = "DHCPv6 Information Request."; else if $ID == 11009 $Message = "DHCPv6 Scope Full."; else if $ID == 11010 $Message = "DHCPv6 Started."; else if $ID == 11011 $Message = "DHCPv6 Stopped."; else if $ID == 11012 $Message = "DHCPv6 Audit log paused."; else if $ID == 11013 $Message = "DHCPv6 Log File."; else if $ID == 11014 $Message = "DHCPv6 Bad Address."; else if $ID == 11015 $Message = "DHCPv6 Address is already in use."; else if $ID == 11016 $Message = "DHCPv6 Client deleted."; else if $ID == 11017 $Message = "DHCPv6 DNS record not deleted."; else if $ID == 11018 $Message = "DHCPv6 Expired."; else if $ID == 11019 $Message = "DHCPv6 Leases Expired and Leases Deleted."; else if $ID == 11020 $Message = "DHCPv6 Database cleanup begin."; else if $ID == 11021 $Message = "DHCPv6 Database cleanup end."; else if $ID == 11022 $Message = "DNS IPv6 Update Request."; else if $ID == 11023 $Message = "DNS IPv6 Update Failed."; else if $ID == 11024 $Message = "DNS IPv6 Update Successful."; else if $ID == 11028 $Message = "DNS IPv6 update request failed as the DNS update " + "request queue limit exceeded."; else if $ID == 11029 $Message = "DNS IPv6 update request failed."; else if $ID == 11030 $Message = "DHCPv6 stateless client records purged."; else if $ID == 11031 $Message = "DHCPv6 stateless client record is purged as the " + "purge interval has expired for this client record."; else if $ID == 11032 $Message = "DHCPV6 Information Request from IPV6 Stateless Client."; else drop(); } # Discard header lines (which do not begin with an event ID) else drop(); </Exec> </Input> #OUTPUT <Output out_syslog> Module om_udp Host 10.0.0.200 Port 514 Exec to_syslog_bsd(); </Output> #ROUTES <Route 1> Path dhcp_server_audit => out_syslog </Route>

I have file log, it has like 2000 lines, when i try to send it through HTTP POST, it is only receive half data. It start from the middle line. Its start from the last 4 hour updated line/ row which the 1000+ rows/line. How to setup/ config the nxlog so it will start from the first line/ the whole file. Please help me... below is my config nxlog : <Extension multiline> Module xm_multiline FixedLineCount 32 </Extension> <Extension multiline_action> Module xm_multiline FixedLineCount 32 </Extension> <Input filein> Module im_file #Wincor File &quot;C:\\DataSend\\&quot; + &quot;data_1.log&quot; InputType multiline SavePos TRUE ReadFromLast TRUE CloseWhenIdle TRUE PollInterval 300 </Input> <Output http> Module om_http URL http://localhost/log-monitor/public/index.php/receive-log #HTTPSAllowUntrusted TRUE </Output> <Route filein_to_http> Path filein => http </Route> Thankyou in advance

Hi all! Actually i have this config in a server to get windows event and send to a graylog: #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _gelf> #Module xm_syslog Module xm_gelf #Module xm_json </Extension> <Input in1> Module im_msvistalog # Uncomment the following to collect specific event logs only Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> <Exec> if $Channel == 'Security' and $ObjectName =~ /.tmp/ drop(); </Exec> </Input> <Output out> Module om_udp Host 11x.11x.11x.11x Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output> <Route 1> Path in1 => out </Route> But i want to know if exist any method to read a lot a .evtx files in a folder and send to graylog. I need help with this because i dont know what is the best "input modules" i need Thanks!

Hi, I monitor file which is overwritten (not appended) with the same numberof logs (3 lines). I am not able to force nxlog to send this change. Example: file.txt before change: 1123 3256 2546 2342 file.txt after overwrite: 8888 3256 2461 2342 There is the same number of lines so size did not changed of file. What I have checked so far: <Schedule> Every 5 sec Exec log_info("scheduled execution at " + now()); module_restart(); </Schedule> SavePos FALSE ReadFromLast FALSE I use as a output file and tcp steam - the same result for both. Any advice how to force nxlog to read file again?

Hi I am trying to post events from my logs files to gelf_tcp (Graylog). My log records are in flat json format. I can push logs to gelf, however with few issues; I appreciate if you can help. Issue 1: My log record has a field called "level". When the record is sent to Graylog, level does not match the one that I have in log file. I do parse_json() first and also I tried to explicitly set the value of level based on NXLog documentation https://nxlog.co/documentation/nxlog-user-guide/xm_gelf.html There is also something wrong with documentation. It says gelf understand field "SeverityLevel" but in the example in the same page it is using "SyslogSeverityLevel". I tried to explicitly set both fields with $level field but nothing changes in graylog. Issue 2: I have timestamp filed in my log record which is ISO 8601 format. I could not find an easy way to parse it. the parsedata() function does not help there. I end up extracting date and time part from my field and then send it to parsedate($1 + " " + $2). If you know a better way, please let me know. Thanks Kev

Hello to All! I have an issue using the nxlog agent community edition. My output part of the config file looks like this: Module om_udp Host graylog.domain.local Port 12201 OutputType GELF As failover mechanism I use the DNS record for graylog.domain.local. When the primary graylog node is unreachable, the DNS record is updated to the secondary node. For some log sources, the OS is picking up the change almost immediately (there is no cache enable) but the agent doesn't deliver logs to the secondary node until the nxlog service is restarted. Couple of questions: Does the nxlog service keep any DNS cache? If yes, is there any way to bypass it? Any advice on how to fix the issue? Many thanks in advance

Hi, I want to log from my nxlog.conf to my LogFile. Example: nxlog.conf: include additonal/*.conf additonal/module.conf Log DEBUG "module.conf included" How can i do this?

Dear team, I was currently using nxLog agent for windows (community edition) to forward logs to a centOS server. And I was looking for a documentation on how to build nxLog agent for windows plateform but could not find any. I hope the build documentation with some insight on how to prepare development environment will bring more people with project. Since many of us would love to add some of features to personalize it for personal use rather than opening issues with feature request. I am a Linux user with some c development background so I hope to have some support on how to create development environment to build windows agent. My goal would be to add latest openSSL version support. Thanks

Hi, Here's my case : I'm trying to create Syslog Server Configuration, based on CentOS. Server must : -- Receive log from tcp -- store log into a NFS Volume automaticaly mounted by fstab on server startup ---- in case of fail of the NFS Volume, create a buffer on local disk. Here's my configuration : <Extension _syslog> Module xm_syslog </Extension> <Extension _xm_file> Module xm_fileop </Extension> <Input in> Module im_tcp Host 0.0.0.0 Port 39458 <Exec> $SERVER = hostname(); parse_syslog_ietf(); $DATE = strftime($EventTime, "%Y%m%d"); </Exec> </Input> <Processor buffer> Module pm_buffer maxSize 204800 Type Disk Directory %LOGDIR% WarnLimit 190000 </Processor> <Processor blocker> Module pm_blocker <Exec> $CHEMIN = "/var/partage/"+$SERVER; $test = dir_exists($CHEMIN); if $test == TRUE { blocker->block(FALSE); } else blocker->block(TRUE); </Exec> </Processor> <Output out> Module om_file File "/var/partage/"+$SERVER+"/"+$Hostname+"/"+$DATE+"_"+$Hostname+".log" CreateDir TRUE Exec to_syslog_ietf(); </Output> <Route tcp_to_file> Path in => buffer => blocker => out </Route> My Problem : When NFS Volume is down, pm_blocker write " can't check condition "dir_exists(<Path>)" : input/outpur Error" When i try without pm_blocker, om_file fails with same error and no buffer created.... Question : What is the best solution to implement a disk buffer system for om_file module ? How can i catch NFS volume exceptions ? Thanks in advance, best regards, Julien

I have a im_file to om_tcp route on the community edition. My applications starts generating logs at 5am and runs every 15 minutes. I have observed that it essentially skips the first X lines and syncs things only after the x lines starting 5:15 and works fine throughout the day. There is nothing in logs at 5am. I have tried routing it to another om_file on the same machine, it works ok. Can you please help me fix it? Here is the config: ######################################## # Global directives # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules # ######################################## <Extension _syslog> Module xm_syslog </Extension> <Extension _fileop> Module xm_fileop </Extension> ######################################## # Input # ######################################## <Input in-my-important-app> Module im_file File "/var/log/tomcat/my-important-app-*.log" ReadFromLast True SavePos True Exec $Filename = "myApp/" +file_basename(file_name()); Exec $Hostname = hostname_fqdn(); </Input> ######################################## # Output # ######################################## # Output to destination <Output out-destination> Module om_tcp Host [destination host] Port [destination port] OutputType Binary </Output> ######################################## # Routes # ######################################## <Route to_destination> Path in-my-important-app => out-destination </Route>