Hi, I want to log from my nxlog.conf to my LogFile. Example: nxlog.conf: include additonal/*.conf additonal/module.conf Log DEBUG "module.conf included" How can i do this?

Dear team, I was currently using nxLog agent for windows (community edition) to forward logs to a centOS server. And I was looking for a documentation on how to build nxLog agent for windows plateform but could not find any. I hope the build documentation with some insight on how to prepare development environment will bring more people with project. Since many of us would love to add some of features to personalize it for personal use rather than opening issues with feature request. I am a Linux user with some c development background so I hope to have some support on how to create development environment to build windows agent. My goal would be to add latest openSSL version support. Thanks

Hi, Here's my case : I'm trying to create Syslog Server Configuration, based on CentOS. Server must : -- Receive log from tcp -- store log into a NFS Volume automaticaly mounted by fstab on server startup ---- in case of fail of the NFS Volume, create a buffer on local disk. Here's my configuration : <Extension _syslog> Module xm_syslog </Extension> <Extension _xm_file> Module xm_fileop </Extension> <Input in> Module im_tcp Host 0.0.0.0 Port 39458 <Exec> $SERVER = hostname(); parse_syslog_ietf(); $DATE = strftime($EventTime, "%Y%m%d"); </Exec> </Input> <Processor buffer> Module pm_buffer maxSize 204800 Type Disk Directory %LOGDIR% WarnLimit 190000 </Processor> <Processor blocker> Module pm_blocker <Exec> $CHEMIN = "/var/partage/"+$SERVER; $test = dir_exists($CHEMIN); if $test == TRUE { blocker->block(FALSE); } else blocker->block(TRUE); </Exec> </Processor> <Output out> Module om_file File "/var/partage/"+$SERVER+"/"+$Hostname+"/"+$DATE+"_"+$Hostname+".log" CreateDir TRUE Exec to_syslog_ietf(); </Output> <Route tcp_to_file> Path in => buffer => blocker => out </Route> My Problem : When NFS Volume is down, pm_blocker write " can't check condition "dir_exists(<Path>)" : input/outpur Error" When i try without pm_blocker, om_file fails with same error and no buffer created.... Question : What is the best solution to implement a disk buffer system for om_file module ? How can i catch NFS volume exceptions ? Thanks in advance, best regards, Julien

I have a im_file to om_tcp route on the community edition. My applications starts generating logs at 5am and runs every 15 minutes. I have observed that it essentially skips the first X lines and syncs things only after the x lines starting 5:15 and works fine throughout the day. There is nothing in logs at 5am. I have tried routing it to another om_file on the same machine, it works ok. Can you please help me fix it? Here is the config: ######################################## # Global directives # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules # ######################################## <Extension _syslog> Module xm_syslog </Extension> <Extension _fileop> Module xm_fileop </Extension> ######################################## # Input # ######################################## <Input in-my-important-app> Module im_file File "/var/log/tomcat/my-important-app-*.log" ReadFromLast True SavePos True Exec $Filename = "myApp/" +file_basename(file_name()); Exec $Hostname = hostname_fqdn(); </Input> ######################################## # Output # ######################################## # Output to destination <Output out-destination> Module om_tcp Host [destination host] Port [destination port] OutputType Binary </Output> ######################################## # Routes # ######################################## <Route to_destination> Path in-my-important-app => out-destination </Route>

I am trying to send parse each of these XML fields into a field for graylog to handle, any ideas would help. I've added <Extension xml> Module xm_xml </Extension> and Exec parse_windows_eventlog_xml(); to_xml(); but I'm not sure what else to do, I'm trying to work with this in the 'message' field The Federation Service validated a new credential. See XML for details. Activity ID: 494a36f8-9b89-4477-8676-0080000000e1 Additional Data XML: <?xml version="1.0" encoding="utf-16"?> <AuditBase xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="FreshCredentialAudit"> <AuditType>FreshCredentials</AuditType> <AuditResult>Success</AuditResult> <FailureType>None</FailureType> <ErrorCode>N/A</ErrorCode> <ContextComponents> <Component xsi:type="ResourceAuditComponent"> <RelyingParty>https://xxxxxx.xxxxxxx.edu/adfs/services/trust</RelyingParty> <ClaimsProvider>AD AUTHORITY</ClaimsProvider> <UserId>UNIVERSITY\xxxxxxxxxxxxxx</UserId> </Component> <Component xsi:type="AuthNAuditComponent"> <PrimaryAuth>N/A</PrimaryAuth> <DeviceAuth>false</DeviceAuth> <DeviceId>N/A</DeviceId> <MfaPerformed>false</MfaPerformed> <MfaMethod>N/A</MfaMethod> <TokenBindingProvidedId>false</TokenBindingProvidedId> <TokenBindingReferredId>false</TokenBindingReferredId> <SsoBindingValidationLevel>NotSet</SsoBindingValidationLevel> </Component> <Component xsi:type="ProtocolAuditComponent"> <OAuthClientId>N/A</OAuthClientId> <OAuthGrant>N/A</OAuthGrant> </Component> <Component xsi:type="RequestAuditComponent"> <Server>https://xxxxx.xxxxxx.edu/adfs/services/trust</Server> <AuthProtocol>WSFederation</AuthProtocol> <NetworkLocation>Intranet</NetworkLocation> <IpAddress>x.x.94.22</IpAddress> <ForwardedIpAddress>x.x.128.226</ForwardedIpAddress> <ProxyIpAddress>N/A</ProxyIpAddress> <NetworkIpAddress>N/A</NetworkIpAddress> <ProxyServer>N/A</ProxyServer> <UserAgentString>Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0</UserAgentString> <Endpoint>/adfs/ls/</Endpoint> </Component> </ContextComponents> </AuditBase>

Hello, I am collecting syslog from many different firewalls and I want to extract only a single field from these messages and output that to a file separate of the output file I am currently writing. Basically I want to match the field devname= and extract it and output the results into a list. Is there a way to avoid duplication? So I have firewall1, firewall2, firewall3 etc. and they are in the devname= field so in the log file it has devname="firewall1", devname="firewall2", devname="firewall3" etc. I am wondering how can I look for all these names and create a list of them. Thank you.

I have tried multiple ways to get the wildcard includes to work but just cant get it to work in CE. If I put the path in then it works. define ROOT C:\Program Files (x86)\nxlog define PLUGIN %ROOT%\additional works: include C:\Program Files (x86)\nxlog\additional\nxlog_exchange.conf doesn't work: include additional*.conf include additional/.conf include %PLUGIN%*.conf include %PLUGIN%/.conf

Hi, My working nxlog.conf relies on Query directives aimed at explicitly named Channels together with the im_msvistalog Module. <Input blu_eventlog_iis> Module im_msvistalog Query &lt;QueryList&gt;\ &lt;Query Id=&quot;0&quot;&gt;\ &lt;Select Path=&quot;Microsoft-IIS-Configuration/Administrative&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;Microsoft-IIS-Configuration/Operational&quot;&gt;*&lt;/Select&gt;\ &lt;Select Path=&quot;Microsoft-IIS-Logging/Logs&quot;&gt;*&lt;/Select&gt;\ &lt;/Query&gt;\ &lt;/QueryList&gt; # Filter noise from IIS schema issues Exec if ($Message =~ /Unable to find schema/) drop(); # Workaround for local time so as to standardize to absolute microseconds since epoch Exec $EventTime = integer($EventTime); # JSON is required because some Windows logs contain new-line characters. Exec $Message = to_json(); </Input> <Output blu_out_eventlog_iis> Module om_tcp Host %SIEM% Port %PORT% Exec to_syslog_bsd(); </Output> <Route route_eventlog_iis> Path blu_eventlog_iis => blu_out_eventlog_iis </Route> I hoped to clean up some 15007 errors in the nxlog.log files included below: 2020-07-06 14:42:55 ERROR failed to subscribe to msvistalog events,the channel was not found [error code: 15007]; The specified channel could not be found. I suspect the 15007 errors are being generated because IIS isn't setup. In other words, the named IIS Channels are queried but not found. Is it possible to add an If/Else statement as part of those Query directives? Looking over the NXlog CE Manual I only see If/Else statement used with Exec. Any guidance would be greatly appreciated!

Hi all, I must send the event logs (only event ID 4626 and 4625) in the log collector Linux. can you help me with configuration? Thank you.

I'm very new to NXLog and have no clue how to fix this. I was setting up an nxlog.conf file to one of our servers and we keep getting this error. It starts with the 'not enough' fields error but then after a few lines it says 'Too many fields'. There are 21 fields on the W3c fields, but I'm not sure why it seems like it's adding Fields as an input? I added an additional string in the Fieldtypes and also combined date-time instead of 'date, time' but didn't work. Is there a way to fix this in the Conf file? Or should I be looking in the IIS? Log expected 20, got 6 in input '#Software: Microsoft Internet Information Services 8.5' expected 20, got 2 in input '#Version: 1.0' expected 20, got 3 in input '#Date: 2020-06-30 08:35:19' Too many fields in CSV input, expected 20, got 21 in input '#Fields: date time c-ip c-port cs-username s-sitename s-computername cs-host s-ip s-port cs-method cs-uri-stem sc-status sc-win32-status sc-substatus sc-bytes cs-bytes time-taken x-session x-fullpath x-debug' Conf <Extension w3c_ftp> Module xm_csv Fields date, time, c-ip, cs-username, s-sitename, s-computername, s-ip, cs-method, cs-uri-stem, sc-status, sc-win32-status, sc-bytes, cs-bytes, time-taken, s-port, cs-host, sc-substatus, x-session, x-fullpath, x-debug, c-port FieldTypes string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string Delimiter ' ' </Extension> TIA!

Hi, I'm trying to send my custom jsotn to GrayLog. My json file is: { "Faxes": "68", "Last30DaysEmails": "9728", "TodayEmails": "66" } This is my nxlog config file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> <Extension multiline> Module xm_multiline HeaderLine /\d{4}-\d{2}-\d{2}\s\d*:\d\d:\d*,\d{3}\s[a-zA-Z]*\s\s\S\s\[Begin Lead\]/ EndLine /\d{4}-\d{2}-\d{2}\s\d*:\d\d:\d*,\d{3}\s[a-zA-Z]*\s\s\S\s\[End Lead\]/ </Extension> <Extension json> Module xm_json </Extension> <Input im_file> Module im_file File "C:\\logs\\myfile.json" InputType multiline SavePos FALSE ReadFromLast FALSE </Input> <Output om_udp> Module om_udp Host 192.168.1.100 Port 3514 OutputType GELF </Output> <Route 1> Path im_file => om_udp </Route> Now my problems is: When I send the file to GrayLog server I see multi line with different messages (as json variable), and not all in a single message. How can do to configure correctly my system? Thanks for the support. Marco

Hi, I've want to include nxlog to send all Windows logs to syslog plus Microsoft-Windows-Windows Defender/Operational, however as "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog" doesn't seem to include "Microsoft-Windows-Windows Defender" it doesn't read that config in at start and send those Windows Defender logs. How can I get nxlog to send everything it does now, plus Microsoft-Windows-Windows Defender logs, without having to define everything? The only way I can see to do this is with two im_msvistalog defines, i.e. define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension syslog> Module xm_syslog </Extension> <Input eventlog> Module im_msvistalog </Input> <Input eventlog_defender> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Microsoft-Windows-Windows Defender/Operational">*</Select> </Query> </QueryList> </Input> <Processor eventlog_transformer> Module pm_transformer Exec $Hostname = hostname(); OutputFormat syslog_rfc3164 </Processor> <Output syslogout> Module om_ssl [removed, but its to Papertrail] </Output> <Route 1> Path eventlog => eventlog_transformer => syslogout </Route> <Route 2> Path eventlog_defender => eventlog_transformer => syslogout </Route> That seems to work except there are three problems compared to when I only have the Microsoft-Windows-Windows Defender going and the other blocks deleted: The Windows Defender logging is a little inconsistent, testing with an EICAR test virus doesn't always appear, probably 50% chance it appears in the syslog (all events appear in Windows log) The severity and source is copied from the previous log line (usually info rather than warn severity, and not Microsoft-Windows-Windows Defender), maybe a threading issue? Each rows of text is in its own line rather than all in a single message which happens when its the only logging to send Is there a better way to do 'everything' plus Microsoft-Windows-Windows Defender?

I am trying to reduce the size of the message sent from my Windows event logs to graylog and I cannot for the life of me figuring out how tell it to drop certain fields The only thing I can figure out is that i'm supposed to use delete() but how to use and where to place it in my config is very frustrating.

I'm currently using nxlog to filter and forward syslog: Source => Filter Logs on intermediate server with nxlog installed => forward udp 514 (syslog). Config looks like the following: <Extension _syslog> Module xm_syslog </Extension> #syslog to Nxlog01 <Input _sys_in> Module im_udp Port 514 <Exec> if (($Message =~ /REGEX/) or ($Message =~ /REGEX/)) drop(); </Exec> Exec if $MessageSourceAddress != "1.1.1.1" drop(); </Input> <Processor _sys_norepeat> Module pm_norepeat CheckFields Hostname, Message, SourceName </Processor> #Syslog Output to DST server <Output _sys_out> Module om_udp Host dstserver_ip Port 514 Exec to_syslog_bsd(); </Output> <Route Logs> Path _sys_in => _sys_norepeat => _sys_out </Route> Firstly is this possible with the CE agent? I already performed a tcpdump on the intermediate server where nxlog is installed and i can see the syslog being received but not sent after filtered. There is udp 514 connectivity between nxlog server and destination server. I tried: Removing all regex filtering (since i thought my regex wasn't good enough), no results. Changed im_udp to im_tcp (i thought maybe you can't used the same port in both input and output modules). Checked whether there actually is network connectivity between nxlog server and destination server using nc by sending a udp 514 packet to dst server. Can anyone help with this?

Hi All, Needed help with parsing/modify. Would greatly appreciate some direction. At the moment, I am parsing a plain-test log-file and sending to a remote server: Jun 19 16:29:28 server12345 [...] Jun 19 16:29:28 server12345 --- Jun 19 16:29:27 server12345 [program.state :3371] The above is what I get with parsing. I was hoping to make every line transform like this Jun 19 16:29:28 server12345 **programName1** [...] Jun 19 16:29:28 server12345 **programName1** --- Jun 19 16:29:27 server12345 **programName1** [program.state :3371] Can someone help me understand, how I can get programName1 appended to each file, after the server hostname? Also, is it possible to parse the server-name, and replace it as follows: Jun 19 16:29:28 **server12345--NA** programName1 [...] Jun 19 16:29:28 **server12345--NA** programName1 --- Jun 19 16:29:27 **server12345--NA** programName1 [program.state :3371] Would appreciate some guidance on how to change this via config. I currently, am reading in a textfile via the om_file method. Update: Relevant snippets of config: <Input log_file> Module im_file File 'C:\program\var\log\file.log' #InputType multiline_parser Exec parse_syslog(); </Input> <Processor norepeat> Module pm_norepeat CheckFields Hostname, Message, SourceName OutputFormat syslog_rfc3164 </Processor> <Route log_output> Path log_file => norepeat => log_output </Route> <Output log_output> Module om_udp Host x.x.x.x Port 514 </Output>

Hello - new to nxlog. Trying to end nxlog process when EOF reached on input file. Any help appreciated. Thanks, Andrew

Hello, I am fairly new to NXlog and we are trying to understand if It's possible to route/forward logs based on the Source when only one input module for 514 traffic is definied. I understand that the Host directive in the Input module is actually related to the host itself where Nxlog is installed. Would adding some kind of Exec instructions in there help?? Perhaps using a processor in the route module? Since this is a new deployment, we'd like to set it up by following best practices from the get go. I appreciate your help.

Hi all, I have an issue when view agent internal logs "View Log" in NXLog Manager. When I clicked on the View Log button, it will pop up error message as follow: Error Executing the requested operation: Could not parse XML; nested exception is org.xml.sax.SAXParseException; lineNumber:x, columnNumber:x; Character reference "&#. Error Executing the requested operation: Could not parse XML; nested exception is org.xml.sax.SAXParseException; lineNumber:x, columnNumber:x; The element type "SOAP-ENV:Envelope" must be terminated by matching end-tag"</SOAP-ENV:Envelope>" Does anyone know what are those two errors about, and how should we fix it ? Where can we view / find the SOAP-ENV:Envelope or mentioned character from NXLog Manger? Thanks.

Good Morning All, I was hoping to get some direction with a log file we want to parse. We have a directory containing log files on a network share. A new log file is created every day. The issue with these logs is that there are no newline or carriage returns, it is one gigantic line. New events are just added at the end of the string. I am familiar with NXLog to the point of inputs / outputs / routes but I am not sure exactly my next step here. I have some regex that when ran manually will break the log down into individual lines but I am not sure how to implement that in NXLog in a way that it will read the log file, split the entirety of it into individual lines and then export those lines without seeing a bunch of duplication. Or in what blocks to implement which step. Some broad stroke guidance (or details) would be appreciated.

Hi Guys, I have a problem, I must to send the Event log a the LogCollecotr Linux, the problem is when Logcollector riceved the log, the formatting is much long, I want riceved an log so. Example Log LInux: " Jun 19 16:23:01 MSI su: pam_unix(su:session): session opened for user ema by (uid=0) " I can receive the log in this format?? Thank you.