I have my NXlog server successfully receiving rsyslog messages from client Linux boxes. Now I'm trying to have NXlog forward those messages to my Graylog server using GELF. NXlog and Graylog are both running on CentOS 7 If anyone can point me in the right direction config file wise I'd be greatly appreciative.
jerald.harrison created
micsnare created
With the following config file I am capturing the entire MS DNS logs. This includes the DNS header info which I want to filter out. I need help figuring out what I can add that will allow me to filter out the DNS header information.
define TAP_Sender_IP XXX.XXX.XXX.XXX define TAP_Sender_Port XXX
define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO
<Extension _syslog> Module xm_syslog </Extension>
<Input DNS>
Module im_file
File "C:\DNSlogs.txt"
SavePos True
<Exec>
if ($raw_event =~ /^#/) OR ($raw_event == '') drop();
else
{
to_syslog_bsd();
}
</Exec>
</Input>
<Output Tap> Module om_udp Host %TAP_Sender_IP% Port %TAP_Sender_Port% </Output>
<Route primary> Path DNS => Tap </Route>
This config removes the blank spaces between DNS entries but leaves the file header. I'm Not sure what I need to change to prevent this from capturing the DNS header information. Does anyone have any suggestions Thanks in Advance Dags
Dagron created
Hello,
I am still very new with nxlog, and currently, I am very stuck. I need to configure nxlog to pick up application files (currently we are only receiving standard windows logs). For testing purposes, I have installed and configured nxlog on my test machine.
We are using extra.conf to avoid making changes in nxlog config itself, which we install on the server automatically.
Here is my configuration which works fine on my test machine, but on the real application server, it only produces an empty file. Can it be related to the server itself, or am I just missing something? Logs are not showing any error messages.
<Extension xmlparser> Module xm_xml </Extension>
<Extension multiline_1> Module xm_multiline HeaderLine /^<Message>/ EndLine /^</Message>/ </Extension>
<Input timmsg> Module im_file File 'C:\Users\Administrator\Desktop\msg.log' SavePos FALSE ReadFromLast FALSE InputType multiline_1 <Exec>
Parse the xml event
parse_xml();
# Rewrite some fields
$EventTime = parsedate($timestamp);
delete($timestamp);
delete($EventReceivedTime);
# Convert to JSON
to_json();
</Exec>
</Input>
Define the output that goes to LogPoint for analysis
<Output timout> Module om_file File "C:\Users\administrator\Desktop\1.txt" </Output>
Tie together inputs to outputs
<Route 2> Path timmsg => timout </Route>
Lauxna created
Here at ABB Facts we have been using solarwinds forwarding client with Kiwi but the client is showing its age and no longer works properly.
I have set up nxlog as a replacement forwarder and it works fine but the priority is now set to Debug on all the windows events instead of Notice or Info as it was with the Solarwinds forwarder.
My conf file is very simple and shouldnt cause this problem. I have searched for a solution but without any luck.
Here is my conf file:
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log
<Extension syslog> Module xm_syslog </Extension>
<Input internal> Module im_internal </Input>
<Input eventlog>
Module im_msvistalog
Query <QueryList>
<Query Id="0">
<Select Path="System"></Select>
<Select Path="Security"></Select>
</Query>
</QueryList>
</Input>
<Output out> Module om_udp Host 10.250.254.19 Port 514 </Output>
kenneth.karlsson created
I am trying to install nxlog on Windows server 2000. However, I get the error "Installation directory must be on a local hard drive." I have tried using administrative command prompt, Same Error.
Can anyone help me out here?
BibekShrestha created
Hello I have a server that send logs (with nxlog-ce), that has 3 network interface (3 differents IPs). So any time that reboot server, I see (on my SIEM) incoming logs from a different IP. Is there a way to set on nxlog the default interface?
_omar_ created
Hi how can I modify my query to remember the last recordid? Right now im using the timestamp and its working but I want to use the recordid as the timestamp is causing another issue.
<Input in> Module im_odbc IdType integer ConnectionString DSN=SMS SQL;database=CM_IMG;encrypt=true;trustServerCertificate=true; SQL SELECT RecordID as id,'Microsoft_SCEP' as Name, MAL.DetectionTime as SCEPMalDetectTime, SY.Name0 as TargetHost,SY.Resource_Domain_OR_Workgr0 as NTdomain, U.UserName,MAL.Process,MAL.Path,MAL.ThreatID, MAL.ThreatName,MAL.SeverityID,TR.Category,CA.DefaultAction as CleanAction, PA.DefaultAction as PendingAction,MAL.ExecutionStatus,MAL.ActionSuccess,MAL.ErrorCode, CASE WHEN (LEN(MAL.Path) - LEN(REPLACE(MAL.Path, ';file:', ''))) = 0 THEN 1 ELSE (LEN(MAL.Path) - LEN(REPLACE(MAL.Path, ';file:', '')))/6 END AS MaliciousFileCt FROM CM_IMG.SCCM_Ext.vex_EP_Malware MAL LEFT JOIN CM_IMG.dbo.v_R_System SY on SY.ResourceID = MAL.ResourceID LEFT JOIN CM_IMG.dbo.v_Users U on U.UserID = MAL.UserID LEFT JOIN CM_IMG.dbo.EP_ThreatCategories TR on TR.CategoryID = MAL.CategoryID LEFT JOIN CM_IMG.dbo.EP_ThreatDefaultActions CA on CA.DefaultActionID = MAL.CleaningAction LEFT JOIN CM_IMG.dbo.EP_ThreatDefaultActions PA on PA.DefaultActionID = MAL.PendingActions where MAL.DetectionTime >= dateadd(minute, -5, GetDate()) SavePos TRUE </Input>
alain112 created
Hi,
I'm testing out the nxlog-ce-2.10.2150 and am trying to do simple exclusion of log files, as explained should be possible here: https://nxlog.co/documentation/nxlog-user-guide-full#im_file_config_exclude
However, no matter how I try, I get this:
2019-01-09 13:31:19 ERROR invalid keyword: Exclude at /etc/nxlog/nxlog.conf:109
Is this something that simply is not included in the community edition or what gives?
Thanks for any help,
Mika
MikaAleksandroff created
I've been unable to install Nxlog on Centos7 command line. I've researched the documentation and have used
yum install nxlog-ce-2.10.2150-1_rhel7.x86_64.rpm
at the command prompt and getting
No package nxlog-ce-2.10.2150-1_rhel7.x86_64.rpm available Error: nothing to do
Can you please assist? Thank you.
malwaresecninja created
finkle created
chown: cannot access ‘wmiusername:nxlog’: No such file or directory. where am I going wrong? My config is as below:
```
########################################
User nxlog
Group nxlog
LogFile /opt/nxlog/var/log/nxlog/nxlog.log
LogLevel DEBUG
########################################
# Inputs #
########################################
Module im_wmi
Host 10.x.x.x
Username wmiusername
Password SomePassword
Domain domain.com
ReadFromLast True
Module om_file
File "/opt/nxlog/var/windows/" + $AccountName + ".log"
Path wmi => file
```
yajitux created
I have installed the nxlog package on Linux. I am trying to use nxlog as syslog with SSL support. Following is the configuration file:
## This is a sample configuration file. See the nxlog reference manual about the
## configuration options. It should be installed locally under
## /usr/share/doc/nxlog-ce/ and is also available online at
## http://nxlog.org/docs
########################################
# Global directives #
########################################
User nxlog
Group nxlog
LogFile /var/log/nxlog/nxlog.log
LogLevel INFO
########################################
# Modules #
########################################
<Extension _syslog>
Module xm_syslog
</Extension>
<Input ssl>
Module im_ssl
Host localhost
Port 6292
CertFile /var/lib/nxlog/cert/certificate.pem
CertKeyFile /var/lib/nxlog/cert/key.pem
InputType Syslog_TLS
Exec parse_syslog_ietf();
</Input>
<Input in1>
Module im_udp
Port 514
Exec parse_syslog_bsd();
</Input>
<Input in2>
Module im_tcp
Port 514
</Input>
<Output fileout1>
Module om_file
File "/var/log/logmsg.txt"
Exec if $Message =~ /error/ $SeverityValue = syslog_severity_value("error");
Exec to_syslog_bsd();
</Output>
<Output fileout2>
Module om_file
File "/var/log/logmsg2.txt"
</Output>
########################################
# Routes #
########################################
<Route 1>
Path in1 => fileout1
</Route>
<Route tcproute>
Path in2 => fileout2
</Route>
Itry to run this in foreground using nxlog -f. nxLog starts but it shows the warning - "WARNING not starting unused module ssl ". Does this mean SSL is not being used by nxLog? Why it says unused and how to correct it.
infogatherer created
I'm trying to use the NXLog to Transfer my MS Exchange Server Transport Logs to an Elastic Search Server. I added the follwing config to the nxlog.conf File:
define BASEDIR C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\MessageTracking
<Input in_exchange> Module im_file File '%BASEDIR%\MSGTRK????????-.LOG' # Exports all logs in Directory SavePos TRUE Exec if $raw_event =~ /HealthMailbox/ drop(); Exec if $raw_event =~ /^#/ drop(); </Input>
<Output out_exchange> Module om_udp Host <IP of Elastic Search Server> Port <Port of the Elasioc Search Server> Exec $SyslogFacilityValue = 2; Exec $SourceName = 'exchange_msgtrk_log'; Exec to_syslog_bsd(); </Output>
<Route exchange> Path in_exchange => out_exchange </Route>
When I start the nxlog Service it states in the log: INFO nxlog-ce-2.10.2150 started
I don't get any information on the ElasticSearch Server. In the Wireshark I don't see any UDP packet and so I guess the NXLOg Service don't do anything.
Are there any more inormation or debugging logs to use?
Thanks for any hint
Michael
michaelb created
Hello All
Happy New Year!!!
I am trying to delete IIS logs older than 7 days.
I tried using the following:
<Extension _fileop> Module xm_fileop <Schedule> Every 1 day Exec file_remove('E:\IIS Logs\W3SVC1\*.log', (now() - 6048000 )); </Schedule> </Extension>
But the above did not work - the file(s) were not deleted and no entries in the nxlog logfile.
Note: the nxlog-ce documentation states the following regarding wildcard usage:
file_remove(string file); Remove file. It is possible to specify a wildcard in the filename (but not in the path). The backslash () must be escaped if used as the directory separator with wildcards (for example, C:\test\*.log). This procedure will reopen the LogFile if it is removed. An error is logged if the operation fails.
For Debugging purposes, I tried using the following:
<Extension _fileop> Module xm_fileop
<Schedule> Every 1 day Exec if (file_ctime("E:\IIS Logs\W3SVC1\*.log") <= now()) log_info("File is older"); </Schedule> </Extension>
The following entry is in the nxlog logfile:
2019-01-02 18:10:19 ERROR failed to read file creation time on 'E:\IIS Logs\W3SVC1*.log': The filename, directory name, or volume label syntax is incorrect.
Note: Maybe file_cname cannot except wildcards in the filename because if I use the following:
<Schedule> Every 10 sec Exec if (file_ctime("E:\IIS Logs\W3SVC1\u_ex181224.log") <= now()) log_info("File is older"); </Schedule>
Then I receive the following in the nxlog logfile:
2019-01-02 18:23:08 INFO File is older
If I try the following:
<Schedule> Every 10 sec Exec if (file_ctime("E:\IIS Logs\W3SVC1\u_ex181224.log") <= now()) log_info("File is older"); Exec file_remove("E:\IIS Logs\W3SVC1\u_ex181224.log", (now() - 6048000 )); </Schedule>
The following appears in nxlog logfile:
2019-01-02 18:26:26 INFO File is older 2019-01-02 18:26:28 WARNING input file was deleted: E:\IIS Logs\W3SVC1\u_ex181224.log
All of the above is leading me to believe the file_remove procedure does not support wildcards in the name.
Does anyone have any experience with this?
Anyone have a suggestion on how I can implement the above? Ideally, I would like to pass in the IIS Logs parent folder and recursively delete files older than 7 days in all sub-folders.
Using nxlog-ce-2.10.2150
Many thanks
Stephen
stephen created
I am having this issue on both Ubuntu and FreeBSD. While trying to compile the pattern parser, it is failing because xcc doesn't exist:
Ubuntu:
xcc -b -i patterndb_parser.xcc -o patterndb_parser.c
/bin/bash: xcc: command not found
make[5]: [Makefile:738: patterndb_parser.c] Error 127 (ignored)
FreeBSD:
Making all in pattern
xcc -b -i patterndb_parser.xcc -o patterndb_parser.c
make[4]: exec(xcc) failed (No such file or directory)
*** Error code 1 (ignored)
Both of these lead to the build failing because patterndb_parser.c was not generated:
Ubuntu:
gcc: error: patterndb_parser.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
make[5]: *** [Makefile:480: patterndb_parser.lo] Error 1
FreeBSD:
cc: error: no such file or directory: 'patterndb_parser.c'
cc: error: no input files
*** Error code 1
Stop.
Where does xcc come from? Google is not helping me at all, nor find, nor looking though the git repository.
squigley created
Hi everybody,
I use nxlog with module om_batchcompress + SSL (self-signed certificates).
In nxlog client's log many errors sometimes:
ERROR remote ssl socket was reset? (SSL_ERROR_SYSCALL with errno=9); End of file found
What is means? Thank you so much.
versions:
- client nxlog-3.2.2002
- server nxlog-4.1.4046
hatula created
hatula created
Hello,
I have an application that logs some API requests and responses. Each request is logged in a different file, as a single line. In the system there are thousands of files, and Nxlog seems to have issues sending the logs to Elasticsearch. It reads the file, I can see im_file_add_file command in logs, but it takes a long time to actually send the message.
How does Nxlog process multiple files in a single directory? Thanks
kullboys created
Hi, I am very new to this and trying to configure the nxLog as a syslog server on windows 2012. I added the following to the default config file:
<Extension _syslog>
Module xm_syslog
</Extension>
<Input ssl>
Module im_ssl
Host localhost
Port 6393
CertFile %CERTDIR%/ukfff6292.pem
CertKeyFile %CERTDIR%/private.pfx
InputType Syslog_TLS
Exec parse_syslog_ietf();
</Input>
Then on cmd, I start the nxLog with -f option. I get the following error
2018-12-24 11:32:55 ERROR SSL error, couldn't read cert, no start line, 2018-12-24 11:32:55 WARNING no functional input modules! 2018-12-24 11:32:55 INFO nxlog-ce-2.10.2150 started 2018-12-24 11:32:55 ERROR SSL error, couldn't read cert, no start line,
Please help me regarding this.
infogatherer created