Ask questions. Get answers. Find technical product solutions from passionate experts in the NXLog community.

IETF Timestamp Format

Hi,

Im trying to change the IETF timestamp format (number of precision). I have used to_syslog_ietf to create the formatted log message. Is anyone know how can we change the time from 6 decimal to 3 or no decimal precision)?

Sample Logs - (Current timestamp with 6 decimal) 2012-01-01T17:15:52.873750+08:00 myhost - - - [NXLOG@14506 TestField="test value"] test message

Required (timestamp with 3 decimal or no decimal) 2012-01-01T17:15:52.000+08:00 myhost - - - [NXLOG@14506 TestField="test value"] test message

Thanks.


md22 created
Replies: 1
View post »
last updated
Need help in Configuring nxlog running on Windows to send DNS Analytical logs

Hello,

Are there any links or discussion related to how nxlog will be configured, so a Windows server can send DNS Analytical logs to a remote collector.

Thank you. Any direction is much appreciated.

Temy


tomolte created
Replies: 1
View post »
last updated
windows to syslog to splunk using json
Hello, we are working on centralizing logging onto a syslog-ng server and using nxlog for the windows part of the infrastructure. We are using the IP of the sender to route the message to the correct file on the syslog server and I am trying to force nxlog to use the host's IP instead of it's name in the syslog header without much success. The end goal is to forward everything (Windows, Linux, Network) to syslog-ng and from there to Splunk using the Splunk Forwarder on the syslog server. For Windows I am using the to_json() function on the part of the message so that the logs are parsed automatically on the splunk side. There are 3 different QueryXML so as to assign different severity/facility to the messages and, again, route the messages to the correct file on the syslog server. I have tried both to_syslog_bsd() and to syslog_ietf(). For the "old" BSD format I am unable to force the IP in the syslog header when I use IETF format, the IP is correctly used but I then end up with a huge header and data (time, source, ..) being repeated several times which is not efficient BSD format: Apr 27 14:38:45 WIN-0KNG645FE0L Microsoft-Windows-Security-Auditing[628]: {"EventTime":"2020-04-27 14:38:45","Hostname":"10.10.231.53","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":4672,"SourceName":"Microsoft-Windows-Security-Auditing","ProviderGuid":"{54849625-5478-4994-A5BA-3E3B0328C30D}","Version":0,"Task":12548,"OpcodeValue":0,"RecordNumber":22998,"ActivityID":"{9B9FF64D-1C66-0003-EBF6-9F9B661CD601}","ProcessID":628,"ThreadID":692,"Channel":"Security","Category":"Special Logon","Opcode":"Info","SubjectUserSid":"S-1-5-18","SubjectUserName":"SYSTEM","SubjectDomainName":"NT AUTHORITY","SubjectLogonId":"0x3e7","PrivilegeList":"SeAssignPrimaryTokenPrivilege SeTcbPrivilege SeSecurityPrivilege SeTakeOwnershipPrivilege SeLoadDriverPrivilege SeBackupPrivilege SeRestorePrivilege SeDebugPrivilege SeAuditPrivilege SeSystemEnvironmentPrivilege SeImpersonatePrivilege SeDelegateSessionUserImpersonatePrivilege","EventReceivedTime":"2020-04-27 14:38:47","SourceModuleName":"in_sec","SourceModuleType":"im_msvistalog","SyslogFacilityValue":17,"SyslogSeverityValue":2} IETF format: Apr 27 13:39:38 10.10.231.53 1 2020-04-27T13:39:36.528628+02:00 WIN-0KNG645FE0L Microsoft-Windows-Security-Auditing 628 - [NXLOG@14506 Keywords="-9214364837600034816" EventType="AUDIT_SUCCESS" EventID="5379" ProviderGuid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Version="0" Task="13824" OpcodeValue="0" RecordNumber="22664" ActivityID="{9B9FF64D-1C66-0003-EBF6-9F9B661CD601}" ThreadID="1580" Channel="Security" Category="User Account Management" Opcode="Info" SubjectUserSid="S-1-5-21-4288340226-4165896404-4155820673-500" SubjectUserName="Administrator" SubjectDomainName="WIN-0KNG645FE0L" SubjectLogonId="0x53ac84" TargetName="ServerManager*" Type="0" CountOfCredentialsReturned="0" ReadOperation="%%8100" ReturnCode="3221226021" ProcessCreationTime="2020-04-27T11:39:34.659068400Z" ClientProcessId="2716" EventReceivedTime="2020-04-27 13:39:38" SourceModuleName="in_sec" SourceModuleType="im_msvistalog"] {"EventTime":"2020-04-27 13:39:36","Hostname":"10.10.231.53","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":5379,"SourceName":"Microsoft-Windows-Security-Auditing","ProviderGuid":"{54849625-5478-4994-A5BA-3E3B0328C30D}","Version":0,"Task":13824,"OpcodeValue":0,"RecordNumber":22664,"ActivityID":"{9B9FF64D-1C66-0003-EBF6-9F9B661CD601}","ProcessID":628,"ThreadID":1580,"Channel":"Security","Category":"User Account Management","Opcode":"Info","SubjectUserSid":"S-1-5-21-4288340226-4165896404-4155820673-500","SubjectUserName":"Administrator","SubjectDomainName":"WIN-0KNG645FE0L","SubjectLogonId":"0x53ac84","TargetName":"ServerManager*","Type":"0","CountOfCredentialsReturned":"0","ReadOperation":"%%8100","ReturnCode":"3221226021","ProcessCreationTime":"2020-04-27T11:39:34.659068400Z","ClientProcessId":"2716","EventReceivedTime":"2020-04-27 13:39:38","SourceModuleName":"in_sec","SourceModuleType":"im_msvistalog","SyslogFacilityValue":17,"SyslogSeverityValue":2} - BSD format seems preferred as it make for smaller messages - Sending over TCP so as to not loose some of the largest messages Edit: - NXLog nxlog-ce-2.10.2150 - Windows 2019 nxlog.conf define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_syslog Module xm_json Module im_internal Module im_msvistalog * $SyslogFacilityValue = syslog_facility_value("local1"); $SyslogSeverityValue = syslog_severity_value("emerg"); # $Message = to_json(); # $raw_event = to_json(); # to_syslog_bsd(); Module im_msvistalog * $SyslogFacilityValue = syslog_facility_value("local1"); $SyslogSeverityValue = syslog_severity_value("alert"); # $Message = to_json(); # to_syslog_bsd(); Module im_msvistalog * $SyslogFacilityValue = syslog_facility_value("local1"); $SyslogSeverityValue = syslog_severity_value("crit"); # $Hostname = host_ip(); # parse_syslog(); # $Message = to_json(); # to_syslog_bsd(); delete($Message); Module om_tcp Host 10.10.231.11 port 514 $Hostname = host_ip(); # parse_syslog(); # log_info($raw_event); $Message = to_json(); $Message =~ s/\\[r|n|t]/ /g; # to_syslog_ietf(); to_syslog_bsd(); $raw_event =~ s/\s{2,}/ /g; Path in_sys, in_app, in_sec => syslog-srv

vguyard created
Replies: 1
View post »
last updated
im_file configuration - use wildcard chars in path

HI,

Is there any solution to use wildcard chars in the path in the file input setting?

I have these folders: /path/to/logs/t-01test/log/app.log /path/to/logs/t-02apple/log/app_20200427.log /path/to/logs/t-03horse/log/app.log /path/to/logs/t-04table/log/app.log

I used this input settings:

<Input in_app> Module im_file File "/path/to/logs/t-/log/app.log" SavePos TRUE InputType LineBased </Input>

I always get the ERROR failed to open directory: /path/to/logs/t-*/logs: No such file or directory

I used this setting before: <Input in_app> Module im_file File "/path/to/logs/app.log" SavePos TRUE InputType LineBased </Input>

This is working correctly but we have added some folders under the /path/to/logs/t-01test/ where the nxlog user cant read for security reasons. The nxlog log is full with errors because of enough rights: ERROR failed to open directory: path/to/logs/t-01test/conf: Permission denied

Is it allowed to use wildcards in the path?

Thanks!


Zola created
Replies: 2
View post »
last updated
ERROR apr_file_write failed in om_exec on windows os when try to send logs to Azure Log Analytics
Hello, I'm trying configure NXlog CE (installed on windows server 2012r2) to collect syslog (from cisco asa), saving it to file and send to Azure Log Analytics (aka Microsoft OMS). In first step I try to collect syslog, convert to json and saving it to file. It's work well. For the next i installed last Pyton (3.8) and check that all libs installed. After that I changed the nxlog.conf according to the manual (https://nxlog.co/documentation/nxlog-user-guide/azure-oms.html#forwarding-data-to-log-analytics). Buy NXlog gives me the following error: `ERROR apr_file_write failed in om_exec; The pipe is being closed.` How can I fix this error? My nxlog.conf: Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log define JSONLOGFILE C:\Program Files (x86)\nxlog\data\json.txt LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data Module xm_syslog Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 Module xm_exec Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') >= 5M)) \ file_cycle('%LOGFILE%', 8); # Rotate our log file every week on Sunday at midnight When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); Module xm_json Module im_udp Port 514 Host 192.168.1.2 Exec parse_syslog(); to_json(); Module om_file File '%JSONLOGFILE%' Module om_exec Command "C:\\Users\\user\\AppData\\Local\\Programs\\Python\\Launcher\\py.exe" Arg "C:\Program Files (x86)\nxlog\oms-pipe.py" Path udp => file, azure_oms My configuration is different from the example in manual in the "Output azure_oms" part. If use this part as in the manual an error appears: Module om_exec Command oms-pipe.py Exec to_json(); Error: `ERROR couldn't execute process oms-pipe.py; The system cannot find the file specified. ` Please help me fix this error.

Anton.I created
Replies: 1
View post »
last updated
sending time with milliseconds to graylog

I have been using nxlog-ce-2.9.1716 to send logs in graylog. In graylog timestemp is without milliseconds. For parsing string to date I use such code if ( $raw_event =~ /^(\d{4}-\d\d-\d\d\s\d{1,2}:\d{1,2}:\d{1,2},\d{1,3})\s[\d+]\s(\S+)\s+\s+(.*)/ ) { $EventTime = parsedate($1); $EventTime = parsedate(strftime($EventTime, '%Y-%m-%d %H:%M:%SZ'));
} In this case I lost milliseconds. How can I use parsedate() method with milliseconds and converting time to UTC?


spodius created
Replies: 1
View post »
last updated
Service stops unexpectedly

Hi

https://portal.msrc.microsoft.com/en-us/security-guidance/advisory/ADV190023

Set according to the above site. result,Unexpected service outage occurs. I want you to tell me the measures.


taiei created
Replies: 4
View post »
last updated
Code error in line where there is no code

When I use xm_perl with this code from /usr/libexec/nxlog/modules/extension/perl/event1.pl:

use Log::Nxlog;
use strict;
use warnings;
use feature 'say';
use JSON;
use utf8;


sub rec2msg {
    my $msgsrcaddr = Log::Nxlog::get_field($event, 'MessageSourceAddress');
    if ( defined($msgsrcaddr) )
    {
        Log::Nxlog::set_field_string($event, 'MessageSourceCountry', "France");
    }
}

It gives me strange error:

syntax error at /usr/libexec/nxlog/modules/extension/perl/event1.pl line 16, at EOF

However , there is no code at line 16 at all. What does that mean? It happens when I use any perl code.


EZ created
Replies: 1
View post »
last updated
How to extract data from unnamed EventData Data fields of Event
Hy! Sometimes one has to analyze Microsoft events containing EventData fields including Data fields without name, or let's say unnamed Data fields, e.g. EventID 2889 - unsigned LDAP requests. ``` - - 2889 0 4 16 0 0x8080000000000000 242410 Directory Service PC1.DOMAINXY.local - 172.172.172.172:33426 DOMAINXY\USERXY 0 ``` Simply sending this to syslog works like charm. But in this case I woul like to get the values from all three fields into $raw_event with a special text,like: ``` define EventID_2889_REGEX /(?x) \ (?(^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$)) \ (?([\w\d]+)) \ (?(\d+))/ Module im_msvistalog Query \ \ *[System[(EventID=2889)]]\ \ Exec $EventData =~ %EventID_2889_REGEX%; Exec $raw_event = "Unsigned or simple non TLS LDAP bind request: [" + $EventData + "]"; Exec $SyslogFacilityValue = 21; Exec $SyslogFacility = "LOCAL5"; Exec $ProcessID = $SubjectUserName; Exec $SourceName = "2889"; Exec parse_syslog_ietf(); Exec to_syslog_ietf(); ``` I tried a lot, read the NXLog CE documentation hints, googled for days now...to list it all up here would explode the forum ;-) Could you please give me some hints how to process these values? Maybe this is not even possible in NXLog CE, see https://nxlog.co/question/4158/windows-eventdata-not-captured? Thanks in advance!

DH created
Replies: 1
View post »
last updated
im_exec on schedule
Hello, I'm trying to implement some kind of "inventory" module in nxlog for windows-based machines. The idea is to run CMD and get basic info like OS information, members of local admin group or active sessions of users - those are just examples, the goal would be to execute CMD and get data every X hours/days. as a PoC, I have following configuration: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_syslog Module xm_json Module im_exec Command "C:\Windows\System32\cmd.exe" Arg /k Arg dir Module om_file File "D:\\nxlog_debug.log" Path message => out_debug It works, I can see output of dir command in a .log file. However I cannot find a right way to add a "schedule". Tried in several ways, but it always fails with config error. Based on this example, is it possible to run a CMD with arguments with a schedule?

kumdabur created
Replies: 1
View post »
last updated
Windows Events appear with escaping xml characters

Hello for, windows events proceed in JSON but some fields like "CommandLine", "TaskContent", "EventData" e.t.c arrives with XML escaping character, like &lt; is < &gt; is > &amp; is & &quot; is "

Is it possible to disable escaping globally? Or the only option to solve the problem is using unescape_xml function for every field like:

$CommandLine = unescape_xml($CommandLine); $TaskContent = unescape_xml($TaskContent);


RAZR created
Replies: 1
View post »
last updated
Emitting Metrics from NXLog-EE

Hi,
Can NXLog collect and emit metrics on Linux and Windows?

It looks like Windows can be achieved with im_winperfcount but I haven't found metrics for Linux.

Does this exist in the Enterprise Edition?

Thanks!


casey1234 created
Replies: 1
View post »
last updated
Nxlog export Windows Logs to Rsyslog Server

How to fix output log file name abnormal Ext. WAP01.tks.co.th-",&quot;Sec-Fetch-Mode&quot;.log WAP01.tks.co.th-\r\n-.log WAP01.tks.co.th-l+xml,application.log

My configuration NXlog

This is a sample NXLog configuration file created by Loggly. June 2013

See the nxlog reference manual about the configuration options.

It should be installed locally and is also available

online at http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

Please set the ROOT to the folder your nxlog was installed into,

otherwise it will not start.

#define ROOT C:\Program Files\nxlog #define ROOT_STRING C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define ROOT_STRING C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

Include fileop while debugging, also enable in the output module below

#<Extension fileop>

Module xm_fileop

#</Extension>

<Extension json> Module xm_json </Extension>

<Extension syslog> Module xm_syslog </Extension>

<Input in> Module im_msvistalog

ReadFromLast TRUE

SavePos TRUE

Query <QueryList>
<Query Id="0">
<Select Path="Application"></Select>
<Select Path="System">
</Select>
<Select Path="Security">*</Select>
</Query>
</QueryList> </Input>

<Output out> Module om_tcp Host 10.6.5.133 Port 514

Exec $Hostname = hostname_fqdn(); Exec $SyslogFacility = 2; #Exec $SourceName = 'eventlogs'; #Exec to_syslog_bsd(); #Exec to_syslog_snare(); Exec to_syslog_ietf(); Exec $Message = to_json(); to_syslog_bsd(); </Output>

<Route 1> Path in => out </Route>

I want to . Input <Select Path="Application"></Select>\ ---> output hostname.application.log Input <Select Path="System"></Select>\ ---> output hostname.system.log Input <Select Path="Security">*</Select>\ ---> output hostname.security.log


kunagorn.g created
Replies: 1
View post »
last updated
Kafka output to Azure Event Hub

I have a SaaS vendor that uses NXLog to send logs to customers and I would like them to send to an Azure Event Hub that has Kafka enabled so we don't need to use VPNs. I'm able to connect and produce messages to the topic with python and logstash but they seem to be having issues with NXLog.

This is the error they tell me they are getting: 2020-04-09 09:41:27 ERROR rdkafka: [thrd:sasl_ssl://servername.servicebus.windows.net:9093/boots]: sasl_ssl://servername.servicebus.windows.net:9093/bootstrap: SASL GSSAPI mechanism handshake failed: Local: Broker transport failure: broker's supported mechanisms: (n/a) (after 0ms in state DOWN)

Azure seems to have specific requirements/limitations, the configuration below works for logstash and it uses sasl_jaas_config to pass the authentication data:

output { kafka { codec => json topic_id => "kafkatopic" bootstrap_servers => "eventhubname.servicebus.windows.net:9093" client_id => "azshrelasea01p" compression_type => "none" sasl_jaas_config => "org.apache.kafka.common.security.plain.PlainLoginModule required username='$ConnectionString' password='Endpoint=<eventhubconnectionstring>';" sasl_mechanism => "PLAIN" security_protocol => "SASL_SSL" ssl_truststore_password => "password" ssl_truststore_location => "/elastic/trust.jks" ssl_truststore_type => "jks" batch_size => 250 linger_ms => 6000 } }

Looking around the internet, it seems that librdkafka has some requirements that have to be met but I don't know how the nxlog config would look to implement these:

https://github.com/Azure/azure-event-hubs-for-kafka/issues/51 https://brokenco.de/2019/04/04/azure-eventhubs-rust.html

Anyone have any experience with this or could help me convert these logstash configs to a nxlog one? It looks like I need to use the Option function and pass the settings there for librdkafka but I don't have a lot of experience with NXLog.

Thanks!


sera123k created
Replies: 1
View post »
last updated
NXlog and Graylog sidecar - ERROR Couldn't open nxlog service; The specified service does not exist as an installed service.

I am trying to ship Windows logs to Graylog using Nxlog and when trying to apply my configuration, the Sidecar gives me the following error:

ERROR Couldn't open nxlog service; The specified service does not exist as an installed service.

Any idea what could be wrong? I followed the instructions here https://docs.graylog.org/en/3.2/pages/sidecar.html#nxlog-on-windows to set it up.

Thank you.


junior466 created
Replies: 1
View post »
last updated
NXLog CE is not sending any data

Hello,

I am having the problem that nothing is sent to the server and I dont know why. Here is my config:

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% LogLevel INFO

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

define BASEDIR C:\MessageTracking

<Input in_exchange> Module im_file File '%BASEDIR%\MSGTRK*-*.LOG' # Exports all logs in Directory SavePos TRUE Exec if $raw_event =~ /HealthMailbox/ drop(); Exec if $raw_event =~ /^#/ drop(); </Input>

<Output out_exchange> Module om_udp Host 10.2.1.22 Port 5141 Exec $SyslogFacilityValue = 2; Exec $SourceName = 'exchange_msgtrk_log'; Exec to_syslog_bsd(); </Output>

<Route exchange> Path in_exchange => out_exchange </Route>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') &gt;= 5M)) \
                file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>


EliWallic created
Replies: 2
View post »
last updated
How could I pass output of to_json() from nxlog.conf to perl file?
Hi, I have this type of input in nxlog.conf: ``` Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); to_json(); perl_call("process"); ``` My question is, how should I include that JSON output that I get from to_json() to my perl code? Should I write like this?: ``` my ( $event ) = @_; ``` Or it's only the output of parse_syslog_bsd() (as in example for xm_perl https://nxlog.co/documentation/nxlog-user-guide/xm_perl.html)? **More generally, my question is how to include JSON output that i get from to_json() to perl code of xm_perl module?**

EZ created
Replies: 1
View post »
last updated
QueryXML conditionally, based on what's installed on Windows server?
Hi all, How do I divide my NxLog configuration file into conditionals based on queries? I have an nxlog.conf that doesn't apply equally to all hosts, and NxLog fails if it has to query something that doesn't exist on a Windows server. I'd like to keep 1 nxlog.conf for all servers instead of having to maintain many confs for a variety of servers. In psuedocode: ``` if [ exists Microsoft-Windows-TerminalServices ] then * ``` However I'm having a hard time translating a lot of the "if statements" I see to NxLog query formats.. anyone have any resources or tips to guide me? Here's what I tried, obviously wrong, \ cannot be in \ ``` if ($Channel == 'Microsoft-Windows-TerminalServices-SessionBroker/Operational') * ``` Also tried: ``` if ($Channel == 'Microsoft-Windows-TerminalServices-SessionBroker/Operational') \ \ \ \ * \ \ \ ``` Thanks for your help.

wbollock created
Replies: 1
View post »
last updated
NXlog is not connecting to ElasticSearch server

I am configuring the NXLog to Elastic Search Server using om_elasticserch, but it doesnt connect

Here is the nxlog.conf file:

User root Group system Panic Soft

default values:

PidFile /opt/nxlog/var/run/nxlog/nxlog.pid CacheDir /opt/nxlog/var/spool/nxlog

ModuleDir /opt/nxlog/lib/nxlog/modules

ModuleDir /opt/nxlog/libexec/nxlog/modules SpoolDir /opt/nxlog/var/spool/nxlog

define CERTDIR /opt/nxlog/var/lib/nxlog/cert define CONFDIR /opt/nxlog/var/lib/nxlog

Note that these two lines define constants only; the log file location

is ultimately set by the LogFile directive (see below). The

MYLOGFILE define is also used to rotate the log file automatically

(see the _fileop block).

define LOGDIR /opt/nxlog/var/log/nxlog define MYLOGFILE %LOGDIR%/nxlog.log

<Extension _json> Module xm_json </Extension>

<Input file> Module im_file File '/<path/to/my/logfile/*log' # Parse log here if needed # $EventTime should be set here </Input>

<Output out> Module om_elasticsearch URL http://linuxServer:9200/_bulk FlushInterval 2 FlushLimit 100

# Create an index daily
Index       strftime($EventTime, &quot;nxlog-%Y%m%d&quot;)

# Use the following if you do not have $EventTime set
#Index      strftime($EventReceivedTime, &quot;nxlog-%Y%m%d&quot;)

</Output>

By default, LogFile %MYLOGFILE% is set in log4ensics.conf. This

allows the log file location to be modified via NXLog Manager. If you

are not using NXLog Manager, you can instead set LogFile below and

disable the include line.

LogFile %MYLOGFILE% #include %CONFDIR%/log4ensics.conf

<Extension _syslog> Module xm_syslog </Extension>

This block rotates %MYLOGFILE% on a schedule. Note that if LogFile

is changed in log4ensics.conf via NXLog Manager, rotation of the new

file should also be configured there.

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
&lt;Schedule&gt;
    Every   1 hour
    &lt;Exec&gt;
        if ( file_exists('%MYLOGFILE%') and
             (file_size('%MYLOGFILE%') &gt;= 5M) )
        {
             file_cycle('%MYLOGFILE%', 8);
        }
    &lt;/Exec&gt;
&lt;/Schedule&gt;

# Rotate our log file every week on Sunday at midnight
&lt;Schedule&gt;
    When    @weekly
    Exec    if file_exists('%MYLOGFILE%') file_cycle('%MYLOGFILE%', 8);
&lt;/Schedule&gt;

</Extension>

NXlog is starting, but it waiting to connect to ElasticSearch server, not proceeding further. Here is the log in nxlog.log. And I could not see nxlog* index patter in Kibana

2020-03-31 09:37:44 WARNING already running as gid 0 2020-03-31 09:37:44 WARNING already running as uid 0 2020-03-31 09:37:44 WARNING no routes defined! 2020-03-31 09:37:44 WARNING default route added 2020-03-31 09:37:44 INFO nxlog-4.7.4715 started 2020-03-31 09:37:44 INFO om_elasticsearch_add_reconnect_event reconnecting in 0 seconds 2020-03-31 09:37:44 INFO connecting to linuxServer:9200

Any help on this is appricaited.

Thanks in advance

Santosh


santosh.p created
Replies: 1
View post »
last updated
Hiding/Masking Confidential Information in Logs

Hi,

Can NXLog be configured to automatically hide confidential information (PII) like credit card numbers, social security numbers, etc?

I thought you could maybe use Regex but thought there might be a better way (like a module) since this is a pretty common thing to do.

Thanks in advance!!


casey1234 created
Replies: 1
View post »
last updated