I'm seeing we can add an agent https://nxlog.co/documentation/nxlog-user-guide/nxlog_manager_installation.html#nxlog_manager_agentinstall But I'm not seeing it in the downloads https://nxlog.co/products/nxlog-manager/download so, I'm not able to download the agent. Could someone help me to connect to NXLog Service from NXLog Manager to query or search.

Hello,

I am having issues setting up the config file for NXlog to pull security events from DC and send to Graylog for my school Capstone project. In the log I keep getting an error "Couldnt parse Exec block at ... couldnt parse statement at line 58, character 218 in .... sytnax error, unexpected )". I utilized a config setup from NXlog documentation and adjusted for my needs. I am not sure what is wrong as the exec block looks proper to me. Here is what I have in my config file:

define SecuritySrc Microsoft-Windows-Security-Auditing

<Input DC_events> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Security">*[System[Provider[ @Name='%SecuritySrc%']]] </Select> </Query> </QueryList> </QueryXML> <Exec> if not (defined($SourceName) and ($EventID IN (%SecurityIDs%) and $SourceName == "%SecuritySrc%")) <------ line 58 drop(); </Exec> </Input>

I appreciate any help on this!

Hi Team,

We are planning to upgrade nxlog enterprise edition from 3.x to 5.x version.

While planning we were looking at the compatibility matrix of nxlog collector , agent , elastic search and kibana.

to be precise what version of Elastic search will be supoourted with Nxlong v5.x.

Regards Anjan Kumar Tripathy

Hello everyone,

I have been noticing some seemingly unnecessary logs being sent to our SIEM when using nxlog for Windows DNS syslog forwarding.

Here is my current configuration:

nxlog.conf

define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension syslog> Module xm_syslog </Extension>

<Input in> Module im_file File 'C:\Windows\Sysnative\dns\dns.log' SavePos TRUE ReadFromLast TRUE PollInterval 1 Exec $Message = $raw_event; $SyslogFacilityValue = 22; </Input>

<Output out1> Module om_tcp Host 10.5.1.3 Port 1470 Exec to_syslog_bsd(); </Output>

<Route 1> Path in => out1 </Route>

<Extension _fileop> Module xm_fileop </Extension>

Windows DNS Debug Logging Configuration:

Log packets for debugging: check Packet direction: Outgoing: check Incoming: check

Transport protocol: UDP: check TCP: check

Packet contents: Queries/Transfers: check Updates: check Notifications: not checked

Packet Type: Request: check Response: check

Other options: Log unmatched incoming response packets: not checked Details: not checked Filter packets by IP address: not checked

File path and name: c:\windows\system32\dns\dns.log Maximum size (bytes): 500000000

The result is that all of the necessary logs are being sent to our SIEM, but there are a bunch of unnormalized logs being sent that I do not quite understand.

Here are examples:

<181>(date/time) (System Name) As an example, for a server named DC1 on March 19th, 14:39. <181>Mar 19 14:39:17 DC1

A ton of these are being sent, close to 1 for each normal DNS log. As you could imagine this is using a lot of our event log storage. Does anyone know what is causing this? Or what this <181> is referring to?

Thank you!

Hi, we have a test setup with one Fortigate (v6.4.4) and we wanted to use tcp for log collection. We can see the Forti sending the packets (tcpdump) to our NXLog-Server and we can see them arriving (tcpdump) but the packets are not being processed by the NXLog. Using udp evertyhing works fine.

The config on the Forti is standard:

config log syslogd setting
    set status enable
    set server "10.0.172.41"
    set mode reliable
    set port 2570
end

If we switch to mode legacy-reliable we can see log entries but the look rubbish. On the NXLog we use im_tcp as input and we route it with om_file into a text file. Pretty straight forward but it does not work.

Has anyone ever used Fortinet tcp syslog with NXLog?

Regards Hardy

Hi All,

I've setup NXLog to read an application log file and forward the raw_event to Graylog manager. It seems initially when I start NXLog everything works correctly and the Headerline correctly breaks the log sections but after a while it seems to ignore the headerline and individual lines start coming through to graylog. I've loaded the log into regex testers and confirmed that the headerline is correctly recognised, is there anything else I can try? Can I confirm this is an NXLog issue instead of Graylog in some way?

<Extension proforma_email_parse> Module xm_multiline HeaderLine /\d\d\d\d/\d\d/\d\d \d\d:\d\d:\d\d.\d\d\d .... ........ ----------------------------------------------------/ </Extension>

<Input proforma_email> Module im_file File "D:\Program Files\ProformaEmailService\Logs\ProformaEmail_*.txt" SavePos true ReadFromLast true Exec $Message = $raw_event; InputType proforma_email_parse </Input>

Thanks

Hi All,

I need to change this format 'C:\logfile\directory1\directory2'

into '\logfile\directory1\directory2\'

using regex and the command

exec $directory =

this is to be able to set the destination folder the same as the location folder recursively

Hi all,

I'm trying to integrate the Exchange Logs to NXLog CE using the example config from https://nxlog.co/documentation/nxlog-user-guide/exchange.html which does'nt work. After reading through the forums I have come up with the following nxlog.conf which still doesn't work producing the errors at the end of this post. Is there any comprehensive guide to setup it up?

---------------- nxlog.conf ----------------------

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log define BASEDIR E:\Exchange Server LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension> <Extension csv_parser> Module xm_csv Fields date-time, client-ip, client-hostname, server-ip, server-hostname,
source-context, connector-id, source, event-id,
internal-message-id, message-id, network-message-id,
recipient-address, recipient-status, total-bytes, recipient-count,
related-recipient-address, reference, message-subject,
sender-address, return-path, message-info, directionality,
tenant-id, original-client-ip, original-server-ip, custom-data,
transport-traffic-type, log-id, schema-version,
session, source, Destination, direction, description,
session-id, sequence-number,
local-endpoint, remote-endpoint, event, data, context Exec rename_field('sender-address', 'sender_address'); $sender_address = lc($sender_address); Exec rename_field('date-time', 'date_time'); $date_time = lc($date_time);

</Extension>

<Input messagetracking> Module im_file File '%BASEDIR%\TransportRoles\Logs\MessageTracking\MSGTRK*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input connectivity> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\Connectivity\CONNECTLOG*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input smtp_receive> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive\RECV*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Input smtp_send> Module im_file File '%BASEDIR%\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend\SEND*.LOG' <Exec> if $raw_event =~ /^(\xEF\xBB\xBF)?(date-time,|#)/ drop(); else { csv_parser->parse_csv(); $EventTime = parsedate($date_time); } </Exec> </Input>

<Extension _json> Module xm_json </Extension>

<Output out> Module om_http URL http://kibanaip:9200 ContentType application/json <Exec> set_http_request_path(strftime($EventTime, "/nxlog-%Y%m%d/" + $SourceModuleName)); rename_field("timestamp", "@timestamp"); to_json(); </Exec> </Output>

---------------- nxlog.conf ----------------------

Errors: 2021-03-05 11:46:03 WARNING no routes defined! 2021-03-05 11:46:03 WARNING not starting unused module messagetracking 2021-03-05 11:46:03 WARNING not starting unused module connectivity 2021-03-05 11:46:03 WARNING not starting unused module smtp_receive 2021-03-05 11:46:03 WARNING not starting unused module smtp_send 2021-03-05 11:46:03 WARNING not starting unused module out 2021-03-05 11:46:03 INFO nxlog-ce-2.10.2150 started

Thank You in advance! Gregory

Hi, we have windows event forwarding configured with a lot of subscripts and filtering already configured across a deployemnt of Windows servers.

I would like to use NXLOG EE to install on our two Windows event collectors to forward these events off to a SIEM. Looking at the documentation seems I need to setup a Linux server to act as the WEC for this purpose?

I've got a simple config listening on 514 UDP/TCP and forwarding everything received out to another server for ingest. One of the things I've been having trouble figuring out is how to do simple event aggregation for firewall logs. Ideally it would aggregate over a time window and append the message with a new field containing the count of messages.

I know something like this used to be done via module "pm_norepeat", but I think this is being deprecated, and I'm not aware that it is capable of appending message count to the original message. It seems this should somehow be done using variables going forward.

To add to the complexity, we have two separate firewall types within our environment, (Cisco ASA's and Palo's). Greatly appreciate if anyone can point me in the right direction.

Hello NXLog user,

I'm new to Nxlog and I'm looking to send the contents of files (*.log) located in a single directory to a Syslog Watcher server (snmpsoft). A trace can be on one or multiple lines but each start of a new trace starts with the time.

Here is an example of a trace on one line (columns => time type severity thread file function content) :

17:13:00.000 APP__ WARNING 1aa20b54100 <capp.cpp:56> [[CApp::ExistInDirectory]] File 'tool.exe' not found in C:/prog

Here is an example of multiple traces and one on multiple line (columns => time type severity thread file function content) :

10:57:25.924 SQL__ ERROR__ 20d962b1270 <csqlquery.cpp:292> [[CSqlQuery::ErrQuery]] [cnx2238] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Violation de la contrainte PRIMARY KEY « PK__T__3214EC27053CB3F2 ». Impossible d'insérer une clé en double dans l'objet « B.T ». Valeur de clé dupliquée : (44422). [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]L'instruction a été arrêtée. QODBC3: Impossible d'exécuter l'instruction

• Source query * ..
• Query with values * ...

10:57:25.926 APP__ ERROR__ 20d962b1270 <chistosystemautomate.cpp:67> [[CHistoSystemAutomate::MsgReceivedSystem]] { "code": 500004, "details": "", "message": "Impossible d'exécuter la requête SQL." }

Nxlog.conf that I tried to do

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %ROOT%\data\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Input messages> Module im_file File "C:\Users\toto\Documents\nxlog*.log" Exec $Message = $raw_event; </Input>

<Output tcp> Module om_tcp Host 0.0.0.0 Port 514 Exec to_syslog_bsd(); </Output>

<Output udp> Module om_udp Host 0.0.0.0 Port 514 Exec to_syslog_bsd(); </Output>

<Route messages_to_tcp> Path messages => tcp </Route>

<Route messages_to_udp> Path messages => udp </Route>

Can you help me to write the conf file.

A big thank you in advance

Christophe

We probably found a bug in nxlog-5.2.6388. We use om_http to transfer logs to the server. If there are some new logs in our log file, the nxlog on start connects to server, makes ssl handshake and sucessfully sends logs. BUT if there is no new log entry during the nxlog startup, the om_http connects to the server while NOT doing SSL handshake (only tcp handshake), after some time the web server closes the connection (sends FIN+ACK because it expects SSL handshake to occur during its timeout), after that the nxlog sends ACK and client hello. This seems to be very buggy behaviour as nxlog does not perform ssl handshake until the server tries to close the connection BUT ALWAYS tries to after the server initiates TCP closing. After adding some new line to the log file, the nxlog succefully connects to web server, makes ssl handshake and keeps the connection. Could you confirm the bug?

So I'm working on moving our logging away from SolarWinds and into nxlog. We like to punt off our logs to Splunk so that our security department can create dashboards and such for the events they care about. I've just about got the output perfect, besides ONE random extra comma and I can't figure out where it's coming from.

Here is my config:

define ROOT C:\Program Files (x86)\nxlog

define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LOGFILE %ROOT%\data\nxlog.log LogLevel INFO

<Extension syslog> Module xm_syslog </Extension>

define MonitoredEventIDs 1100, 1102, 1104, 4608, 4609, 4624, 4625, 4634, 4647, 4648,
4656, 4658, 4659, 4660, 4661, 4662, 4663, 4664, 4670, 4672,
4673, 4724, 4719, 4720, 4722, 4723, 4724, 4725, 4726, 4727,
4728, 4729, 4731, 4732, 4733, 4734, 4735, 4737, 4738, 4739,
4740, 4766, 4767, 4768, 4776, 4781, 4801, 4825, 4907, 5136,
5137, 5139, 5141, 5145, 6416, 13002, 13003, 18500, 18502, 307

<Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Security"></Select> <Select Path="Microsoft-Windows-PrintService/Operational"></Select> </Query> </QueryList> </QueryXML> <Exec> if $EventID NOT IN (%MonitoredEventIDs%) drop(); </Exec> </Input>

<Input GFI> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="GFI EndPointSecurity">*</Select> </Query> </QueryList> </QueryXML> </Input>

<Output out> Module om_udp Host 10.1.0.1 Port 514 Exec to_syslog_snare();$raw_event = replace($raw_event, "\t", ","); </Output>

<Route> Path eventlog, GFI => out </Route>

That creates this beautiful output in Splunk, except for the random extra comma before 'An account was logged off.' This single extra comma is throwing off the parsing of the logs that our customer receives and I need to get rid of it... Any advice?

Mar 1 21:58:26 mycomputer.com MSWinEventLog,1,Security,79,Mon Mar 01 14:58:26 2021,4634,Microsoft-Windows-Security-Auditing,N/A,N/A,Success Audit,mycomputer.com,Logoff,,An account was logged off. Subject: Security ID: S-1-5-21-2294171146-2094350030-1588952898-500 Account Name: pcgroup Account Domain: mycomputer Logon ID: 0x2D069A2F Logon Type: 2 This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.,1023042

I am having an issue with nxlog manager docker version starting up. After running docker-compose up and waiting, the webpage is never accessible. When looking in the logs I see nxlog-manager constantly exiting with code 1 and restarting. Below are some of the logs, anyone have ideas on this?

nxlog-manager_1 | 2021-02-25 01:48:39.757:INFO:oejr.Runner:main: Runner nxlog-manager_1 | 2021-02-25 01:48:39.947:INFO:oejs.Server:main: jetty-9.0.7.v20131107 nxlog-manager_1 | SLF4J: Class path contains multiple SLF4J bindings. nxlog-manager_1 | SLF4J: Found binding in [jar:file:/opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class] nxlog-manager_1 | SLF4J: Found binding in [jar:file:/opt/nxlog-manager/lib/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class] nxlog-manager_1 | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. nxlog-manager_1 | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] nxlog-manager_1 | 2021-02-25 01:48:52.957:INFO:/:main: 2 Spring WebApplicationInitializers detected on classpath nxlog-manager_1 | 2021-02-25 01:48:53.429:INFO:/:main: Initializing Spring root WebApplicationContext nxlog-manager_1 | 2021-02-25 01:49:04.694:INFO:/:main: Initializing Spring FrameworkServlet 'dispatcher' nxlog-manager_1 | 2021-02-25 01:56:28.869:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@636be97c{/nxlog-manager,[file:/opt/nxlog-manager/webapps/nxlog-manager/, jar:file:/opt/nxlog-manager/webapps/nxlog-manager/WEB-INF/lib/springfox-swagger-ui-2.9.2.jar!/META-INF/resources/],AVAILABLE}{/nxlog-manager/} nxlog-manager_1 | 2021-02-25 01:56:28.870:WARN:oejsh.RequestLogHandler:main: !RequestLog nxlog-manager_1 | 2021-02-25 01:56:28.881:INFO:oejs.ServerConnector:main: Started ServerConnector@31c1dce1{HTTP/1.1}{0.0.0.0:9090} nxlog-manager_1 | log4j:WARN No appenders could be found for logger (com.nxsec.log4ensics.dbmanager.common.server.util.ssl.SslContextFactory). nxlog-manager_1 | log4j:WARN Please initialize the log4j system properly. nxlog-manager_1 | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. nxlog-manager_1 | 2021-02-25 01:56:29.023:WARN:oejuc.AbstractLifeCycle:main: FAILED SslContextFactory@73ea0918(null,null): java.io.FileNotFoundException: /opt/nxlog-manager/conf/jetty9-cert.pem (No such file or directory) nxlog-manager_1 | java.io.FileNotFoundException: /opt/nxlog-manager/conf/jetty9-cert.pem (No such file or directory) nxlog-manager_1 | at java.io.FileInputStream.open0(Native Method) nxlog-manager_1 | at java.io.FileInputStream.open(FileInputStream.java:195) nxlog-manager_1 | at java.io.FileInputStream.<init>(FileInputStream.java:138) nxlog-manager_1 | at com.nxsec.log4ensics.dbmanager.common.server.util.ssl.SslContextFactory.initializeKeyStore(SslContextFactory.java:39) nxlog-manager_1 | at com.nxsec.log4ensics.dbmanager.common.server.util.ssl.SslContextFactory.doStart(SslContextFactory.java:56)

I'm trying to collect EventID 4624 and 4634 for Logon Type 10, to store RDP access to my 2 Domain Controllers.

• same Windows version (2012 R2)
• same audit config in windows
• same NXlog version installed (community edition)
• same nxlog.conf file

My issue:

• from DC 1 I'm getting both 4624 and 4634
• from DC 2 I'm getting only 4634 :(

Additional info:

• in windows Event Viewer I have my 4624 in DC2 ...
• reinstalled nxlog
• rebooted my DC
• DEBUG level in nxlog but no evidence of problem

Thx a lot for your support, Benno

My team is currently experiencing an issue with duplicate logs being produced in NXLog's outbound syslog feed. NXLog is reading from a flat file and sending a syslog feed to another machine for processing. Running a TCPdump for the incoming data on that second machine shows that multiple copies of the same log are being sent by NXlog. Moreover, the duplicates are not sent sequentially, but instead are sent almost exactly 5 minutes apart as show by five minute gaps between the "EventReceivedTime" syslog header values. We have reviewed the flat file and confirmed that it does not produce the duplicates, and thus must be related to the configuration of NXLog and the way it reads the flat file. I'd appreciate any insight others might have on what is causing this problem.

The NXLog configuration is as follows:

Panic Soft #NoFreeOnExit TRUE

define ROOT D:\NX Log\Program Files define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

<Input in> Module im_file File "D:[Ingested File Path].cef" InputType LineBased SavePos TRUE ReadFromLast TRUE PollInterval 1 </Input>

<Processor norepeat> Module pm_norepeat </Processor>

<Output out> Module om_ssl Host [Second Machine IP] Port 515 Exec to_syslog_ietf();

Allows using self-signed certificates

AllowUntrusted  FALSE

Certificate from the peer host

CAFile          D:\[CA File Path].pem

Certificate file

CertFile        D:\[Certificate File Path].pem

Keypair file

CertKeyFile     D:\[Key File Path].pem

</Output>

<Route sitecollector> Path in => norepeat => out </Route>

Is there a package to install the 'help' button's content for NXLog-Manager's UI? When any of the help buttons are clicked in any section, an error is produced. Looking where the content should be, doesn't reflect the links the button is attempting to access. Suggestions?

Example error text: (clicking 'Help' from 'Home' page): Help page not found: en/dashboard.html#nxlog_manager_dashboard (clicking 'Help' from 'Agent List' tab): Help page not found: en/agents.html#nxlog_manager_agent_list

CLI listing contents of the help/en directory: [XXXXX@XXXXXXXX en]$ pwd /opt/nxlog-manager/webapps/nxlog-manager/help/en [XXXXX@XXXXXXXX en]$ ls ch01.html ch02.html ch03.html ch04.html ch05.html ch06.html ch07.html ch08.html ch09.html ch10.html ch11.html ch12.html ch13.html ch14.html images index.html [XXXXX@XXXXXXXX en]$

Hi Team,

Hope all are well.

I am new to nxlog and trying to plan for an upgrade from vUpgrade from nxlog-ce-2.9 to latest 5.x.

Can someone please help us in below queries.

1. Can we upgrade to the latest version
2. IS upgrading to latest version is recomended?
3. can we directly upgrade it or its a multihop upgrade from v3.x -> 4.x -> 5.x
4. is there any link to go through the upgrade process in windows environment?

Thanks in advance and any info would be helpfull

Regards Anjan Kumar Tripathy