I'm trying to query a local Sybase database using NXLog and then write the output to a file. So far I'm unable to connect to the database. NXLog is saying that there are no drivers present. I'm running this on Centos7. I'm not seeing any database drivers available. Do I need to download these drivers separately? What's the name of the correct driver for Sybase? Where would I find the correct file? ``` 2020-03-09 12:02:18 INFO nxlog-ce-2.10.2150 started 2020-03-09 12:02:18 ERROR failed to open tmp/output;No such file or directory 2020-03-09 12:02:18 ERROR dbi_initialize failed, no drivers present? ``` Config below: ``` Module im_dbi Driver Sybase Option host 127.0.0.1 Option username ******** Option password ******** Option dbname ******** SQL SELECT * from ******* Module om_file File "tmp/output.out" Path dbi => file ``` Any help would be appreciated. Thanks in advance!!

For some reason I realized NXlog wasn't sending logs to graylog (after previously doing so flawlessly)

I went to the Nxlog log and found this just before it stopped responding:

What does this mean? I can't find anything online, but I know the udp arguments work because restarting nxlog works fine.

2020-03-09 08:29:36 ERROR om_udp apr_socket_send failed; An invalid argument was supplied.

However, Graylog received a message 20ms later from that machine (the last message that was sent before nxlog went offline)

Any ideas?

We are using a graylog server in hopes to capture 2 things (Logons and Disk Errors). NXlog is forwarding most logon attempts, but not all of them for some reason. NXlog is not forwarding any Disk error logs.

Here is my config:

Panic Soft #NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE%

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data

<Extension _syslog> Module xm_syslog </Extension>

<Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension>

<Extension _exec> Module xm_exec </Extension>

<Extension _fileop> Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
    Every   1 hour
    Exec    if (file_exists('%LOGFILE%') and \
               (file_size('%LOGFILE%') >= 5M)) \
                file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
    When    @weekly
    Exec    if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>

</Extension>

define LogonEventIds 4648

<Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='Security'>*</Select> </Query> </QueryList> </QueryXML> <Exec> if $EventID NOT IN (%LogonEventIds%) drop(); </Exec> </Input>

define DiskEventIds 9, 11, 50, 51, 54, 55, 57, 129, 1066, 6008

<Input diskcheck> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='System'>*</Select> </Query> </QueryList> </QueryXML> <Exec> if $EventID NOT IN (%DiskEventIds%) drop(); </Exec> </Input>

<Output udpLogon> Module om_udp Host 10.0.0.220 Port 1517 </Output>

<Output udpDisk> Module om_udp Host 10.0.0.220 Port 1518 </Output>

<Route 1> Path eventlog => udpLogon </Route>

<Route 2> Path diskcheck => udpDisk </Route>

I don't know what the issue is. I am using tcpdump on the graylog server and am not receiving anything on that port (1518) despite event viewer showing several logs with 129 and 55 EventIDs.

Any help would be appreciated. Yes, the port is open.

Hi Team,

I have installed the NXlog community edition (nxlog-ce-2.10.2150.msi) on our windows server and I am trying to collect the Firewall (ASA) logs in windows server through NXlog.

I have used the following configuration but I am not receiving any logs. Can you help me on this

<Extension json>
Module xm_json
</Extension>
##Extension to format the message in syslog format
<Extension syslog>
Module xm_syslog
</Extension>

########## INPUTS ###########

<Input in_syslog_tcp>
Module im_tcp
Host 0.0.0.0
Port 1514
Exec parse_syslog();
</Input>


############ OUTPUTS ##############

<Output file>
Module om_file
File "C:\\test\\asa.log"
Exec to_syslog_ietf();
</Output>

<Route file>
Path in_syslog_tcp => file
</Route>

Hi,

I am trying to collect Windows DNS debug logs with Nxlog xm_multiline. I reference below link: Parsing Detailed DNS Logs With Regular Expressions (https://nxlog.co/documentation/nxlog-user-guide/windows-dns-server.html#parsing-detailed)

But, Windows DNS Debug Logs includes Traditional Chinese characters, it won't let me combine multiline into one log, What is correct "HEADER_REGEX" that should I use?

DNS Debug Logs sample is (I beleive problem is 上午, By the way, 上午=AM and 下午=PM): 2020/3/6 上午 11:58:01 0E80 PACKET 000001D80FE9BD40 UDP Snd 10.0.35.101 a3f5 R Q [8081 DR NOERROR] A (5)e3998(1)d(10)akamaiedge(3)net(0) UDP response info at 000001D80FE9BD40 Socket = 724 Remote addr 10.0.35.101, port 56423 Time Query=283057, Queued=283057, Expire=283060 Buf length = 0x0200 (512) Msg length = 0x0038 (56) Message: XID 0xa3f5 Flags 0x8180 QR 1 (RESPONSE) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 1 Z 0 CD 0 AD 0 RCODE 0 (NOERROR) QCOUNT 1 ACOUNT 1 NSCOUNT 0 ARCOUNT 0 QUESTION SECTION: Offset = 0x000c, RR count = 0 Name "(5)e3998(1)d(10)akamaiedge(3)net(0)" QTYPE A (1) QCLASS 1 ANSWER SECTION: Offset = 0x0028, RR count = 0 Name "C00Ce3998(1)d(10)akamaiedge(3)net(0)" TYPE A (1) CLASS 1 TTL 20 DLEN 4 DATA 96.7.252.200 AUTHORITY SECTION: empty ADDITIONAL SECTION: empty

Nxlog configuration sample is: define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension _charconv> Module xm_charconv AutodetectCharsets BIG-5, utf-8, utf-16, utf-32, iso8859-2 </Extension>

<Extension gelf> Module xm_gelf </Extension>

define EVENT_REGEX /(?x)(?<Date>\d+(?:/\d+){2})\s
(?<Time>\d+(?::\d+){2})\s
(?<ThreadId>\w+)\s+
(?<Context>\w+)\s+
(?<InternalPacketIdentifier>[[:xdigit:]]+)\s+
(?<Protocol>\w+)\s+
(?<SendReceiveIndicator>\w+)\s
(?<RemoteIP>[[:xdigit:].:]+)\s+
(?<Xid>[[:xdigit:]]+)\s
(?<QueryType>\s|R)\s
(?<Opcode>[A-Z]|?)\s
(?<QFlags>[(.?)])\s+
(?<QuestionType>\w+)\s+
(?<QuestionName>.)\s+
(?<LogInfo>.+)\s+.+=\s
(?<Socket>\d+)\s+ Remote\s+ addr\s
(?<RemoteAddr>.+),\sport\s
(?<PortNum>\d+)\s+Time\sQuery=
(?<TimeQuery>\d+),\sQueued=
(?<Queued>\d+),\sExpire=
(?<Expire>\d+)\s+.+(
(?<BufLen>\d+))\s+.+(
(?<MsgLen>\d+))\s+Message:\s+
(?<Message>(?s).*)/

define HEADER_REGEX /(?x)(?<Date>\d+(?:/\d+){2})\s
(?<AMPM>\x{e4}\x{b8}\x{8a}\x{e5}\x{8d}\x{88})\s
(?<Time>\d+(?::\d+){2})\s
(?<ThreadId>\w+)\s+
(?<Context>\w+)\s+
(?<InternalPacketIdentifier>[[:xdigit:]]+)\s+
(?<Protocol>\w+)\s+
(?<SendReceiveIndicator>\w+)\s
(?<RemoteIP>[[:xdigit:].:]+)\s+
(?<Xid>[[:xdigit:]]+)\s
(?<QueryType>\s|R)\s
(?<Opcode>[A-Z]|?)\s
(?<QFlags>[(.?)])\s+
(?<QuestionType>\w+)\s+
(?<QuestionName>.)/

<Extension multiline> Module xm_multiline HeaderLine %HEADER_REGEX% </Extension>

<Input windnsdetaillog> Module im_file File 'C:\dns.log' Exec convert_fields("BIG-5", "utf-8"); InputType multiline Exec if $raw_event =~ /(\d+)/(\d+/\d+)\s(上午\s)(\d+:\d+:\d+\s)((.|\n))/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'AM ' + $5; Exec if $raw_event =~ /(\d+)/(\d+/\d+)\s(下午\s)(\d+:\d+:\d+\s)((.|\n))/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'PM ' + $5; <Exec> if $raw_event =~ %EVENT_REGEX% { $EventTime = parsedate($Date + " " + $Time + " " + $AMPM); delete($Date); delete($Time); } </Exec> </Input>

<Input wineventin> Module im_msvistalog </Input>

<Output windnsdetaillogout> Module om_tcp Host 192.168.11.3 Port 12198 OutputType GELF_TCP </Output>

<Output wineventout> Module om_udp Host 192.168.11.3 Port 12196 OutputType GELF </Output>

<Route 1> Path wineventin => wineventout </Route>

<Route 2> Path windnsdetaillog => windnsdetaillogout </Route>

Hello NXLog community! I was hoping you can help me with the problem I've been dealing with. I'm trying to configure NXLog to collect Sharepoint audit logs using a PowerShell script following the offical documentation here: I have enabled Sharepoint audit logging, configured the PowerShell script and the NXLog input, but when I'm trying to run it it does not work. Can you please help me figure out what I'm doing wrong here? Here is what I'm getting in the log file: 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; The local farm is not accessibl; (right here) ------^; [The local farm is not accessible. Cmdlets with FeatureDependencyId are not registered.] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; C:\Audit\auditlog.ps1 : An unha; (right here) ------^; [C:\Audit\auditlog.ps1 : An unhandled exception occurred!] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; + CategoryInfo : NotSp; (right here) ------^; [ + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid string in json text.; tion; (right here) ------^; [ tion] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; + FullyQualifiedErrorId : Micro; (right here) ------^; [ + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid string in json text.; n,auditlog.ps1; (right here) ------^; [ n,auditlog.ps1] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, parse error: premature EOF; ; (right here) ------^; [ ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR Module audit_powershell got EOF, process exited? Here's my config file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO Module xm_fileop When @daily Exec file_cycle('%ROOT%\data\nxlog.log', 7); Module xm_gelf # Avoid truncation of the short_message field to 64 characters. ShortMessageLength 65536 Module xm_json Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "-ExecutionPolicy" Arg "Bypass" Arg "-NoProfile" Arg "-File" Arg "C:\Audit\auditlog.ps1" parse_json(); $EventTime = parsedate($EventTime); Module om_tcp Host 192.168.98.5 Port 12208 OutputType GELF_TCP # These fields are needed for Graylog $gl2_source_collector = '3a5aa0c9-aba3-4384-8691-43ed7d1ebbab'; $collector_node_id = 'Scooby'; Path audit_powershell => gelf And the PowerShell script I've created: # This script can be used with NXLog to fetch Audit logs via the SharePoint # API. See the configurable options below. Based on: # #Requires -Version 3 # The timestamp is saved to this file for resuming. $CacheFile = 'C:\Audit\nxlog_sharepoint_auditlog_position.txt' # The database is queried at this interval in seconds. $PollInterval = 10 # Allow this many seconds for new logs to be written to database. $ReadDelay = 30 # Use this to enable debug logging (for testing outside of NXLog). #$DebugPreference = 'Continue' ################################################################################ # If running 32-bit on a 64-bit system, run 64-bit PowerShell instead. if ($env:PROCESSOR_ARCHITEW6432 -eq "AMD64") { Write-Debug "Running 64-bit PowerShell." &"$env:WINDIR\SysNative\WindowsPowerShell\v1.0\powershell.exe" ` -NonInteractive -NoProfile -ExecutionPolicy Bypass ` -File "$($myInvocation.InvocationName)" $args exit $LASTEXITCODE } Add-PSSnapin "Microsoft.SharePoint.Powershell" -ErrorAction Stop # Return description for event function Event-Description { param( $entry ) switch ($entry.Event) { AuditMaskChange {"The audit flags are changed for the audited object."} ChildDelete {"A child of the audited object is deleted."} ChildMove {"A child of the audited object is moved."} CheckIn {"A document is checked in."} 'Copy' {"The audited item is copied."} Delete {"The audited object is deleted."} EventsDeleted {"Some audit entries are deleted from SharePoint database."} 'Move' {"The audited object is moved."} Search {"The audited object is searched."} SecGroupCreate {"A group is created for the site collection (this action "` + "also generates an Update event)."} SecGroupDelete {"A group on the site collection is deleted."} SecGroupMemberAdd {"A user is added to a group."} SecGroupMemberDelete {"A user is removed from a group."} SecRoleBindBreakInherit {"A subsite's inheritance of permission level "` + "definitions (that is, role definitions) is severed."} SecRoleBindInherit {"A subsite is set to inherit permission level "` + "definitions (that is, role definitions) from its parent."} SecRoleBindUpdate {"The permissions of a user or group for the audited "` + "object are changed."} SecRoleDefCreate {"A new permission level (a combination of permissions "` + "that are given to people holding a particular role for the site "` + "collection) is created."} SecRoleDefDelete {"A permission level (a combination of permissions that "` + "are given to people holding a particular role for the site "` + "collection) is deleted."} SecRoleDefModify {"A permission level (a combination of permissions that "` + "are given to people holding a particular role for the site "` + "collection) is modified."} Update {"An existing object is updated."} CheckOut {"A document is checked out."} View {"The object is viewed by a user."} ProfileChange {"Change in a profile that is associated with the object."} SchemaChange {"Change in the schema of the object."} Undelete {"Restoration of an object from the Recycle Bin."} Workflow {"Access of the object as part of a workflow."} FileFragmentWrite {"A File Fragment has been written for the file."} Custom {"Custom action or event."} default {"The event description could not be determined."} } } # Get audit data from $site in range $start to $end. Timestamps should use # seconds precision only. A record with timestamp equal to $start time is # included in output; a record with timestamp equal to $end time is not. function Get-Audit-Data { param( $site, $start, $end ) Write-Debug "Getting audit log for $site.Url from $start to $end" $query = New-Object -TypeName Microsoft.SharePoint.SPAuditQuery($site) $query.setRangeStart($start.AddSeconds(-1)) $query.setRangeEnd($end) $coll = $site.Audit.GetEntries($query) $root = $site.RootWeb for ($i=0; $i -le ($coll.Count)-1 ; $i++) { # Get the entry item from the collection $entry = $coll.Item($i) # Find the current user name foreach($User in $root.SiteUsers) { if($entry.UserId -eq $User.Id) { $UserName = $User.UserLogin } } # Find the item name foreach($List in $root.Lists) { if($entry.ItemId -eq $List.Id) { $ItemName = $List.Title } } # Create hash table $record = @{ # AuditData table fields SiteID = $entry.SiteId; ItemID = $entry.ItemId; ItemType = $entry.ItemType; UserID = $entry.UserId; AppPrincipalID = $entry.AppPrincipalId; MachineName = $entry.MachineName; MachineIP = $entry.MachineIP; DocLocation = $entry.DocLocation; LocationType = $entry.LocationType; EventTime = ($entry.Occurred.ToString('o') + "Z"); Event = $entry.Event; EventName = $entry.EventName; EventSource = $entry.EventSource; SourceName = $entry.SourceName; EventData = $entry.EventData; # Additional fields ItemName = $ItemName; Message = Event-Description $entry; SiteURL = $site.Url; UserName = $UserName; } # Return record as JSON $record | ConvertTo-Json -Compress | Write-Output } } # Get position timestamp from cache file. On first run, create file using # current time. function Get-Position { param( $file ) Try { if (Test-Path $file) { $time = (Get-Date (Get-Content $file -First 1)) $time = $time.ToUniversalTime() $time = $time.AddTicks(-($time.Ticks % 10000000)) } else { $time = [System.DateTime]::UtcNow $time = $time.AddTicks(-($time.Ticks % 10000000)) Save-Position $file $time } return $time } Catch { Write-Error "Failed to read timestamp from position file." exit 1 } } # Save position timestamp to cache file. function Save-Position { param( $file, $time ) Try { Out-File -FilePath $file -InputObject $time.ToString('o') } Catch { Write-Error "Failed to write timestamp to position file." exit 1 } } # Main Try { $start = Get-Position $CacheFile Write-Debug "Got start time of $($start.ToString('o'))." $now = [System.DateTime]::UtcNow $now = $now.AddTicks(-($now.Ticks % 10000000)) Write-Debug "Got current time of $($now.ToString('o'))." $diff = ($now - $start).TotalSeconds # Check whether waiting is required to comply with $ReadDelay. if (($diff - $PollInterval) -lt $ReadDelay) { $wait = $ReadDelay - $diff + $PollInterval Write-Debug "Waiting $wait seconds to start collecting logs." Start-Sleep -Seconds $wait } # Repeatedly read from the audit log while($true) { Write-Debug "Using range start time of $($start.ToString('o'))." $now = [System.DateTime]::UtcNow $now = $now.AddTicks(-($now.Ticks % 10000000)) $end = $now.AddSeconds(-($ReadDelay)) Write-Debug "Using range end time of $($end.ToString('o'))." $sites = Get-SPSite -Limit All foreach($site in $sites) { Get-Audit-Data $site $start $end } Write-Debug "Saving position timestamp to cache file." Save-Position $CacheFile $end Write-Debug "Waiting $PollInterval seconds before reading again." Start-Sleep -Seconds $PollInterval $start = $end } } Catch { Write-Error "An unhandled exception occurred!" exit 1 }

Good evening!

I have a probem with configuring im_odbc module to connect to the Oracle database from unix.

So...

As I understood (maybe I'm wrong somewhere), this module don't work "from the box" and for correctrly working I need to set up oracle instant client wich provide me a driver, odbc.ini, odbcinst.ini files. Also I created tnsnames.ora file. odbc.ini, odbcinst.ini and tnsnames.ora configured correctly. I trying to test them with isql and it's work fine.

I tried to differently configured ConnectionString parameter in configuration file, but it's did not work for me.

/etc/odbcinst.ini:

[libsqora.so.12.1] Description = Oracle ODBC driver for Oracle 12c Driver = /usr/lib/oracle/12.2/client64/lib/libsqora.so.12.1 Setup = FileUsage = CPTimeout = CPReuse

/etc/odbc.ini:

[ODBC Data Sources] OracleODBC = Oracle ODBC driver for Oracle 12c

[OracleODBC] Application Attributes = T Attributes = W BatchAutocommitMode = IfAllSuccessful BindAsFLOAT = F CloseCursor = F DisableDPM = F DisableMTS = T Driver = libsqora.so.12.1 DSN = OracleODBC EXECSchemaOpt = EXECSyntax = T Failover = T FailoverDelay = 10 FailoverRetryCount = 10 FetchBufferSize = 64000 ForceWCHAR = F Lobs = T Longs = T MaxLargeData = 0 MetadataIdDefault = F QueryTimeout = T ResultSets = T ServerName = DB.ETALON SQLGetData extensions = F Translation DLL = Translation Option = 0 DisableRULEHint = T UserID = XXXX Password = XXXX StatementCache=F CacheBufferSize=20 UseOCIDescribeAny=F SQLTranslateErrors=F MaxTokenSize=8192 AggregateSQLType=FLOAT

tnsnames.ora

DB.ETALON = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = db-etalon.example.com)(PORT = 1530)) ) (CONNECT_DATA = (SERVICE_NAME = etalon) ) )

I don’t understand how to correctly set ConnectionString. Maybe someone had odbc setup experience.

Hello all, I am having an issue reading in a csv file and converting it out up to a log server. The first line/event in the csv gets parsed and converted correctly but then the second line/event doesn't get parsed and is converted to the same line as the first event. I am trying to have it read in the csv file (being exported from sccm for scep alerts) and convert it to syslog and send it up to log server. Please find all my configs below: **NXlog conf (Not pasting full config file)** ######################################## # Application Configuration Includes # ######################################## ## Uncomment additional input modules below if desired. ## Additional configuration may be required for each application in its conf file. # include %ROOT%\conf\ms_dhcpv4.conf ## Must add "MS_DHCPv4" as INPUT to route below. # include %ROOT%\conf\ms_dhcpv6.conf ## Must add "MS_DHCPv6" as INPUT to route below. # include %ROOT%\conf\ms_scep.conf ## Must add "ms_scep" as INPUT to route below. include %ROOT%\conf\ms_scep_csv.conf ## Must add "ms_scep_csv" as INPUT to route below. # include %ROOT%\conf\ms_dns.conf ## Must add "MS_DNS" as INPUT to route below. # include %ROOT%\conf\ms_exchange15.conf ## Must add "MS_EXCH_MT" as INPUT to route below. # include %ROOT%\conf\ms_netlogon.conf ## Must add "MS_NETLOGON" as INPUT to route below. # include %ROOT%\conf\ms_iis.conf ## Must add "MS_IIS" or "MS_FTP" or "MS_SMTP" as INPUT to route below. ######################################## # Output Module Includes # ######################################## ## Uncomment additional OUTPUT modules below if desired. ## You MUST configure an IP or Hostname in each output conf file. include %ROOT%\conf\output_tcp.conf ## Must add "tcp_sender1" as OUTPUT to route below # include %ROOT%\conf\output_udp.conf ## Must add "udp_sender1" as OUTPUT to route below # include %ROOT%\conf\output_encrypted.conf ## Must add "ssl_sender1" as OUTPUT to route below include %ROOT%\conf\output_file.conf ## Must add "file_sender1" as OUTPUT to route below ######################################## # Default Route # ######################################## ## Add additional INPUTS comma separated on LEFT of arrow symbol. ## Add additional OUTPUTS comma separated on RIGHT of arrow symbol. #Primary route for log processing and forwarding. Path ms_scep_csv => file_sender1,tcp_sender1 ############################################################################### ############################################################################### ## DO NOT MODIFY BELOW CONFIGURATIONS UNLESS INSTRUCTED TO DO SO. ######################################## # Global Extensions # ######################################## ## Do not modify extensions as they may be required by included configurations. Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 Module xm_syslog # IETFTimestampInGMT TRUE Module xm_json Module xm_exec **ms_scep_csv conf file** ########################################################### # INPUT Microsft System Center Endpoint Protection # ########################################################### ## DO NOT MODIFY MODULE NAMES AS IT MAY BREAK TAP FUNCTIONALITY Module xm_csv Fields $Type, $RowID, $Name, $Description, $Timestamp, $SchemaVersion, $ObserverHost, $ObserverUser, $ObserverProductName, $ObserverProductVersion, $ObserverProtectionType, $ObserverProtectionVersion, $ObserverProtectionSignatureVersion, $ObserverDetection, $ObserverDetectionTime, $ActorHost, $ActorUser, $ActorProcess, $ActorResource, $ActionType, $TargetHost, $TargetUser, $TargetProcess, $TargetResource, $ClassificationID, $ClassificationType, $ClassificationSeverity, $ClassificationCategory, $RemediationType, $RemediationResult, $RemediationErrorCode, $RemediationPendingAction, $IsActiveMalware Delimiter , Module im_file File "C:\\Temp\\Desktop.csv" ReadFromLast TRUE SavePos TRUE CloseWhenIdle TRUE csv->parse_csv(); to_syslog_ietf(); **Desktop.csv file** "Type","RowID","Name","Description","Timestamp","SchemaVersion","ObserverHost","ObserverUser","ObserverProductName","ObserverProductversion","ObserverProtectionType","ObserverProtectionVersion", "ObserverProtectionSignatureVersion","ObserverDetection","ObserverDetectionTime","ActorHost","ActorUser","ActorProcess","ActorResource","ActionType","TargetHost","TargetUser","TargetProcess","TargetRe source","ClassificationID","ClassificationType","ClassificationSeverity","ClassificationCategory","RemediationType","RemediationResult","RemediationErrorCode","RemediationPendingAction","IsActiveMalware" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:33am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:34am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:35am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:36am" **Testfile.log output** 1 2020-03-03T10:19:28.428851-08:00 DESKTOP-TVVB676 - - - [NXLOG@14506 EventReceivedTime="2020-03-03 10:19:28" SourceModuleName="ms_scep_csv" SourceModuleType="im_file" Type="SecurityIncident" RowID="08be9aba-1326-4ac8-81e1-ace5c4550c76" Name="MalwareInfection" Description="NotImplemented" Timestamp="3/3/2020" SchemaVersion="1.0" ObserverHost="Testing" ObserverUser="" ObserverProductName="SystemCenterEndpointProtection" ObserverProductVersion="4.10.209.0" ObserverProtectionType="AM" ObserverProtectionVersion="" ObserverProtectionSignatureVersion="" ObserverDetection="Realtime" ObserverDetectionTime="3/3/2020" ActorHost="" ActorUser="" ActorProcess="" ActorResource="" ActionType="MalwareInfection" TargetHost="Testing" TargetUser="NT AUTHORITY\\SYSTEM" TargetProcess="System" TargetResource="file:_C:\\Path\\ofw2d3qz.iqf" ClassificationID="2147626289" ClassificationType="Trojan:Win32/Giframe.A" ClassificationSeverity="Severe" ClassificationCategory="Trojan" RemediationType="NoAction" RemediationResult="Testing" RemediationErrorCode="0" RemediationPendingAction="NoActionRequired" IsActiveMalware="Testing 3/3 9:35am"] "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:35am"

Hi, I don't have a lot of skill in NXLog and I need help. I have an application which puts a log file (* .csv) in a directory every 10 min. The file in question must be sent to the SIEM server and also must be compressed. my question how to make compressed the file once sent to the siem server.

Thanks for your help.

I'm doing some tests with an EE trial version for a software selection, in particular I have to verify the possibility of deleting the input file after sending it to a syslog. Does the EE trial version have all the features of the paid version? my configuration file is this: define ROOT C: \ Program Files \ nxlog

#ModuleDir% ROOT% \ modules #CacheDir% ROOT% \ data #SpoolDir% ROOT% \ data

define CERTDIR% ROOT% \ cert define CONFDIR% ROOT% \ conf

define LOGDIR% ROOT% \ data define MYLOGFILE% LOGDIR% \ nxlog.log LogFile% MYLOGFILE% <Input in> Module im_file File 'c: \ temp \ test_fileMW.txt' SavePos True ReadFromLast True <Oneof> Exec file_remove (file_name ()); </ Oneof> </ Input> <Output out> Module om_udp Host 10.1.15.42 Port 514 </ Output> <Route Path> Path in => out </ Route>

Starting the daemon I get this error message: 2020-03-03 09:06:16 ERROR Couldn't parse Exec block at C:\Program Files\nxlog\conf\nxlog.conf:21; couldn't parse statement at line 21, character 38 in C:\Program Files\nxlog\conf\nxlog.conf; procedure 'file_remove()' does not exist or takes different arguments 2020-03-03 09:06:16 WARNING no functional input modules! 2020-03-03 09:06:16 ERROR module 'in' has configuration errors, not adding to route 'Path' at C:\Program Files\nxlog\conf\nxlog.conf:33 2020-03-03 09:06:16 ERROR route Path is not functional without input modules, ignored at C:\Program Files\nxlog\conf\nxlog.conf:33 2020-03-03 09:06:16 WARNING not starting unused module out 2020-03-03 09:06:16 WARNING not starting unused module in 2020-03-03 09:06:16 INFO nxlog-4.6.4692-trial started

Thanks for the support. Paolo

Hello!

I updated my nxlog server to 4.6.4640, and today during the day I saw a new for me error in log file:

2020-03-02 18:03:01 WARNING nxlog received a termination request signal, exiting...
2020-03-02 18:03:11 ERROR failed to stop module in, module is busy
2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable
2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable
2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable
2020-03-02 18:03:17 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable
2020-03-02 18:03:17 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable
2020-03-02 18:04:09 ERROR timed out waiting for threads to exit
2020-03-02 18:04:19 ERROR failed to shutdown module in, module is busy

And nxlog server was down until I restarted it. This situation is repeated several times per hour.

Please help. Thaks a lot!

Hi,

I have a issue with my configuration. I try to send EventID to syslog with NXlog. But I am french and the log have accent.... And NXlog replace by "Ç" or other. For exemple é --> Ç

exemple :

02-20-2020 16:17:25 User.Info 10.28.201.50 1 2020-02-20T16:17:24.248999+01:00 PC-MGMT-INFRA-HDV Microsoft-Windows-Security-Auditing 532 - [NXLOG@14506 Keywords="-9214364837600034816" EventType="AUDIT_SUCCESS" EventID="4726" ProviderGuid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Version="0" Task="13824" OpcodeValue="0" RecordNumber="435937" ActivityID="{40052197-E800-0000-1A22-054000E8D501}" ThreadID="488" Channel="Security" Category="User Account Management" Opcode="Informations" TargetUserName="TEST-LOG" TargetDomainName="PC-MGMT-INFRA-H" TargetSid="S-1-5-21-398120947-1394256007-3495492944-1004" SubjectUserSid="S-1-5-21-398120947-1394256007-3495492944-500" SubjectUserName="Administrateur" SubjectDomainName="PC-MGMT-INFRA-H" SubjectLogonId="0x689a9" PrivilegeList="-" EventReceivedTime="2020-02-20 16:17:25" SourceModuleName="eventlog" SourceModuleType="im_msvistalog"] Un compte dƒ?Tutilisateur a ǸtǸ supprimǸ. Sujet¶ÿ: ID de sǸcuritǸ¶ÿ: S-1-5-21-398120947-1394256007-3495492944-500 Nom du compte¶ÿ: Administrateur Domaine du compte¶ÿ: PC-MGMT-INFRA-H ID dƒ?Touverture de session¶ÿ: 0x689A9 Compte cible¶ÿ: ID de sǸcuritǸ¶ÿ: S-1-5-21-398120947-1394256007-3495492944-1004 Nom du compte¶ÿ: TEST-LOG Domaine du compte¶ÿ: PC-MGMT-INFRA-H Informations supplǸmentaires¶ÿ: PrivilÇùges -

Can you help me ?

Hello:

I have been working on setting up an intermediary SYSLOG Server to receive syslog events from various network devices as part of my Splunk deployment.
Please NOTE: This a WINDOWS 2019 Server environment.

I am a newbie to NXLog . I have been able to get a base configuration working to receive data on port 514. I can successfully write to a file but the only option that seems to work is to write to file using the source IP Address, but I want to write to a file using the source Hostname.

I am using the Community Edition and do not have access to use xm_resolver.

How can I receive syslog data and write that data to file using source HOSTNAME?

I have been researching and trying now for close to a month with no success. Any information / guidance would be greatly appreciated.

Thank you for your time. Regards, --Diane Proscino

We have a requirement to send Windows Event logs over an encrypted channel with client authentication.

The issue is, the certificates in our infrastructure are stored in the Computer Certificates store with private keys that are marked as non-exportable.

I'm looking for a way to either: a) Somehow use NXLog to utilize the client certificate from within the store (ideal but I don't think NXLog is written to handle this) b) Find a scalable method for hundreds of servers to copy the key pair to NXLog-friendly PEM format from within the certificate store. There are ways to do this, but since the key is not marked as exportable it takes a lot of work to export that I don't think can be efficiently automated.

Does anyone have any ideas on this? Our current implementation is sending input from the Event Log to a Syslog server.

Thanks!

Hi there, I'm having a little trouble trying to filter events with patterndb.xml I'm sending logs to our SIEM but despite the corresponding event ID's missing from patterndb they are still getting pushed. I think my configuration setup is over ruling the patterndb config. Can you please review? Thanks for your time. # # Configuration for converting and sending Windows logs # to AlienVault USM Anywhere. # # Version: 0.1.0 # Last modification: 2019-07-03 # define ROOT C:\Program Files (x86)\nxlog define OUTPUT_DESTINATION_ADDRESS x.x..x.xx define OUTPUT_DESTINATION_PORT 12346789 Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log Module xm_json Module xm_syslog Module im_internal Module im_msvistalog Query \ \ *\ *\ *\ \ Exec if ($EventID == 5156) OR ($EventID == 5158) drop(); Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; Exec $Message = to_json(); to_syslog_bsd(); Path eventlog, internal => out ############################################################################ #### NXLOG WITH PATTERNDB ##### #### Uncomment the following lines for Windows Events filtered ##### ############################################################################ Module im_internal Module im_msvistalog Query \ \ *\ *\ *\ \ Module pm_pattern PatternFile %ROOT%\conf\patterndb.xml Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec if not defined $PatternID or not defined $Message { drop(); } Exec $Message = to_json(); to_syslog_bsd(); Path eventlog_Pattern, internal_Pattern => match_events => out_Pattern ############################################################################ ##### /NXLOG WITH PATTERNDB ##### ############################################################################

Hi, This is my sql_fetch command :

$Retval = sql_fetch("SELECT ServerName, Transmission FROM dbo.SrvAuth WHERE ServerName = ?", $MachineCourt);

This command does find the right record based on ServerName but it is always putting the second field, $Transmission, to the value FALSE.

Here is the MS SQL table definition : Colum Name Data Type Allow Nulls ServerName varchar(50) Unchecked Transmission bit Unchecked

Depending of the record that it is fetch, the DB contains about a third of TRUE and 2 third of FALSE for the Transmission field.

Question : Why do I always fetch FALSE for the Transmission field?

Thanks

When NXLog formats the the Event Log as `om_out` it formats the JSON correctly ``` define Format {if defined($EventTime) $timestamp = strftime($EventTime, '%Y-%m-%dT%H:%M:%SZ');else $timestamp = strftime($EventReceivedTime, '%Y-%m-%dT%H:%M:%SZ');rename_field("service_id", "_service_id");rename_field("timestamp", "_timestamp");rename_field("log_type", "_log_type");$body = $raw_event;$attributes = to_json();if defined($tag) $raw_event = "{" + '"timestamp"' + ':"' + $_timestamp + '",' + '"service_id"' + ':"' + $_service_id + '",' + '"tag"' + ':"' + $tag + '",' + '"log_type"' + ':"' + $_log_type + '",' + '"attributes"' + ':' + $attributes + '}';else $raw_event = "{" + '"timestamp"' + ':"' + $_timestamp + '",' + '"service_id"' + ':"' + $_service_id + '",' + '"log_type"' + ':"' + $_log_type + '",' + '"attributes"' + ':' + $attributes + '}';} ``` This is executed in the `` block which formats it into JSON format. When the `om_http` is called the same way as `om_out` an error is logged as an over sized string. At first the JSON looks normal but as the code goes on you get an excessively long string. Packet capture from Wireshark showing the end of the REST POST request. ``` POST / HTTP/1.1 User-Agent: nxlog-ce Content-Length: 621554 Beginning: {"timestamp":"2020-02-17T14:19:33Z","service_id":"id","tag":"security","log_type":"ea2_test","attributes":{"EventTime":"2020-02-17 14:19:33","Hostname":"hostname","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":4663,"SourceName":"Microsoft-Windows-Security-Auditing", End: Accesses:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tWRITE_DAC\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\r\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\r\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tAccess Mask:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t0x40000\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"}\\\\\\\\\\\\\\\"}\\\\\\\"}\\\"}\"}"}} ``` Why is it not working when you use the `om_http` module but works with the `om_out` module. Suggestions? Thanks in advance! ***EDIT:*** It looks like NXLog-CE broke itself. I was able to fix this by deleting everything in the nxlog/data folder and then reinstalled the agent. Now, using the exact same config files it appears to be working.

Hi all,

Sorry to come with an other new question about that but I don't understand why the regex didn't match the Message:

regexp /(?x)^\s?[(\d+):(\d+):(\d+)] (.+?) [Classification: (.+?)] [Priority: (\d+)] {(.+?)} (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})(:(\d{1,5}))? -> (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})(:(\d{1,5}))?\R?/ doesn't match subject string '[129:20:1] TCP session without 3-way handshake [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.1:8080 -> 192.168.0.2:53590'

If I compare that on Online regex site (PCRE), it works.

Thanks

Hi I got to set up a log forwarding to a syslog-server. I managed everthing to work except one thing.

The newest event at the end of the logfile has no CR, CRLF, LF or something similar. When a event occurs it is processed when the next event occurs ...

How can I make NXLog to read and process to the end of the file (EOF) in case the file has changed? So far I uses the im_file module.

Thank you for you help Daniel

Hi all,

Does Nxlog Enterprise has the possibility to request a table in order to convert some field ?

Like EventID 4624 on Windows and replace LogonType ID to a more readable string:

    "2": "Interactive",
    "3": "Network",
    "4": "Batch",
    "5": "Service",
    "7": "Unlock",
    "8": "NetworkCleartext",
    "9": "NewCredentials",
    "10": "RemoteInteractive",
    "11": "CachedInteractive",