Hello NXLog community! I was hoping you can help me with the problem I've been dealing with. I'm trying to configure NXLog to collect Sharepoint audit logs using a PowerShell script following the offical documentation here: https://nxlog.co/documentation/nxlog-user-guide/sharepoint.html#sharepoint_audit I have enabled Sharepoint audit logging, configured the PowerShell script and the NXLog input, but when I'm trying to run it it does not work. Can you please help me figure out what I'm doing wrong here? Here is what I'm getting in the log file: 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; The local farm is not accessibl; (right here) ------^; [The local farm is not accessible. Cmdlets with FeatureDependencyId are not registered.] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; C:\Audit\auditlog.ps1 : An unha; (right here) ------^; [C:\Audit\auditlog.ps1 : An unhandled exception occurred!] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; + CategoryInfo : NotSp; (right here) ------^; [ + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid string in json text.; tion; (right here) ------^; [ tion] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid char in json text.; + FullyQualifiedErrorId : Micro; (right here) ------^; [ + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, lexical error: invalid string in json text.; n,auditlog.ps1; (right here) ------^; [ n,auditlog.ps1] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR failed to parse json string, parse error: premature EOF; ; (right here) ------^; [ ] 2020-03-05 09:42:14 ERROR assignment failed at line 40, character 43 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. statement execution has been aborted; function 'parsedate' failed at line 40, character 42 in C:\Program Files\Graylog\sidecar\generated\nxlog.conf. expression evaluation has been aborted; 'unknown' type argument is invalid 2020-03-05 09:42:14 ERROR Module audit_powershell got EOF, process exited? Here's my config file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log LogLevel INFO <Extension logrotate> Module xm_fileop <Schedule> When @daily Exec file_cycle('%ROOT%\data\nxlog.log', 7); </Schedule> </Extension> <Extension gelfExt> Module xm_gelf # Avoid truncation of the short_message field to 64 characters. ShortMessageLength 65536 </Extension> <Extension _json> Module xm_json </Extension> <Input audit_powershell> Module im_exec Command "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" Arg "-ExecutionPolicy" Arg "Bypass" Arg "-NoProfile" Arg "-File" Arg "C:\Audit\auditlog.ps1" <Exec> parse_json(); $EventTime = parsedate($EventTime); </Exec> </Input> <Output gelf> Module om_tcp Host 192.168.98.5 Port 12208 OutputType GELF_TCP <Exec> # These fields are needed for Graylog $gl2_source_collector = '3a5aa0c9-aba3-4384-8691-43ed7d1ebbab'; $collector_node_id = 'Scooby'; </Exec> </Output> <Route route-2> Path audit_powershell => gelf </Route> And the PowerShell script I've created: # This script can be used with NXLog to fetch Audit logs via the SharePoint # API. See the configurable options below. Based on: # <http://shokochino-sharepointexperience.blogspot.ch/2013/05/create-auditing-reports-in-sharepoint.html> #Requires -Version 3 # The timestamp is saved to this file for resuming. $CacheFile = 'C:\Audit\nxlog_sharepoint_auditlog_position.txt' # The database is queried at this interval in seconds. $PollInterval = 10 # Allow this many seconds for new logs to be written to database. $ReadDelay = 30 # Use this to enable debug logging (for testing outside of NXLog). #$DebugPreference = 'Continue' ################################################################################ # If running 32-bit on a 64-bit system, run 64-bit PowerShell instead. if ($env:PROCESSOR_ARCHITEW6432 -eq "AMD64") { Write-Debug "Running 64-bit PowerShell." &"$env:WINDIR\SysNative\WindowsPowerShell\v1.0\powershell.exe" ` -NonInteractive -NoProfile -ExecutionPolicy Bypass ` -File "$($myInvocation.InvocationName)" $args exit $LASTEXITCODE } Add-PSSnapin "Microsoft.SharePoint.Powershell" -ErrorAction Stop # Return description for event function Event-Description { param( $entry ) switch ($entry.Event) { AuditMaskChange {"The audit flags are changed for the audited object."} ChildDelete {"A child of the audited object is deleted."} ChildMove {"A child of the audited object is moved."} CheckIn {"A document is checked in."} 'Copy' {"The audited item is copied."} Delete {"The audited object is deleted."} EventsDeleted {"Some audit entries are deleted from SharePoint database."} 'Move' {"The audited object is moved."} Search {"The audited object is searched."} SecGroupCreate {"A group is created for the site collection (this action "` + "also generates an Update event)."} SecGroupDelete {"A group on the site collection is deleted."} SecGroupMemberAdd {"A user is added to a group."} SecGroupMemberDelete {"A user is removed from a group."} SecRoleBindBreakInherit {"A subsite's inheritance of permission level "` + "definitions (that is, role definitions) is severed."} SecRoleBindInherit {"A subsite is set to inherit permission level "` + "definitions (that is, role definitions) from its parent."} SecRoleBindUpdate {"The permissions of a user or group for the audited "` + "object are changed."} SecRoleDefCreate {"A new permission level (a combination of permissions "` + "that are given to people holding a particular role for the site "` + "collection) is created."} SecRoleDefDelete {"A permission level (a combination of permissions that "` + "are given to people holding a particular role for the site "` + "collection) is deleted."} SecRoleDefModify {"A permission level (a combination of permissions that "` + "are given to people holding a particular role for the site "` + "collection) is modified."} Update {"An existing object is updated."} CheckOut {"A document is checked out."} View {"The object is viewed by a user."} ProfileChange {"Change in a profile that is associated with the object."} SchemaChange {"Change in the schema of the object."} Undelete {"Restoration of an object from the Recycle Bin."} Workflow {"Access of the object as part of a workflow."} FileFragmentWrite {"A File Fragment has been written for the file."} Custom {"Custom action or event."} default {"The event description could not be determined."} } } # Get audit data from $site in range $start to $end. Timestamps should use # seconds precision only. A record with timestamp equal to $start time is # included in output; a record with timestamp equal to $end time is not. function Get-Audit-Data { param( $site, $start, $end ) Write-Debug "Getting audit log for $site.Url from $start to $end" $query = New-Object -TypeName Microsoft.SharePoint.SPAuditQuery($site) $query.setRangeStart($start.AddSeconds(-1)) $query.setRangeEnd($end) $coll = $site.Audit.GetEntries($query) $root = $site.RootWeb for ($i=0; $i -le ($coll.Count)-1 ; $i++) { # Get the entry item from the collection $entry = $coll.Item($i) # Find the current user name foreach($User in $root.SiteUsers) { if($entry.UserId -eq $User.Id) { $UserName = $User.UserLogin } } # Find the item name foreach($List in $root.Lists) { if($entry.ItemId -eq $List.Id) { $ItemName = $List.Title } } # Create hash table $record = @{ # AuditData table fields SiteID = $entry.SiteId; ItemID = $entry.ItemId; ItemType = $entry.ItemType; UserID = $entry.UserId; AppPrincipalID = $entry.AppPrincipalId; MachineName = $entry.MachineName; MachineIP = $entry.MachineIP; DocLocation = $entry.DocLocation; LocationType = $entry.LocationType; EventTime = ($entry.Occurred.ToString('o') + "Z"); Event = $entry.Event; EventName = $entry.EventName; EventSource = $entry.EventSource; SourceName = $entry.SourceName; EventData = $entry.EventData; # Additional fields ItemName = $ItemName; Message = Event-Description $entry; SiteURL = $site.Url; UserName = $UserName; } # Return record as JSON $record | ConvertTo-Json -Compress | Write-Output } } # Get position timestamp from cache file. On first run, create file using # current time. function Get-Position { param( $file ) Try { if (Test-Path $file) { $time = (Get-Date (Get-Content $file -First 1)) $time = $time.ToUniversalTime() $time = $time.AddTicks(-($time.Ticks % 10000000)) } else { $time = [System.DateTime]::UtcNow $time = $time.AddTicks(-($time.Ticks % 10000000)) Save-Position $file $time } return $time } Catch { Write-Error "Failed to read timestamp from position file." exit 1 } } # Save position timestamp to cache file. function Save-Position { param( $file, $time ) Try { Out-File -FilePath $file -InputObject $time.ToString('o') } Catch { Write-Error "Failed to write timestamp to position file." exit 1 } } # Main Try { $start = Get-Position $CacheFile Write-Debug "Got start time of $($start.ToString('o'))." $now = [System.DateTime]::UtcNow $now = $now.AddTicks(-($now.Ticks % 10000000)) Write-Debug "Got current time of $($now.ToString('o'))." $diff = ($now - $start).TotalSeconds # Check whether waiting is required to comply with $ReadDelay. if (($diff - $PollInterval) -lt $ReadDelay) { $wait = $ReadDelay - $diff + $PollInterval Write-Debug "Waiting $wait seconds to start collecting logs." Start-Sleep -Seconds $wait } # Repeatedly read from the audit log while($true) { Write-Debug "Using range start time of $($start.ToString('o'))." $now = [System.DateTime]::UtcNow $now = $now.AddTicks(-($now.Ticks % 10000000)) $end = $now.AddSeconds(-($ReadDelay)) Write-Debug "Using range end time of $($end.ToString('o'))." $sites = Get-SPSite -Limit All foreach($site in $sites) { Get-Audit-Data $site $start $end } Write-Debug "Saving position timestamp to cache file." Save-Position $CacheFile $end Write-Debug "Waiting $PollInterval seconds before reading again." Start-Sleep -Seconds $PollInterval $start = $end } } Catch { Write-Error "An unhandled exception occurred!" exit 1 }

Good evening! I have a probem with configuring im_odbc module to connect to the Oracle database from unix. So... As I understood (maybe I'm wrong somewhere), this module don't work "from the box" and for correctrly working I need to set up oracle instant client wich provide me a driver, odbc.ini, odbcinst.ini files. Also I created tnsnames.ora file. odbc.ini, odbcinst.ini and tnsnames.ora configured correctly. I trying to test them with isql and it's work fine. I tried to differently configured ConnectionString parameter in configuration file, but it's did not work for me. /etc/odbcinst.ini: [libsqora.so.12.1] Description = Oracle ODBC driver for Oracle 12c Driver = /usr/lib/oracle/12.2/client64/lib/libsqora.so.12.1 Setup = FileUsage = CPTimeout = CPReuse /etc/odbc.ini: [ODBC Data Sources] OracleODBC = Oracle ODBC driver for Oracle 12c [OracleODBC] Application Attributes = T Attributes = W BatchAutocommitMode = IfAllSuccessful BindAsFLOAT = F CloseCursor = F DisableDPM = F DisableMTS = T Driver = libsqora.so.12.1 DSN = OracleODBC EXECSchemaOpt = EXECSyntax = T Failover = T FailoverDelay = 10 FailoverRetryCount = 10 FetchBufferSize = 64000 ForceWCHAR = F Lobs = T Longs = T MaxLargeData = 0 MetadataIdDefault = F QueryTimeout = T ResultSets = T ServerName = DB.ETALON SQLGetData extensions = F Translation DLL = Translation Option = 0 DisableRULEHint = T UserID = XXXX Password = XXXX StatementCache=F CacheBufferSize=20 UseOCIDescribeAny=F SQLTranslateErrors=F MaxTokenSize=8192 AggregateSQLType=FLOAT tnsnames.ora DB.ETALON = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = db-etalon.example.com)(PORT = 1530)) ) (CONNECT_DATA = (SERVICE_NAME = etalon) ) ) I don’t understand how to correctly set ConnectionString. Maybe someone had odbc setup experience.

Hello all, I am having an issue reading in a csv file and converting it out up to a log server. The first line/event in the csv gets parsed and converted correctly but then the second line/event doesn't get parsed and is converted to the same line as the first event. I am trying to have it read in the csv file (being exported from sccm for scep alerts) and convert it to syslog and send it up to log server. Please find all my configs below: NXlog conf (Not pasting full config file) ######################################## # Application Configuration Includes # ######################################## ## Uncomment additional input modules below if desired. ## Additional configuration may be required for each application in its conf file. # include %ROOT%\conf\ms_dhcpv4.conf ## Must add "MS_DHCPv4" as INPUT to route below. # include %ROOT%\conf\ms_dhcpv6.conf ## Must add "MS_DHCPv6" as INPUT to route below. # include %ROOT%\conf\ms_scep.conf ## Must add "ms_scep" as INPUT to route below. include %ROOT%\conf\ms_scep_csv.conf ## Must add "ms_scep_csv" as INPUT to route below. # include %ROOT%\conf\ms_dns.conf ## Must add "MS_DNS" as INPUT to route below. # include %ROOT%\conf\ms_exchange15.conf ## Must add "MS_EXCH_MT" as INPUT to route below. # include %ROOT%\conf\ms_netlogon.conf ## Must add "MS_NETLOGON" as INPUT to route below. # include %ROOT%\conf\ms_iis.conf ## Must add "MS_IIS" or "MS_FTP" or "MS_SMTP" as INPUT to route below. ######################################## # Output Module Includes # ######################################## ## Uncomment additional OUTPUT modules below if desired. ## You MUST configure an IP or Hostname in each output conf file. include %ROOT%\conf\output_tcp.conf ## Must add "tcp_sender1" as OUTPUT to route below # include %ROOT%\conf\output_udp.conf ## Must add "udp_sender1" as OUTPUT to route below # include %ROOT%\conf\output_encrypted.conf ## Must add "ssl_sender1" as OUTPUT to route below include %ROOT%\conf\output_file.conf ## Must add "file_sender1" as OUTPUT to route below ######################################## # Default Route # ######################################## ## Add additional INPUTS comma separated on LEFT of arrow symbol. ## Add additional OUTPUTS comma separated on RIGHT of arrow symbol. <Route 1> #Primary route for log processing and forwarding. Path ms_scep_csv => file_sender1,tcp_sender1 </Route> ############################################################################### ############################################################################### ## DO NOT MODIFY BELOW CONFIGURATIONS UNLESS INSTRUCTED TO DO SO. ######################################## # Global Extensions # ######################################## ## Do not modify extensions as they may be required by included configurations. <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _syslog> Module xm_syslog # IETFTimestampInGMT TRUE </Extension> <Extension _json> Module xm_json </Extension> <Extension _exec> Module xm_exec </Extension> ms_scep_csv conf file ########################################################### # INPUT Microsft System Center Endpoint Protection # ########################################################### ## DO NOT MODIFY MODULE NAMES AS IT MAY BREAK TAP FUNCTIONALITY <Extension csv> Module xm_csv Fields $Type, $RowID, $Name, $Description, $Timestamp, $SchemaVersion, $ObserverHost, $ObserverUser, $ObserverProductName, $ObserverProductVersion, $ObserverProtectionType, $ObserverProtectionVersion, $ObserverProtectionSignatureVersion, $ObserverDetection, $ObserverDetectionTime, $ActorHost, $ActorUser, $ActorProcess, $ActorResource, $ActionType, $TargetHost, $TargetUser, $TargetProcess, $TargetResource, $ClassificationID, $ClassificationType, $ClassificationSeverity, $ClassificationCategory, $RemediationType, $RemediationResult, $RemediationErrorCode, $RemediationPendingAction, $IsActiveMalware Delimiter , </Extension> <Input ms_scep_csv> Module im_file File "C:\\Temp\\Desktop.csv" ReadFromLast TRUE SavePos TRUE CloseWhenIdle TRUE <Exec> csv->parse_csv(); to_syslog_ietf(); </Exec> </Input> Desktop.csv file "Type","RowID","Name","Description","Timestamp","SchemaVersion","ObserverHost","ObserverUser","ObserverProductName","ObserverProductversion","ObserverProtectionType","ObserverProtectionVersion", "ObserverProtectionSignatureVersion","ObserverDetection","ObserverDetectionTime","ActorHost","ActorUser","ActorProcess","ActorResource","ActionType","TargetHost","TargetUser","TargetProcess","TargetRe source","ClassificationID","ClassificationType","ClassificationSeverity","ClassificationCategory","RemediationType","RemediationResult","RemediationErrorCode","RemediationPendingAction","IsActiveMalware" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:33am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:34am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:35am" "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:36am" Testfile.log output <13>1 2020-03-03T10:19:28.428851-08:00 DESKTOP-TVVB676 - - - [NXLOG@14506 EventReceivedTime="2020-03-03 10:19:28" SourceModuleName="ms_scep_csv" SourceModuleType="im_file" Type="SecurityIncident" RowID="08be9aba-1326-4ac8-81e1-ace5c4550c76" Name="MalwareInfection" Description="NotImplemented" Timestamp="3/3/2020" SchemaVersion="1.0" ObserverHost="Testing" ObserverUser="" ObserverProductName="SystemCenterEndpointProtection" ObserverProductVersion="4.10.209.0" ObserverProtectionType="AM" ObserverProtectionVersion="" ObserverProtectionSignatureVersion="" ObserverDetection="Realtime" ObserverDetectionTime="3/3/2020" ActorHost="" ActorUser="" ActorProcess="" ActorResource="" ActionType="MalwareInfection" TargetHost="Testing" TargetUser="NT AUTHORITY\SYSTEM" TargetProcess="System" TargetResource="file:_C:\Path\ofw2d3qz.iqf" ClassificationID="2147626289" ClassificationType="Trojan:Win32/Giframe.A" ClassificationSeverity="Severe" ClassificationCategory="Trojan" RemediationType="NoAction" RemediationResult="Testing" RemediationErrorCode="0" RemediationPendingAction="NoActionRequired" IsActiveMalware="Testing 3/3 9:35am"] "SecurityIncident","08be9aba-1326-4ac8-81e1- ace5c4550c76","MalwareInfection","NotImplemented","3/3/2020","1.0","Testing","","SystemCenterEndpointProtection","4.10.209.0","AM","","","Realtime","3/3/2020","","","","","MalwareInfection","Testing","NT AUTHORITY\SYSTEM","System","file:_C:\Path\ofw2d3qz.iqf","2147626289","Trojan:Win32/Giframe.A","Severe","Trojan","NoAction","Testing","0","NoActionRequired","Testing 3/3 9:35am"

Hi, I don't have a lot of skill in NXLog and I need help. I have an application which puts a log file (* .csv) in a directory every 10 min. The file in question must be sent to the SIEM server and also must be compressed. my question how to make compressed the file once sent to the siem server. Thanks for your help.

I'm doing some tests with an EE trial version for a software selection, in particular I have to verify the possibility of deleting the input file after sending it to a syslog. Does the EE trial version have all the features of the paid version? my configuration file is this: define ROOT C: \ Program Files \ nxlog #ModuleDir% ROOT% \ modules #CacheDir% ROOT% \ data #SpoolDir% ROOT% \ data define CERTDIR% ROOT% \ cert define CONFDIR% ROOT% \ conf define LOGDIR% ROOT% \ data define MYLOGFILE% LOGDIR% \ nxlog.log LogFile% MYLOGFILE% <Input in> Module im_file File 'c: \ temp \ test_fileMW.txt' SavePos True ReadFromLast True <Oneof> Exec file_remove (file_name ()); </ Oneof> </ Input> <Output out> Module om_udp Host 10.1.15.42 Port 514 </ Output> <Route Path> Path in => out </ Route> Starting the daemon I get this error message: 2020-03-03 09:06:16 ERROR Couldn't parse Exec block at C:\Program Files\nxlog\conf\nxlog.conf:21; couldn't parse statement at line 21, character 38 in C:\Program Files\nxlog\conf\nxlog.conf; procedure 'file_remove()' does not exist or takes different arguments 2020-03-03 09:06:16 WARNING no functional input modules! 2020-03-03 09:06:16 ERROR module 'in' has configuration errors, not adding to route 'Path' at C:\Program Files\nxlog\conf\nxlog.conf:33 2020-03-03 09:06:16 ERROR route Path is not functional without input modules, ignored at C:\Program Files\nxlog\conf\nxlog.conf:33 2020-03-03 09:06:16 WARNING not starting unused module out 2020-03-03 09:06:16 WARNING not starting unused module in 2020-03-03 09:06:16 INFO nxlog-4.6.4692-trial started Thanks for the support. Paolo

Hello! I updated my nxlog server to 4.6.4640, and today during the day I saw a new for me error in log file: 2020-03-02 18:03:01 WARNING nxlog received a termination request signal, exiting... 2020-03-02 18:03:11 ERROR failed to stop module in, module is busy 2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable 2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable 2020-03-02 18:03:16 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable 2020-03-02 18:03:17 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable 2020-03-02 18:03:17 ERROR Another instance is already running (pid 5075);Resource temporarily unavailable 2020-03-02 18:04:09 ERROR timed out waiting for threads to exit 2020-03-02 18:04:19 ERROR failed to shutdown module in, module is busy And nxlog server was down until I restarted it. This situation is repeated several times per hour. Please help. Thaks a lot!

Hi, I have a issue with my configuration. I try to send EventID to syslog with NXlog. But I am french and the log have accent.... And NXlog replace by "Ç" or other. For exemple é --> Ç exemple : 02-20-2020 16:17:25 User.Info 10.28.201.50 1 2020-02-20T16:17:24.248999+01:00 PC-MGMT-INFRA-HDV Microsoft-Windows-Security-Auditing 532 - [NXLOG@14506 Keywords="-9214364837600034816" EventType="AUDIT_SUCCESS" EventID="4726" ProviderGuid="{54849625-5478-4994-A5BA-3E3B0328C30D}" Version="0" Task="13824" OpcodeValue="0" RecordNumber="435937" ActivityID="{40052197-E800-0000-1A22-054000E8D501}" ThreadID="488" Channel="Security" Category="User Account Management" Opcode="Informations" TargetUserName="TEST-LOG" TargetDomainName="PC-MGMT-INFRA-H" TargetSid="S-1-5-21-398120947-1394256007-3495492944-1004" SubjectUserSid="S-1-5-21-398120947-1394256007-3495492944-500" SubjectUserName="Administrateur" SubjectDomainName="PC-MGMT-INFRA-H" SubjectLogonId="0x689a9" PrivilegeList="-" EventReceivedTime="2020-02-20 16:17:25" SourceModuleName="eventlog" SourceModuleType="im_msvistalog"] Un compte dƒ?Tutilisateur a ǸtǸ supprimǸ. Sujet¶ÿ: ID de sǸcuritǸ¶ÿ: S-1-5-21-398120947-1394256007-3495492944-500 Nom du compte¶ÿ: Administrateur Domaine du compte¶ÿ: PC-MGMT-INFRA-H ID dƒ?Touverture de session¶ÿ: 0x689A9 Compte cible¶ÿ: ID de sǸcuritǸ¶ÿ: S-1-5-21-398120947-1394256007-3495492944-1004 Nom du compte¶ÿ: TEST-LOG Domaine du compte¶ÿ: PC-MGMT-INFRA-H Informations supplǸmentaires¶ÿ: PrivilÇùges - Can you help me ?

Hello: I have been working on setting up an intermediary SYSLOG Server to receive syslog events from various network devices as part of my Splunk deployment. Please NOTE: This a WINDOWS 2019 Server environment. I am a newbie to NXLog . I have been able to get a base configuration working to receive data on port 514. I can successfully write to a file but the only option that seems to work is to write to file using the source IP Address, but I want to write to a file using the source Hostname. I am using the Community Edition and do not have access to use xm_resolver. How can I receive syslog data and write that data to file using source HOSTNAME? I have been researching and trying now for close to a month with no success. Any information / guidance would be greatly appreciated. Thank you for your time. Regards, --Diane Proscino

We have a requirement to send Windows Event logs over an encrypted channel with client authentication. The issue is, the certificates in our infrastructure are stored in the Computer Certificates store with private keys that are marked as non-exportable. I'm looking for a way to either: a) Somehow use NXLog to utilize the client certificate from within the store (ideal but I don't think NXLog is written to handle this) b) Find a scalable method for hundreds of servers to copy the key pair to NXLog-friendly PEM format from within the certificate store. There are ways to do this, but since the key is not marked as exportable it takes a lot of work to export that I don't think can be efficiently automated. Does anyone have any ideas on this? Our current implementation is sending input from the Event Log to a Syslog server. Thanks!

Hi there, I'm having a little trouble trying to filter events with patterndb.xml I'm sending logs to our SIEM but despite the corresponding event ID's missing from patterndb they are still getting pushed. I think my configuration setup is over ruling the patterndb config. Can you please review? Thanks for your time. # # Configuration for converting and sending Windows logs # to AlienVault USM Anywhere. # # Version: 0.1.0 # Last modification: 2019-07-03 # define ROOT C:\Program Files (x86)\nxlog define OUTPUT_DESTINATION_ADDRESS x.x..x.xx define OUTPUT_DESTINATION_PORT 12346789 Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> <Input eventlog> Module im_msvistalog Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> Exec if ($EventID == 5156) OR ($EventID == 5158) drop(); </Input> <Output out> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route 1> Path eventlog, internal => out </Route> ############################################################################ #### NXLOG WITH PATTERNDB ##### #### Uncomment the following lines for Windows Events filtered ##### ############################################################################ <Input internal_Pattern> Module im_internal </Input> <Input eventlog_Pattern> Module im_msvistalog Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> </Input> <Processor match_events> Module pm_pattern PatternFile %ROOT%\conf\patterndb.xml </Processor> <Output out_Pattern> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec if not defined $PatternID or not defined $Message { drop(); } Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route route_Pattern> Path eventlog_Pattern, internal_Pattern => match_events => out_Pattern </Route> ############################################################################ ##### /NXLOG WITH PATTERNDB ##### ############################################################################

Hi, This is my sql_fetch command : $Retval = sql_fetch("SELECT ServerName, Transmission FROM dbo.SrvAuth WHERE ServerName = ?", $MachineCourt); This command does find the right record based on ServerName but it is always putting the second field, $Transmission, to the value FALSE. Here is the MS SQL table definition : Colum Name Data Type Allow Nulls ServerName varchar(50) Unchecked Transmission bit Unchecked Depending of the record that it is fetch, the DB contains about a third of TRUE and 2 third of FALSE for the Transmission field. Question : Why do I always fetch FALSE for the Transmission field? Thanks

When NXLog formats the the Event Log as om_out it formats the JSON correctly define Format {if defined($EventTime) $timestamp = strftime($EventTime, '%Y-%m-%dT%H:%M:%SZ');else $timestamp = strftime($EventReceivedTime, '%Y-%m-%dT%H:%M:%SZ');rename_field("service_id", "_service_id");rename_field("timestamp", "_timestamp");rename_field("log_type", "_log_type");$body = $raw_event;$attributes = to_json();if defined($tag) $raw_event = "{" + '"timestamp"' + ':"' + $_timestamp + '",' + '"service_id"' + ':"' + $_service_id + '",' + '"tag"' + ':"' + $tag + '",' + '"log_type"' + ':"' + $_log_type + '",' + '"attributes"' + ':' + $attributes + '}';else $raw_event = "{" + '"timestamp"' + ':"' + $_timestamp + '",' + '"service_id"' + ':"' + $_service_id + '",' + '"log_type"' + ':"' + $_log_type + '",' + '"attributes"' + ':' + $attributes + '}';} This is executed in the <exec> block which formats it into JSON format. When the om_http is called the same way as om_out an error is logged as an over sized string. At first the JSON looks normal but as the code goes on you get an excessively long string. Packet capture from Wireshark showing the end of the REST POST request. POST / HTTP/1.1 User-Agent: nxlog-ce Content-Length: 621554 Beginning: {"timestamp":"2020-02-17T14:19:33Z","service_id":"id","tag":"security","log_type":"ea2_test","attributes":{"EventTime":"2020-02-17 14:19:33","Hostname":"hostname","Keywords":-9214364837600034816,"EventType":"AUDIT_SUCCESS","SeverityValue":2,"Severity":"INFO","EventID":4663,"SourceName":"Microsoft-Windows-Security-Auditing", End: Accesses:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tWRITE_DAC\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\r\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\r\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\n\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\tAccess Mask:\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\t0x40000\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\"}\\\\\\\\\\\\\\\"}\\\\\\\"}\\\"}\"}"}} Why is it not working when you use the om_http module but works with the om_out module. Suggestions? Thanks in advance! EDIT: It looks like NXLog-CE broke itself. I was able to fix this by deleting everything in the nxlog/data folder and then reinstalled the agent. Now, using the exact same config files it appears to be working.

Hi all, Sorry to come with an other new question about that but I don't understand why the regex didn't match the Message: regexp /(?x)^\s?[(\d+):(\d+):(\d+)] (.+?) [Classification: (.+?)] [Priority: (\d+)] {(.+?)} (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})(:(\d{1,5}))? -> (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})(:(\d{1,5}))?\R?/ doesn't match subject string '[129:20:1] TCP session without 3-way handshake [Classification: Potentially Bad Traffic] [Priority: 2] {TCP} 192.168.0.1:8080 -> 192.168.0.2:53590' If I compare that on Online regex site (PCRE), it works. Thanks

Hi I got to set up a log forwarding to a syslog-server. I managed everthing to work except one thing. The newest event at the end of the logfile has no CR, CRLF, LF or something similar. When a event occurs it is processed when the next event occurs ... How can I make NXLog to read and process to the end of the file (EOF) in case the file has changed? So far I uses the im_file module. Thank you for you help Daniel

Hi all, Does Nxlog Enterprise has the possibility to request a table in order to convert some field ? Like EventID 4624 on Windows and replace LogonType ID to a more readable string: &quot;2&quot;: &quot;Interactive&quot;, &quot;3&quot;: &quot;Network&quot;, &quot;4&quot;: &quot;Batch&quot;, &quot;5&quot;: &quot;Service&quot;, &quot;7&quot;: &quot;Unlock&quot;, &quot;8&quot;: &quot;NetworkCleartext&quot;, &quot;9&quot;: &quot;NewCredentials&quot;, &quot;10&quot;: &quot;RemoteInteractive&quot;, &quot;11&quot;: &quot;CachedInteractive&quot;,

Hello, I have installed NXLog community edition to collect table data from PostgreSQL database but, the table isn't contain an ID column. As I understand, NXLog required to this field to bookmark but, we don't have. I'm looking for a workaround to solve the issue. On the other hand I can see a workaround from the following link and we can configure the ID with select statements but, the article isn't about the PostgreSQL. Could you please someone help me for PostgreSQL? https://nxlog.co/documentation/nxlog-user-guide/mssql.html the second question: Can we define a specific column (such as eventime) for ID (bookmark) with the following sample data? 2020-02-11 15:00:00.0000 2020-02-11 15:00:01.0001 2020-02-11 15:00:02.0002 2020-02-11 15:00:03.0000 Thanks in Advance! Best Regards SD

Hi there, a little bit of a novice here. Hope you dont mind pointing me in the right direction. I’m having some difficulty getting the configuration for using patternDB on windows 2003 servers, the configuration works for windows 2008+ The logs I have are as follows: 2020-02-05 13:48:32 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:40 2020-02-05 13:48:32 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:76 2020-02-05 13:48:32 ERROR module 'eventlog' has configuration errors, not adding to route '1' at C:\Program Files\nxlog\conf\nxlog.conf:57 2020-02-05 13:48:32 ERROR module 'eventlog_Pattern' has configuration errors, not adding to route 'route_Pattern' at C:\Program Files\nxlog\conf\nxlog.conf:94 2020-02-05 13:48:32 WARNING not starting unused module eventlog 2020-02-05 13:48:32 WARNING not starting unused module eventlog_Pattern 2020-02-05 13:48:32 INFO nxlog-ce-2.10.2150 started The section of conf is: ############################################################################ #### NXLOG WITH PATTERNDB ##### #### Uncomment the following lines for Windows Events filtered ##### ############################################################################ <Input internal_Pattern> Module im_internal </Input> <Input eventlog_Pattern> Module im_msvistalog Query <QueryList>\ <Query Id="0">\ <Select Path="Application">*</Select>\ <Select Path="System">*</Select>\ <Select Path="Security">*</Select>\ </Query>\ </QueryList> </Input> <Processor match_events> Module pm_pattern PatternFile %ROOT%\conf\patterndb.xml </Processor> <Output out_Pattern> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec if not defined $PatternID or not defined $Message { drop(); } Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route route_Pattern> Path eventlog_Pattern, internal_Pattern => match_events => out_Pattern </Route> ############################################################################ ##### /NXLOG WITH PATTERNDB ##### ############################################################################ Thanks for reading. Please let me know if any more information needs to be included.

Hi all, According to the documentation found here it indicates that the generic RPM doesn't have all available modules as opposed to the version specific RPM: The generic RPM above contains all the libraries (such as libpcre and libexpat) that are needed by NXLog, the only dependency is libc. However, some modules are not available (im_checkpoint, for example). The advantage of the generic RPM is that it can be installed on most RPM-based Linux distributions. Is there documentation for what modules are not available? Are there any issues for deploying this version that I should know about up front? Thanks!!

How can we rename nxlog package ? while we are placing both the rpm into spacewalk channel these are updating as “nxlog-ce-2.10.2150-1.x86_64.rpm” .So it's making a duplicate,So i hope renaming the rpm name will help us. Any help will be appreciated on this. nxlog-ce-2.10.2150-1_rhel6.x86_64.rpm nxlog-ce-2.10.2150-1_rhel7.x86_64.rpm Thanks! Ela

Our project is planning to reissue certificates for large amount of agents. Do we have API on the certificates so we'll able to reissue on these agents at the same time without doing it manually (one by one)?