Hello community, I have to collect Security logs from a Windows Server 2003. Sometimes, I have the following errors :   2017-11-22 09:03:52 INFO nxlog-ce-2.9.1504 started 2017-11-22 09:03:52 INFO connecting to siem.nutrition.lan:1514 2017-11-22 09:03:52 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 1 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:03:54 INFO Successfully reopened Security EventLog 2017-11-22 09:03:54 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 2 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:03:57 INFO Successfully reopened Security EventLog 2017-11-22 09:03:57 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 4 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:04:02 INFO Successfully reopened Security EventLog 2017-11-22 09:04:02 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 8 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:04:11 INFO Successfully reopened Security EventLog 2017-11-22 09:04:11 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 16 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs.   Here is my config file :      define ROOT C:\Program Files\nxlog   Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log   ############################ # EXTENSION  ############################   # Enable json extension <Extension json>     Module xm_json </Extension>   # Enable syslog extension <Extension syslog>     Module xm_syslog </Extension>   # Enable conversion module <Extension charconv>     Module xm_charconv     AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2 </Extension>     ############################ # INPUT  ############################   # Nxlog internal logs <Input internal>     Module im_internal     Exec to_json(); </Input>   # Windows Event Log for 2003 server <Input eventlog2003>   # Module for Windows 2003 server     Module im_mseventlog Sources Security     Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;     Exec convert_fields("AUTO", "utf-8");     Exec $Message = to_json(); to_syslog_bsd(); </Input>     ############################ # OUTPUT  ############################     <Output siem>     Module         om_tcp     Host    collector.company.com     Port    1514 </Output>   <Route 1>     Path     eventlog2003, internal => siem </Route>   I can solve the issue by restarting nxlog agent but this is not a definitive solution... Anybody has the same issue ? 

Hey, I use nxlog to send highly critical log (legal log). NXLog is configured to remove file after a specified retention time, to release space from drive. My configuration works like a charm, but... If I deploy nxlog with this following configuration, nxlog remove all file older than my retention time but before the processing. Every 1 hour Exec file_remove('%LOGFOLDER%', %RETENTIONTIME%); How do you manage this removing process? Right now, I have to deploy nxlog without this schedule, wait for processing, stop nxlog, add schedule job to the configuration, restart nxlog. Hope my case has not been already replied. Thanks for your help :)

HI, I have NXLog (nxlog-ce-2.9.1716) deployed to over 100 Windows servers to send Windows events to Graylog and its working fantastically on all but one server, our main File server. On this one server, the same 5145 events seem to be repeating events up to 19 times - from what I can see, they are identical on Graylog but don't appear duplicated on the file server - the only difference I can see is the volume of events which (by design we need to see file access successes as well as failures and have ABE enabled so see up to 170k events per min). We don't see the issue on any other server, I have implemented the no repeat module (config below) but still no joy. Any suggestions or advice welcome please? NXLog configuration file. See the nxlog reference manual for more info #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> <Input in> # Use 'im_mseventlog' for Windows XP and 2003 Module im_msvistalog Exec if ($EventID == 4202 or $EventID == 4208 or $EventID == 4302 or $EventID == 4304 or $EventID == 5004 or $EventID == 5156) drop(); else{ if ( $EventType == "INFO" ) $SyslogSeverityValue = 6; if ( $EventType == "WARNING" ) $SyslogSeverityValue = 4; if ( $EventType == "ERROR" ) $SyslogSeverityValue = 3; } </Input> <Processor norepeat> Module pm_norepeat CheckFields Hostname, SourceName, Message </Processor> <Output out> Module om_udp #Our Graylog server Host [our graylog server] Port 12201 OutputType GELF </Output> <Route 1> Path in => norepeat => out </Route>

I am attempting to use NXLog in conjunction with Graylog's Sidecar Collector to send data to my Graylog server. I have been able to successfully telnet to my Graylog server through port 5044, so I know it is not a firewall issue, yet I keep getting these errors in my NXLog error log: 2017-11-08 14:37:09 ERROR om_tcp detected a connection error;Connection reset by peer 2017-11-08 14:37:25 ERROR om_tcp detected a connection error;End of file found And nothing is being received by my Graylog server. Here is my generated NXLog configuration: define ROOT /usr/bin <Extension gelf> Module xm_gelf </Extension> User nxlog Group nxlog Moduledir /usr/libexec/nxlog/modules CacheDir /var/spool/collector-sidecar/nxlog PidFile /var/run/graylog/collector-sidecar/nxlog.pid define LOGFILE /var/log/graylog/collector-sidecar/nxlog.log LogFile %LOGFILE% LogLevel DEBUG <Extension logrotate> Module xm_fileop <Schedule> When @daily Exec file_cycle('%LOGFILE%', 7); </Schedule> </Extension> <Input 59fcda86ccba8e2573422cb4> Module im_file File '/var/log/httpd/syriac_access_log' PollInterval 1 SavePos True ReadFromLast True Recursive True RenameCheck False Exec $FileName = file_name(); # Send file name with each message </Input> <Output 59fcda56ccba8e2573422c80> Module om_tcp Host graylog.library.vanderbilt.edu Port 5044 OutputType GELF_TCP Exec $short_message = $raw_event; # Avoids truncation of the short_message field. Exec $gl2_source_collector = '485f3ca7-ca1e-4959-be00-117a50e2b1db'; Exec $collector_node_id = 'graylog-collector-sidecar'; Exec $Hostname = hostname_fqdn(); </Output> <Route route-0> Path 59fcda86ccba8e2573422cb4 => 59fcda56ccba8e2573422c80 </Route> I would appreciate any advice or guidance. Thank you!

Hello, I have what I think is a fairly straightforward situation. I'm running queries against an MS-SQL server to retrieve data every 5 minutes. The wrinkle is that I am using unixodbc from a ubuntu 16.04 machine with nxlog to do this. I'm able to retrieve all of the data once on a fresh install of nxlog. However, I'm not seeing new data every 5 minutes and I certain there should be more data as our SQL source is continually writing new entries... My input section has a fairly complicated query... <Input in_epo1> Module im_odbc   ConnectionString DSN=Server;Database=ePO_PROD;UID=epo_user;PWD=epo_pwd;       PollInterval 300     SQL          SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents].[ThreatType] as [threat_type], [EPOEvents].[ThreatEventID] as [signature_id], [EPOEvents].[ThreatCategory] as [category], [EPOEvents].[ThreatSeverity] as [severity_id], [EPOEventFilterDesc].[Name] as [event_description], [EPOEvents].[DetectedUTC] as [detected_timestamp], [EPOEvents].[TargetFileName] as [file_name], [EPOEvents].[AnalyzerDetectionMethod] as [detection_method], [EPOEvents].[ThreatActionTaken] as [vendor_action], CAST([EPOEvents].[ThreatHandled] as int) as [threat_handled], [EPOEvents].[TargetUserName] as [logon_user], [EPOComputerProperties].[UserName] as [user], [EPOComputerProperties].[DomainName] as [dest_nt_domain], [EPOEvents].[TargetHostName] as [dest_dns], [EPOEvents].[TargetHostName] as [dest_nt_host], [EPOComputerProperties].[IPHostName] as [fqdn], [dest_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),4,1))) ), [EPOComputerProperties].[SubnetMask] as [dest_netmask], [EPOComputerProperties].[NetAddress] as [dest_mac], [EPOComputerProperties].[OSType] as [os], [EPOComputerProperties].[OSServicePackVer] as [sp], [EPOComputerProperties].[OSVersion] as [os_version], [EPOComputerProperties].[OSBuildNum] as [os_build], [EPOComputerProperties].[TimeZone] as [timezone], [EPOEvents].[SourceHostName] as [src_dns], [src_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),4,1))) ), [EPOEvents].[SourceMAC] as [src_mac], [EPOEvents].[SourceProcessName] as [process], [EPOEvents].[SourceURL] as [url], [EPOEvents].[SourceUserName] as [source_logon_user], [EPOComputerProperties].[IsPortable] as [is_laptop], [EPOEvents].[AnalyzerName] as [product], [EPOEvents].[AnalyzerVersion] as [product_version], [EPOEvents].[AnalyzerEngineVersion] as [engine_version], [EPOEvents].[AnalyzerEngineVersion] as [dat_version], [EPOProdPropsView_VIRUSCAN].[datver] as [vse_dat_version], [EPOProdPropsView_VIRUSCAN].[enginever64] as [vse_engine64_version], [EPOProdPropsView_VIRUSCAN].[enginever] as [vse_engine_version], [EPOProdPropsView_VIRUSCAN].[hotfix] as [vse_hotfix], [EPOProdPropsView_VIRUSCAN].[productversion] as [vse_product_version], [EPOProdPropsView_VIRUSCAN].[servicepack] as [vse_sp] FROM [EPOEvents] left join [EPOLeafNode] on [EPOEvents].[AgentGUID] = [EPOLeafNode].[AgentGUID] left join [EPOProdPropsView_VIRUSCAN] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_VIRUSCAN].[LeafNodeID] left join [EPOComputerProperties] on [EPOLeafNode].[AutoID] = [EPOComputerProperties].[ParentID] left join [EPOEventFilterDesc] on [EPOEvents].[ThreatEventID] = [EPOEventFilterDesc].[EventId] and (EPOEventFilterDesc.Language='0409') WHERE (DATEDIFF(hour, CAST([EPOEvents].[ReceivedUTC] as date), getutcdate()) between 0 and 2) AND [EPOEvents].[AutoID] > ?       Exec delete($timestamp_epo); </Input>   To me this looks like a fairly standard query - nxlog should save its position and know to query [EPOEvents].[AutoID] > lastseen_autoid. We added (DATEDIFF(hour, CAST([EPOEvents].[ReceivedUTC] as date), getutcdate()) between 0 and 2) into the mix to limit our results to the last two hours, however, we could equally remove this.   So I enabled ODBC tracing and it certainly seems like nxlog is querying the database successfully. Here's a short snippet:   [ODBC][10][1509980277.861057][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fa8000eb0 Window Hdl = (nil) Str In = [DSN=DATABASE2;Database=DATABASE2;UID=user2;PWD=*************;][length = 58 (SQL_NTS)] Str Out = 0x7f6fb8eb9bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb8eb9144 Completion = 0 [ODBC][10][1509980277.861922][SQLAllocHandle.c][375] Entry: Handle Type = 2 Input Handle = 0x7f6fa00008c0 [ODBC][10][1509980277.861965][SQLAllocHandle.c][493] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa0000ef0 [ODBC][10][1509980277.862003][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fa0000ef0 Window Hdl = (nil) Str In = [DSN=DATABASE1;Database=DATABASE1;UID=user1;PWD=***;][length = 72 (SQL_NTS)] Str Out = 0x7f6fb6eb5bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb6eb5144 Completion = 0 [ODBC][10][1509980277.862040][SQLSetEnvAttr.c][363] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.862078][SQLAllocHandle.c][375] Entry: Handle Type = 2 Input Handle = 0x7f6fb00009b0 [ODBC][10][1509980277.862093][SQLAllocHandle.c][493] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fb0001270 [ODBC][10][1509980277.862115][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fb0001270 Window Hdl = (nil) Str In = [DSN=DATABASE3;Database=DATABASE3;UID=user2;PWD=*************;][length = 58 (SQL_NTS)] Str Out = 0x7f6fb7eb7bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb7eb7144 Completion = 0 UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE1'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.892817][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE1;UID=user1;PWD=********;WSID=8ef53561e...] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE1'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.892903][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fa0000ef0 [ODBC][10][1509980277.892936][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa003f1b0 [ODBC][10][1509980277.892952][SQLBindParameter.c][217] Entry: Statement = 0x7f6fa003f1b0 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc3b0e28 Value Max = 0 StrLen Or Ind = (nil) [ODBC][10][1509980277.892974][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.893002][SQLExecDirect.c][240] Entry: Statement = 0x7f6fa003f1b0 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE3'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.893946][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE3;UID=user2;PWD=********;WSID=8ef53561e30d;DATABAS...] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE2'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE3'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.894011][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fb0001270 [ODBC][10][1509980277.894023][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE2;UID=user2;PWD=********;WSID=8ef53561e30d;DATABAS...] [ODBC][10][1509980277.894038][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fb0052780 DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE2'.   [ODBC][10][1509980277.894058][SQLBindParameter.c][217] Entry: Statement = 0x7f6fb0052780 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc38ce28 Value Max = 0 StrLen Or Ind = (nil) DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.894079][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.894107][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fa8000eb0 [ODBC][10][1509980277.894112][SQLExecDirect.c][240] Entry: Statement = 0x7f6fb0052780 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] [ODBC][10][1509980277.894138][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa8067230 [ODBC][10][1509980277.894159][SQLBindParameter.c][217] Entry: Statement = 0x7f6fa8067230 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc39ee28 Value Max = 0 StrLen Or Ind = (nil) [ODBC][10][1509980277.894179][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.894211][SQLExecDirect.c][240] Entry: Statement = 0x7f6fa8067230 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] [ODBC][10][1509980278.153007][SQLExecDirect.c][503] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153074][SQLNumResultCols.c][156] Entry: Statement = 0x7f6fb0052780 Column Count = 0x7f6fbc38ce80 [ODBC][10][1509980278.153103][SQLNumResultCols.c][248] Exit:[SQL_SUCCESS] Count = 0x7f6fbc38ce80 -> 44 [ODBC][10][1509980278.153133][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 1 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a0 Column Size = 0x7f6fbc2d83b8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153169][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [AutoID]                 Data Type = 0x7f6fbc2d80a0 -> 4                 Column Size = 0x7f6fbc2d83b8 -> 10                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153197][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 1 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153247][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153274][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 2 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a2 Column Size = 0x7f6fbc2d83c0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153303][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [timestamp_epo]                 Data Type = 0x7f6fbc2d80a2 -> 93                 Column Size = 0x7f6fbc2d83c0 -> 23                 Decimal Digits = 0x7f6fb7eb654e -> 3                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153326][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 2 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153350][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153373][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 3 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a4 Column Size = 0x7f6fbc2d83c8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153401][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [signature]                 Data Type = 0x7f6fbc2d80a4 -> -9                 Column Size = 0x7f6fbc2d83c8 -> 128                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153424][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 3 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153447][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153560][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 4 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a6 Column Size = 0x7f6fbc2d83d0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153589][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [threat_type]                 Data Type = 0x7f6fbc2d80a6 -> -9                 Column Size = 0x7f6fbc2d83d0 -> 32                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153612][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 4 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153635][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153732][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 5 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a8 Column Size = 0x7f6fbc2d83d8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153760][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [signature_id]                 Data Type = 0x7f6fbc2d80a8 -> 4                 Column Size = 0x7f6fbc2d83d8 -> 10                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153783][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 5 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153814][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153837][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 6 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80aa Column Size = 0x7f6fbc2d83e0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153864][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [category]                 Data Type = 0x7f6fbc2d80aa -> -9                 Column Size = 0x7f6fbc2d83e0 -> 128                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153887][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 6 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153909][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.154007][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 7 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80ac Column Size = 0x7f6fbc2d83e8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.154035][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [severity_id]                 Data Type = 0x7f6fbc2d80ac -> -6                 Column Size = 0x7f6fbc2d83e8 -> 3                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0   However nothing is being written to the output json file.   Does it look like I'm doing anything wrong? Is there anything else I should try?  

xm_perl provides enhanced capabilities to run scripts that can enhance the logs/events that are generated. Is there an expected release soon that will include this functionality for the Windows version?

Hello, could someone point me into the direction of  how to configure nxlog enterprise with kafka topic subscriptions for pub/sub? I have only seen mention of the capability, though no specifics or connector?   Thank you,   Rob 

Yet the source code for this module is present in the (relatively old) source .tar.gz. Should the .debs contain im_mvistalog.so? Is there a plan to release newer sources?  If we use nxlog I'd prefer to deploy on Alpine Linux Is there a way to search the forum?  I'm sure these questions have been asked already Thanks!

Hi, i am using the nxlog-ce-2.9.1716-1_rhel7.x86_64.rpm and i have the following conf <Output graylogout> Module om_udp Host somehost.com Port 12201 OutputType GELF </Output> If i do "host somehost.com" -> It returns 2 IP in different order $>host somehost.com X.X.X.X Y.Y.Y.Y $>host somehost.com Y.Y.Y.Y X.X.X.X But nxlog always send messages to only one host (X.X.X.X). Does nxlog do a DNS request once? if is that the case, how do i balance the traffic?

Hello, Does NXlog provide a classification taxonomy or is classification up to the customer? Do you have recommended taxonomies? Thank you.

Hello,   I would like to forward logs generated by HypErV. In event viewer they are in Applications and Services logs -> Microsoft -> Windows -> Hyper-V-* I tried to add a query in the input configuration but I didn't manage to find the correct path to configure. I tried several path but each time I have an error "the channel was not found " Query <QueryList> \            <Query Id="0"> \             <Select Path="Security">*</Select> \             <Select Path="Hyper-V-logs">*</Select> \            </Query> \           </QueryList> </Input> What should I put instead of Hyper-V-logs to send hyperV logs? Can someone help me?   Regards,  

Hi all,  I'm trying to troubleshoot an issue with getting the Checkpoint input module working and pulling syslogs from a Checkpoint management appliance. I'm following the configuration guide provided in the nxlog documentation, but I am running into a few issues.  Specifically on the step where I attempt to retrieve the authentication key by using opsec_putkey. The correct output from that command should be "FW: Received new control security key from IP; Authentication with IP initialized successfully." However, my output is " FW: Received new control security key from IP; Failed to initialize authentication with IP." When I go to test whether the log collection works because I do end up receiving the sslauthkeys.c and sslses.c files, and I have the opsec.p12 cert from a previous step, I get the following response - "Peer IP wants to exchange keys but I don't have a password." Google hasn't been very fruitful so any advice would be appreciated.  -M

Hello good people! Is there any chance for Community Edition to get deb package for Debian 9 (Stretch)? Many thanks in advance!  

  Hi All,  I am on Cent OS 7 and installed nxlog to forward the /var/log/messages and /var/log/secure to my logging server. When I start its starting fine and in logs I am gettting weird log which I am not able to solve.  ERROR 2017-10-19 13:32:16 ERROR om_tcp detected a connection error;End of file found 2017-10-19 13:32:17 INFO connecting to 10.12.86.87:12225 2017-10-19 13:32:17 INFO reconnecting in 1 seconds 2017-10-19 13:32:17 ERROR om_tcp detected a connection error;End of file found Config:  ######################################## # Global directives                    # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules                              # ######################################## <Input messages>     Module              im_file     File                "/var/log/messages" SavePos           TRUE </Input> <Input securelogs>     Module              im_file     File                "/var/log/secure" SavePos TRUE </Input> <Output logserver>     Module              om_tcp     Host                10.12.86.87     Port                12225 </Output> <Route Log-TCP>     Path        messages, securelogs => logserver </Route>   Selinux and Firewall is turned off in my machine. And remote server is listenting on the port 12225 and that can be reachable from this machine without any issues. Can you help me with what I can do to solve this issue.    Thanks in advance 

hi, I am tesing nxlog on Ubuntu 16.04 LTS with the configuration <Extension fileop>         Module xm_fileop </Extension> <Input udpin>         Module im_udp         Host 127.0.0.1         Port 1514         Exec $HOSTIP = string(host_ip());         Exec file_write("/tmp/debug.txt", "HostIP:" + $HOSTIP); </Input> <Output udpfile>         Module om_file         File "/var/log/udp.log" </Output> <Route udp>     Path udpin => udpfile </Route> and  I generated log using linux command nc -u 127.0.0.1 1514 and can not get IP in /tmp/debug.txt. 1. I have reinstall the Ubuntu 16.04 LTS purely and install the deb file from nxlog.co， and it still does not get the IP address  2. I install the nxlog from source code, and it still does not get the IP address 3. I test the same configuration on CentOS7, CentOS6, Ubuntu 14.04 LTS. It works correctly. so， is there someting that does not compatible with the Ubuntu 16.04 LTS?

Hi I'm trying to do a syslog udp to tcp converter using this method from the user's manual: <Input in>     Module im_tcp     Host 0.0.0.0     Port 2345  </Input> <Output out>     Module      om_tcp     Host        mysyslog.domain.local     Port        514 </Output> ~~<Processor buffer>  Module pm_buffer  WarnLimit 800  MaxSize 1000  Type Mem  Exec if buffer_size() >= 80k drop(); </Processor> <Route 1>     Path in => buffer => out </Route> This is working correctly but all the events recieved by mysyslog.domain.local server are coming from the same source (the IP of the NXLOG server). Is there a way to preserve the source IP? Thanks in advance Regards, Olga

the following context is my Input configuration and i get a functong from the manual doc,  but the return value of hos_ip() is ip4addr, and I want to convert the return value to string, I have try the string() function, but it does not work at all. Is there any way to convert the return value to string so i can and a new filed of the Input configuration. ip4addr host_ip(); description Return the first non-loopback IP address the hostname resolves to. return type ip4addr   <Input 59dcb7a6dd48cb088969e300>         Module im_file         File '/usr/local/nginx_raw/logs/access.log'         PollInterval 1         SavePos True         ReadFromLast True         Recursive True         RenameCheck False         Exec $FileName = file_name(); # Send file name with each message         Exec $HOSTIP = host_ip(); </Input>  

Hi , Trying to create an ODBC connect for NXLog to connect to.   NXLog is installed on the same Windows 2012 server as the SQL Server 2008R2 instance.   Scenario 1: 32-bit ODBC is setup as a System DSN with a SQL Server account that has DBO access to the desired database NXLog service is setup to run under the System account.      - I've tried both drivers available on the system ("SQL Server Native Client 10.0"  and "SQL Server") - get the same result in the error log for each: ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][SQL Server Native Client 10.0][SQL Server]Login failed for user ''. (odbc error code: -1)and ERROR im_odbc couldn't connect to the database, 28000:2:18456:[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. (odbc error code: -1)   Scenario 2: Same ODBC, but with a Windows account that has full Admin access to the desired databases, and is the same account logged into Windows NXLog service is setup to run under this same account. Goal is to have the same user account accessing everything, in the hope of getting it to connect.   Same error messages as above.   Login failed for user ' '.     Since the error messages don't show the user that is failing to login, I'm having trouble narrowing down where the failure is at.   NXLOG.conf file: <Input call_logs> Module im_odbc ConnectionString DSN=SIEM_NXLog;database=recorder; SQL SELECT ident as id ,at.audit_time as EventTime ,am.audit_module_name as Message FROM mytables...  WHERE at.ident>? SavePos TRUE </Input>   There's one line in the documentation that has me scratching my head: SECTION 6.2.18 (ODBC) The data source must be accessible by the user which nxlog is running under.   I'm not sure if this means that the NTService account needs database access? Or, if the service must be under a Windows account user that has database access? Or, by using a ODBC->System DSN , shouldn't the ODBC already be accessible to all users on the system?   Any thoughts or insight would be helpful. Thanks in advance.   Cheers,  Peter          

Hello! I noticed strange cases nxlog.exe is loading 60%-90% CPU. Log file has nothings about errors. I have the same config file for win7 and winXP, but winXP is OK in contrast to win7. Also if I use om_udp module instead om_tcp in config file for win7 high load is desappear.   Could anybody say me what is the problem, or how I can get more information about cause of problems.

Hello! I have a question about IETFTimestampInGMT option. In the documentation - section "Syslog (xm_syslog)" - I can read : "IETFTimestampInGMT This optional boolean directive can be used to format the timestamps produced by to_syslog_ietf() in GMT instead of local time. This defaults to FALSE so that local time is used by default with a timezone indicator" So here's a part of my nxlog.conf : <Extension _syslog>     Module  xm_syslog     IETFTimestampInGMT  TRUE </Extension>   <Output  outUDP>     Module om_file     Exec log_info($Raw_event);parse_syslog_ietf(); log_info($EventTime);     CreateDir TRUE     Sync FALSE     File "C:\\" + $SourceName + "\\out.log" </Output>   Output : <7>1 2017-10-10T12:30:59.308493+02:00 JohnDoe-PC appTest 3788 tag1 - Testing debug message 2017-10-10 12:30:59   (I tried to print $timestamp but nothing is printed.) So I tought that setting IETFTimestampInGMT to true will convert the $EventTime in GMT but it didn't. I checked the documentation and test different configurations, I should have misunderstood the way it works. What did I miss?   Thank you for your help :)