Hello, we have compiled latest NXLOG Community Edition on AIX V.7.1 with GCC 4.8.xx. Actually we have one issue with "im_file" and logfiles with wildcards like "*". NXLOG quits after writing an "core dump" .... Follwoing Output we are receiving in "DEBUG" mode: 017-11-28 12:13:10 DEBUG pidfile /usr/local/var/run/nxlog/nxlog.pid created 2017-11-28 12:13:10 DEBUG parsing path: itm6_custom_log => out_file 2017-11-28 12:13:10 DEBUG adding module itm6_custom_log to route 1 2017-11-28 12:13:10 DEBUG adding module out_file to route 1 2017-11-28 12:13:10 DEBUG jobgroup created with priority 99 2017-11-28 12:13:10 DEBUG jobgroup created with priority 10 2017-11-28 12:13:10 DEBUG spawning 4 worker threads 2017-11-28 12:13:10 DEBUG worker thread 0 started 2017-11-28 12:13:10 DEBUG worker thread 1 started 2017-11-28 12:13:10 DEBUG worker thread 2 started 2017-11-28 12:13:10 DEBUG worker thread 3 started 2017-11-28 12:13:10 DEBUG event thread started 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (_syslog) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (json) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (was_sys_multi) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (itm6_custom_log) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 WARNING not starting unused module out 2017-11-28 12:13:10 DEBUG nx_event_to_jobqueue: MODULE_START (out_file) 2017-11-28 12:13:10 DEBUG event added to jobqueue 2017-11-28 12:13:10 INFO nxlog-ce-2.8.1248 started 2017-11-28 12:13:10 DEBUG no events or no future events, event thread sleeping in condwait 2017-11-28 12:13:10 DEBUG worker 3 processing event 0x301763f8 2017-11-28 12:13:10 DEBUG PROCESS_EVENT: MODULE_START (itm6_custom_log) 2017-11-28 12:13:10 DEBUG START: itm6_custom_log 2017-11-28 12:13:10 DEBUG Value specified for File parameter contains wildcards: '/usr/app/sw/log/itm6*.log' 2017-11-28 12:13:10 DEBUG reading directory entries under '/usr/app/sw/log' to check for matching files Segmentation fault (core dumped) Somone has the same failure or could help us soliving this issue ??? Greets Alaettin from Stuttgart/Germany

I currently have NXlog community version installed on Windows 2012 R2 server. SEIM Manager is requesting that I stop sending Windows Security Event ID 5156 traffic from server. Is this possible. Thank you.

Hi, I tried schedule 2 jobs for blocking log-messages, as is described in documentation: Example 4.6. Two scheduled jobs in the context of the im_tcp module But I need change blocking mode every minute. For example: every even minut block messages, and every odd minutes pass all messages. I tried to use default syntax from cron: <Processor blocker> Module pm_blocker <Schedule> When 0-59/2 * * * * Exec blocker->block(TRUE); Exec log_info("Block: True"); </Schedule> <Schedule> When 1-59/2 * * * * Exec blocker->block(FALSE); Exec log_info("Block: False"); </Schedule> </Processor> But all this schedules was run simultaneously. How I can schedule this jobs?

It seems I have a problem with Nxlog-ce and Windows eventlog after power resume/reconnect to the network. On the high level we won't get any logs from a a machine before we restart the nxlog service. It shows as runnig but sends no logs. As soon as you restart it, the logs are sent. I Enabled debug logging and got the following 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 26 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG executing statements 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:3 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:4 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:5 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:6 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:7 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:8 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:9 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:10 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:11 2017-11-27 08:02:40 DEBUG evaluating expression 'string literal' at C:\Program Files (x86)\nxlog\conf\add-on\eventlog_client.conf:12 2017-11-27 08:02:40 DEBUG before nx_logqueue_push, size: 27 2017-11-27 08:02:40 DEBUG nx_event_to_jobqueue: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 ERROR Exception was caused by "apr_sockaddr_info_get(&sa, omconf->host, APR_INET, omconf->port, 0, pool)" at om_udp.c:279/om_udp_connect(); [om_udp.c:279/om_udp_connect()] apr_sockaddr_info failed for Myhost.mydomain.XX:12235; Det begärda namnet är giltigt men data för den begärda typen kunde inte hittas. 2017-11-27 08:02:40 DEBUG worker 2 processing event 0x27a5078 2017-11-27 08:02:40 DEBUG PROCESS_EVENT: DATA_AVAILABLE (eventlogOUT) 2017-11-27 08:02:40 DEBUG om_udp_write 2017-11-27 08:02:40 DEBUG module eventlogOUT is not running, not reading any more data 2017-11-27 08:02:40 DEBUG worker 2 waiting for new event 2017-11-27 08:02:40 DEBUG executing statements my NXlog.conf looks like this Nxlog.conf Created: 10/12/2017 15:21:54 LogLevel DEBUG define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> Include plug-in directory include %ROOT%\conf\add-on\*.conf and I have an include file for the eventlog that looks like this <Input eventlogIN> Module im_msvistalog </Input> <Output eventlogOUT> Module om_udp Host myhost.mydomain.xx Port 12235 OutputType GELF </Output> <Route eventlog> Path eventlogIN => eventlogOUT </Route> Has anyone seen this before or got some ideas?

Hello community, I have to collect Security logs from a Windows Server 2003. Sometimes, I have the following errors :   2017-11-22 09:03:52 INFO nxlog-ce-2.9.1504 started 2017-11-22 09:03:52 INFO connecting to siem.nutrition.lan:1514 2017-11-22 09:03:52 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 1 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:03:54 INFO Successfully reopened Security EventLog 2017-11-22 09:03:54 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 2 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:03:57 INFO Successfully reopened Security EventLog 2017-11-22 09:03:57 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 4 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:04:02 INFO Successfully reopened Security EventLog 2017-11-22 09:04:02 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 8 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs. 2017-11-22 09:04:11 INFO Successfully reopened Security EventLog 2017-11-22 09:04:11 WARNING got ERROR_INVALID_PARAMETER (errorcode: 87) for the Security log, will try to reopen in 16 sec. ReadFromLast is TRUE and will try to restart from the last position. This might result in uncollected logs.   Here is my config file :      define ROOT C:\Program Files\nxlog   Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log   ############################ # EXTENSION  ############################   # Enable json extension <Extension json>     Module xm_json </Extension>   # Enable syslog extension <Extension syslog>     Module xm_syslog </Extension>   # Enable conversion module <Extension charconv>     Module xm_charconv     AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2 </Extension>     ############################ # INPUT  ############################   # Nxlog internal logs <Input internal>     Module im_internal     Exec to_json(); </Input>   # Windows Event Log for 2003 server <Input eventlog2003>   # Module for Windows 2003 server     Module im_mseventlog Sources Security     Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000;     Exec convert_fields("AUTO", "utf-8");     Exec $Message = to_json(); to_syslog_bsd(); </Input>     ############################ # OUTPUT  ############################     <Output siem>     Module         om_tcp     Host    collector.company.com     Port    1514 </Output>   <Route 1>     Path     eventlog2003, internal => siem </Route>   I can solve the issue by restarting nxlog agent but this is not a definitive solution... Anybody has the same issue ? 

Hey, I use nxlog to send highly critical log (legal log). NXLog is configured to remove file after a specified retention time, to release space from drive. My configuration works like a charm, but... If I deploy nxlog with this following configuration, nxlog remove all file older than my retention time but before the processing. Every 1 hour Exec file_remove('%LOGFOLDER%', %RETENTIONTIME%); How do you manage this removing process? Right now, I have to deploy nxlog without this schedule, wait for processing, stop nxlog, add schedule job to the configuration, restart nxlog. Hope my case has not been already replied. Thanks for your help :)

HI, I have NXLog (nxlog-ce-2.9.1716) deployed to over 100 Windows servers to send Windows events to Graylog and its working fantastically on all but one server, our main File server. On this one server, the same 5145 events seem to be repeating events up to 19 times - from what I can see, they are identical on Graylog but don't appear duplicated on the file server - the only difference I can see is the volume of events which (by design we need to see file access successes as well as failures and have ABE enabled so see up to 170k events per min). We don't see the issue on any other server, I have implemented the no repeat module (config below) but still no joy. Any suggestions or advice welcome please? NXLog configuration file. See the nxlog reference manual for more info #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension gelf> Module xm_gelf </Extension> <Input in> # Use 'im_mseventlog' for Windows XP and 2003 Module im_msvistalog Exec if ($EventID == 4202 or $EventID == 4208 or $EventID == 4302 or $EventID == 4304 or $EventID == 5004 or $EventID == 5156) drop(); else{ if ( $EventType == "INFO" ) $SyslogSeverityValue = 6; if ( $EventType == "WARNING" ) $SyslogSeverityValue = 4; if ( $EventType == "ERROR" ) $SyslogSeverityValue = 3; } </Input> <Processor norepeat> Module pm_norepeat CheckFields Hostname, SourceName, Message </Processor> <Output out> Module om_udp #Our Graylog server Host [our graylog server] Port 12201 OutputType GELF </Output> <Route 1> Path in => norepeat => out </Route>

I am attempting to use NXLog in conjunction with Graylog's Sidecar Collector to send data to my Graylog server. I have been able to successfully telnet to my Graylog server through port 5044, so I know it is not a firewall issue, yet I keep getting these errors in my NXLog error log: 2017-11-08 14:37:09 ERROR om_tcp detected a connection error;Connection reset by peer 2017-11-08 14:37:25 ERROR om_tcp detected a connection error;End of file found And nothing is being received by my Graylog server. Here is my generated NXLog configuration: define ROOT /usr/bin <Extension gelf> Module xm_gelf </Extension> User nxlog Group nxlog Moduledir /usr/libexec/nxlog/modules CacheDir /var/spool/collector-sidecar/nxlog PidFile /var/run/graylog/collector-sidecar/nxlog.pid define LOGFILE /var/log/graylog/collector-sidecar/nxlog.log LogFile %LOGFILE% LogLevel DEBUG <Extension logrotate> Module xm_fileop <Schedule> When @daily Exec file_cycle('%LOGFILE%', 7); </Schedule> </Extension> <Input 59fcda86ccba8e2573422cb4> Module im_file File '/var/log/httpd/syriac_access_log' PollInterval 1 SavePos True ReadFromLast True Recursive True RenameCheck False Exec $FileName = file_name(); # Send file name with each message </Input> <Output 59fcda56ccba8e2573422c80> Module om_tcp Host graylog.library.vanderbilt.edu Port 5044 OutputType GELF_TCP Exec $short_message = $raw_event; # Avoids truncation of the short_message field. Exec $gl2_source_collector = '485f3ca7-ca1e-4959-be00-117a50e2b1db'; Exec $collector_node_id = 'graylog-collector-sidecar'; Exec $Hostname = hostname_fqdn(); </Output> <Route route-0> Path 59fcda86ccba8e2573422cb4 => 59fcda56ccba8e2573422c80 </Route> I would appreciate any advice or guidance. Thank you!

Hello, I have what I think is a fairly straightforward situation. I'm running queries against an MS-SQL server to retrieve data every 5 minutes. The wrinkle is that I am using unixodbc from a ubuntu 16.04 machine with nxlog to do this. I'm able to retrieve all of the data once on a fresh install of nxlog. However, I'm not seeing new data every 5 minutes and I certain there should be more data as our SQL source is continually writing new entries... My input section has a fairly complicated query... <Input in_epo1> Module im_odbc   ConnectionString DSN=Server;Database=ePO_PROD;UID=epo_user;PWD=epo_pwd;       PollInterval 300     SQL          SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents].[ThreatType] as [threat_type], [EPOEvents].[ThreatEventID] as [signature_id], [EPOEvents].[ThreatCategory] as [category], [EPOEvents].[ThreatSeverity] as [severity_id], [EPOEventFilterDesc].[Name] as [event_description], [EPOEvents].[DetectedUTC] as [detected_timestamp], [EPOEvents].[TargetFileName] as [file_name], [EPOEvents].[AnalyzerDetectionMethod] as [detection_method], [EPOEvents].[ThreatActionTaken] as [vendor_action], CAST([EPOEvents].[ThreatHandled] as int) as [threat_handled], [EPOEvents].[TargetUserName] as [logon_user], [EPOComputerProperties].[UserName] as [user], [EPOComputerProperties].[DomainName] as [dest_nt_domain], [EPOEvents].[TargetHostName] as [dest_dns], [EPOEvents].[TargetHostName] as [dest_nt_host], [EPOComputerProperties].[IPHostName] as [fqdn], [dest_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),4,1))) ), [EPOComputerProperties].[SubnetMask] as [dest_netmask], [EPOComputerProperties].[NetAddress] as [dest_mac], [EPOComputerProperties].[OSType] as [os], [EPOComputerProperties].[OSServicePackVer] as [sp], [EPOComputerProperties].[OSVersion] as [os_version], [EPOComputerProperties].[OSBuildNum] as [os_build], [EPOComputerProperties].[TimeZone] as [timezone], [EPOEvents].[SourceHostName] as [src_dns], [src_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),4,1))) ), [EPOEvents].[SourceMAC] as [src_mac], [EPOEvents].[SourceProcessName] as [process], [EPOEvents].[SourceURL] as [url], [EPOEvents].[SourceUserName] as [source_logon_user], [EPOComputerProperties].[IsPortable] as [is_laptop], [EPOEvents].[AnalyzerName] as [product], [EPOEvents].[AnalyzerVersion] as [product_version], [EPOEvents].[AnalyzerEngineVersion] as [engine_version], [EPOEvents].[AnalyzerEngineVersion] as [dat_version], [EPOProdPropsView_VIRUSCAN].[datver] as [vse_dat_version], [EPOProdPropsView_VIRUSCAN].[enginever64] as [vse_engine64_version], [EPOProdPropsView_VIRUSCAN].[enginever] as [vse_engine_version], [EPOProdPropsView_VIRUSCAN].[hotfix] as [vse_hotfix], [EPOProdPropsView_VIRUSCAN].[productversion] as [vse_product_version], [EPOProdPropsView_VIRUSCAN].[servicepack] as [vse_sp] FROM [EPOEvents] left join [EPOLeafNode] on [EPOEvents].[AgentGUID] = [EPOLeafNode].[AgentGUID] left join [EPOProdPropsView_VIRUSCAN] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_VIRUSCAN].[LeafNodeID] left join [EPOComputerProperties] on [EPOLeafNode].[AutoID] = [EPOComputerProperties].[ParentID] left join [EPOEventFilterDesc] on [EPOEvents].[ThreatEventID] = [EPOEventFilterDesc].[EventId] and (EPOEventFilterDesc.Language='0409') WHERE (DATEDIFF(hour, CAST([EPOEvents].[ReceivedUTC] as date), getutcdate()) between 0 and 2) AND [EPOEvents].[AutoID] > ?       Exec delete($timestamp_epo); </Input>   To me this looks like a fairly standard query - nxlog should save its position and know to query [EPOEvents].[AutoID] > lastseen_autoid. We added (DATEDIFF(hour, CAST([EPOEvents].[ReceivedUTC] as date), getutcdate()) between 0 and 2) into the mix to limit our results to the last two hours, however, we could equally remove this.   So I enabled ODBC tracing and it certainly seems like nxlog is querying the database successfully. Here's a short snippet:   [ODBC][10][1509980277.861057][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fa8000eb0 Window Hdl = (nil) Str In = [DSN=DATABASE2;Database=DATABASE2;UID=user2;PWD=*************;][length = 58 (SQL_NTS)] Str Out = 0x7f6fb8eb9bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb8eb9144 Completion = 0 [ODBC][10][1509980277.861922][SQLAllocHandle.c][375] Entry: Handle Type = 2 Input Handle = 0x7f6fa00008c0 [ODBC][10][1509980277.861965][SQLAllocHandle.c][493] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa0000ef0 [ODBC][10][1509980277.862003][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fa0000ef0 Window Hdl = (nil) Str In = [DSN=DATABASE1;Database=DATABASE1;UID=user1;PWD=***;][length = 72 (SQL_NTS)] Str Out = 0x7f6fb6eb5bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb6eb5144 Completion = 0 [ODBC][10][1509980277.862040][SQLSetEnvAttr.c][363] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.862078][SQLAllocHandle.c][375] Entry: Handle Type = 2 Input Handle = 0x7f6fb00009b0 [ODBC][10][1509980277.862093][SQLAllocHandle.c][493] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fb0001270 [ODBC][10][1509980277.862115][SQLDriverConnect.c][726] Entry: Connection = 0x7f6fb0001270 Window Hdl = (nil) Str In = [DSN=DATABASE3;Database=DATABASE3;UID=user2;PWD=*************;][length = 58 (SQL_NTS)] Str Out = 0x7f6fb7eb7bd0 Str Out Max = 1024 Str Out Ptr = 0x7f6fb7eb7144 Completion = 0 UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE1'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.892817][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE1;UID=user1;PWD=********;WSID=8ef53561e...] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE1'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.892903][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fa0000ef0 [ODBC][10][1509980277.892936][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa003f1b0 [ODBC][10][1509980277.892952][SQLBindParameter.c][217] Entry: Statement = 0x7f6fa003f1b0 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc3b0e28 Value Max = 0 StrLen Or Ind = (nil) [ODBC][10][1509980277.892974][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.893002][SQLExecDirect.c][240] Entry: Statement = 0x7f6fa003f1b0 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE3'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.893946][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE3;UID=user2;PWD=********;WSID=8ef53561e30d;DATABAS...] DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE2'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE3'.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.894011][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fb0001270 [ODBC][10][1509980277.894023][SQLDriverConnect.c][1582] Exit:[SQL_SUCCESS_WITH_INFO] Connection Out [DSN=DATABASE2;UID=user2;PWD=********;WSID=8ef53561e30d;DATABAS...] [ODBC][10][1509980277.894038][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fb0052780 DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed database context to 'DATABASE2'.   [ODBC][10][1509980277.894058][SQLBindParameter.c][217] Entry: Statement = 0x7f6fb0052780 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc38ce28 Value Max = 0 StrLen Or Ind = (nil) DIAG [01000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Changed language setting to us_english.   [ODBC][10][1509980277.894079][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.894107][SQLAllocHandle.c][540] Entry: Handle Type = 3 Input Handle = 0x7f6fa8000eb0 [ODBC][10][1509980277.894112][SQLExecDirect.c][240] Entry: Statement = 0x7f6fb0052780 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] [ODBC][10][1509980277.894138][SQLAllocHandle.c][1081] Exit:[SQL_SUCCESS] Output Handle = 0x7f6fa8067230 [ODBC][10][1509980277.894159][SQLBindParameter.c][217] Entry: Statement = 0x7f6fa8067230 Param Number = 1 Param Type = 1 C Type = -18 SQL_C_ULONG SQL Type = -5 SQL_BIGINT Col Def = 0 Scale = 0 Rgb Value = 0x7f6fbc39ee28 Value Max = 0 StrLen Or Ind = (nil) [ODBC][10][1509980277.894179][SQLBindParameter.c][397] Exit:[SQL_SUCCESS] [ODBC][10][1509980277.894211][SQLExecDirect.c][240] Entry: Statement = 0x7f6fa8067230 SQL = [SELECT [EPOEvents].[AutoID], [EPOEvents].[ReceivedUTC] as [timestamp_epo], [EPOEvents].[ThreatName] as [signature], [EPOEvents]....][length = 3793 (SQL_NTS)] [ODBC][10][1509980278.153007][SQLExecDirect.c][503] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153074][SQLNumResultCols.c][156] Entry: Statement = 0x7f6fb0052780 Column Count = 0x7f6fbc38ce80 [ODBC][10][1509980278.153103][SQLNumResultCols.c][248] Exit:[SQL_SUCCESS] Count = 0x7f6fbc38ce80 -> 44 [ODBC][10][1509980278.153133][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 1 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a0 Column Size = 0x7f6fbc2d83b8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153169][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [AutoID]                 Data Type = 0x7f6fbc2d80a0 -> 4                 Column Size = 0x7f6fbc2d83b8 -> 10                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153197][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 1 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153247][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153274][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 2 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a2 Column Size = 0x7f6fbc2d83c0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153303][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [timestamp_epo]                 Data Type = 0x7f6fbc2d80a2 -> 93                 Column Size = 0x7f6fbc2d83c0 -> 23                 Decimal Digits = 0x7f6fb7eb654e -> 3                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153326][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 2 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153350][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153373][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 3 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a4 Column Size = 0x7f6fbc2d83c8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153401][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [signature]                 Data Type = 0x7f6fbc2d80a4 -> -9                 Column Size = 0x7f6fbc2d83c8 -> 128                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153424][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 3 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153447][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153560][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 4 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a6 Column Size = 0x7f6fbc2d83d0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153589][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [threat_type]                 Data Type = 0x7f6fbc2d80a6 -> -9                 Column Size = 0x7f6fbc2d83d0 -> 32                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153612][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 4 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153635][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153732][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 5 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80a8 Column Size = 0x7f6fbc2d83d8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153760][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [signature_id]                 Data Type = 0x7f6fbc2d80a8 -> 4                 Column Size = 0x7f6fbc2d83d8 -> 10                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153783][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 5 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153814][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.153837][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 6 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80aa Column Size = 0x7f6fbc2d83e0 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.153864][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [category]                 Data Type = 0x7f6fbc2d80aa -> -9                 Column Size = 0x7f6fbc2d83e0 -> 128                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0 [ODBC][10][1509980278.153887][SQLColAttribute.c][293] Entry: Statement = 0x7f6fb0052780 Column Number = 6 Field Identifier = SQL_DESC_TYPE_NAME Character Attr = 0x7f6fb7eb7000 Buffer Length = 256 String Length = (nil) Numeric Attribute = (nil) [ODBC][10][1509980278.153909][SQLColAttribute.c][664] Exit:[SQL_SUCCESS] [ODBC][10][1509980278.154007][SQLDescribeCol.c][247] Entry: Statement = 0x7f6fb0052780 Column Number = 7 Column Name = 0x7f6fb7eb6f00 Buffer Length = 255 Name Length = 0x7f6fb7eb654c Data Type = 0x7f6fbc2d80ac Column Size = 0x7f6fbc2d83e8 Decimal Digits = 0x7f6fb7eb654e Nullable = 0x7f6fb7eb6550 [ODBC][10][1509980278.154035][SQLDescribeCol.c][497] Exit:[SQL_SUCCESS]                 Column Name = [severity_id]                 Data Type = 0x7f6fbc2d80ac -> -6                 Column Size = 0x7f6fbc2d83e8 -> 3                 Decimal Digits = 0x7f6fb7eb654e -> 0                 Nullable = 0x7f6fb7eb6550 -> 0   However nothing is being written to the output json file.   Does it look like I'm doing anything wrong? Is there anything else I should try?  

xm_perl provides enhanced capabilities to run scripts that can enhance the logs/events that are generated. Is there an expected release soon that will include this functionality for the Windows version?

Hello, could someone point me into the direction of  how to configure nxlog enterprise with kafka topic subscriptions for pub/sub? I have only seen mention of the capability, though no specifics or connector?   Thank you,   Rob 

Yet the source code for this module is present in the (relatively old) source .tar.gz. Should the .debs contain im_mvistalog.so? Is there a plan to release newer sources?  If we use nxlog I'd prefer to deploy on Alpine Linux Is there a way to search the forum?  I'm sure these questions have been asked already Thanks!

Hi, i am using the nxlog-ce-2.9.1716-1_rhel7.x86_64.rpm and i have the following conf <Output graylogout> Module om_udp Host somehost.com Port 12201 OutputType GELF </Output> If i do "host somehost.com" -> It returns 2 IP in different order $>host somehost.com X.X.X.X Y.Y.Y.Y $>host somehost.com Y.Y.Y.Y X.X.X.X But nxlog always send messages to only one host (X.X.X.X). Does nxlog do a DNS request once? if is that the case, how do i balance the traffic?

Hello, Does NXlog provide a classification taxonomy or is classification up to the customer? Do you have recommended taxonomies? Thank you.

Hello,   I would like to forward logs generated by HypErV. In event viewer they are in Applications and Services logs -> Microsoft -> Windows -> Hyper-V-* I tried to add a query in the input configuration but I didn't manage to find the correct path to configure. I tried several path but each time I have an error "the channel was not found " Query <QueryList> \            <Query Id="0"> \             <Select Path="Security">*</Select> \             <Select Path="Hyper-V-logs">*</Select> \            </Query> \           </QueryList> </Input> What should I put instead of Hyper-V-logs to send hyperV logs? Can someone help me?   Regards,  

Hi all,  I'm trying to troubleshoot an issue with getting the Checkpoint input module working and pulling syslogs from a Checkpoint management appliance. I'm following the configuration guide provided in the nxlog documentation, but I am running into a few issues.  Specifically on the step where I attempt to retrieve the authentication key by using opsec_putkey. The correct output from that command should be "FW: Received new control security key from IP; Authentication with IP initialized successfully." However, my output is " FW: Received new control security key from IP; Failed to initialize authentication with IP." When I go to test whether the log collection works because I do end up receiving the sslauthkeys.c and sslses.c files, and I have the opsec.p12 cert from a previous step, I get the following response - "Peer IP wants to exchange keys but I don't have a password." Google hasn't been very fruitful so any advice would be appreciated.  -M

Hello good people! Is there any chance for Community Edition to get deb package for Debian 9 (Stretch)? Many thanks in advance!  

  Hi All,  I am on Cent OS 7 and installed nxlog to forward the /var/log/messages and /var/log/secure to my logging server. When I start its starting fine and in logs I am gettting weird log which I am not able to solve.  ERROR 2017-10-19 13:32:16 ERROR om_tcp detected a connection error;End of file found 2017-10-19 13:32:17 INFO connecting to 10.12.86.87:12225 2017-10-19 13:32:17 INFO reconnecting in 1 seconds 2017-10-19 13:32:17 ERROR om_tcp detected a connection error;End of file found Config:  ######################################## # Global directives                    # ######################################## User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO ######################################## # Modules                              # ######################################## <Input messages>     Module              im_file     File                "/var/log/messages" SavePos           TRUE </Input> <Input securelogs>     Module              im_file     File                "/var/log/secure" SavePos TRUE </Input> <Output logserver>     Module              om_tcp     Host                10.12.86.87     Port                12225 </Output> <Route Log-TCP>     Path        messages, securelogs => logserver </Route>   Selinux and Firewall is turned off in my machine. And remote server is listenting on the port 12225 and that can be reachable from this machine without any issues. Can you help me with what I can do to solve this issue.    Thanks in advance 

hi, I am tesing nxlog on Ubuntu 16.04 LTS with the configuration <Extension fileop>         Module xm_fileop </Extension> <Input udpin>         Module im_udp         Host 127.0.0.1         Port 1514         Exec $HOSTIP = string(host_ip());         Exec file_write("/tmp/debug.txt", "HostIP:" + $HOSTIP); </Input> <Output udpfile>         Module om_file         File "/var/log/udp.log" </Output> <Route udp>     Path udpin => udpfile </Route> and  I generated log using linux command nc -u 127.0.0.1 1514 and can not get IP in /tmp/debug.txt. 1. I have reinstall the Ubuntu 16.04 LTS purely and install the deb file from nxlog.co， and it still does not get the IP address  2. I install the nxlog from source code, and it still does not get the IP address 3. I test the same configuration on CentOS7, CentOS6, Ubuntu 14.04 LTS. It works correctly. so， is there someting that does not compatible with the Ubuntu 16.04 LTS?

Hi I'm trying to do a syslog udp to tcp converter using this method from the user's manual: <Input in>     Module im_tcp     Host 0.0.0.0     Port 2345  </Input> <Output out>     Module      om_tcp     Host        mysyslog.domain.local     Port        514 </Output> ~~<Processor buffer>  Module pm_buffer  WarnLimit 800  MaxSize 1000  Type Mem  Exec if buffer_size() >= 80k drop(); </Processor> <Route 1>     Path in => buffer => out </Route> This is working correctly but all the events recieved by mysyslog.domain.local server are coming from the same source (the IP of the NXLOG server). Is there a way to preserve the source IP? Thanks in advance Regards, Olga