Hello, I hope someone can help me. I need to pick application logs from a few MSSQL database tables. 3 of 4 works perfect -there are no issues getting the data, but one table is causing issues. Below you can see input I am using. As you can see, there is one column name with the Danish character, however, I am not sure if only this is causing trouble. The error I receive is: ERROR SQLExecDirect failed, 42000:2:102:[Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near 'Ą'.; 42000:3:319:[Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a s (odbc error code: -1) However, If I remove this column from SELECT, another error occurs: ERROR id column not found or its type is unsupported. I received a similar error than I left some additional spaces in the query, but after that, I used notepad++ to make sure I don't leave any hidden characters. ========================================================= </Input> <Input odbc_besked> Module im_odbc ConnectionString Driver={SQL Server Native Client 11.0};Server=SERVERNAME\mssql2014,port;Database=database_name;Trusted_Connection=yes; SQL SELECT BeskedType, Oprettet, InternBeskedId, SvarPåInternBeskedId, Retning, BeskedArt, SoapBeskedId, BeskedId, HttpStatusCode, Fra, Til, Failover, EnvelopeStartTag, SoapHeaderElement, IndeholderSoapFault, Stack FROM table_name WITH (NOLOCK) WHERE InternBeskedId > ? </Input>

define ROOT C:\Program Files (x86)\nxlog define ROOT_STRING C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension charconv> Module xm_charconv AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2 </Extension> <Extension fileop> Module xm_fileop </Extension> <Input im_file> Module im_file File "C:\inetpub\vhost\Auth\logs\nlog-test.log" SavePos TRUE Exec $raw_event = '[xxxxxxxxxxxxxxxxxxx]' + $raw_event; Exec $Message = $raw_event; </Input> <Output out> Module om_tcp Host listener.logz.io Port 8010 </Output> <Route route> Path im_file => out </Route>

Hi, I am collecting events (im_msvistalog) from system 7001 and 7002 logon and logoff, I would like to know which field I use to catch the user who is logged on or off, because $UserID returns a SID. Thank you.

Hi, I'm trying to transfer two different csv files having different set of columns to a location. I'm trying to define them in one config file. I would like to know how we define the Extension module for this scenario since xm_csv is going to be the common one for the entire file. Is there any option to use more than one extension module with xm_csv specific to each file? Sample: Is this possible in one config file? Extension <Extension csv_parser> Module xm_csv Fields A,B,C,D Delimiter , </Extension> <Extension csv_parser> Module xm_csv Fields E,F,G Delimiter , </Extension>

I got the okta add-on as part of a trial, but when I am trying to run the nxlog using the below config it doesn't show any data in the output file. Please advise Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogLevel DEBUG NoCache True <Extension _json> Module xm_json </Extension> <Extension _syslog> Module xm_syslog </Extension> <Input okta> Module im_exec Command C:\Program files (x86)\nxlog-okta\nxlog-okta.exe Exec parse_syslog(); <Exec> parse_syslog(); parse_json($Message); </Exec> </Input> <Output file> Module om_file File 'C:\syslog\o0.log' Exec to_json(); </Output> <Route r> Path okta => file </Route> It has following data in nxlog.log file: 2019-04-04 11:10:50 DEBUG new event in event_thread [okta:READ] 2019-04-04 11:10:50 DEBUG nx_event_to_jobqueue: READ (okta) 2019-04-04 11:10:50 DEBUG event added to jobqueue 2019-04-04 11:10:50 DEBUG no events or no future events, event thread sleeping in condwait 2019-04-04 11:10:50 DEBUG worker 1 got signal for new job 2019-04-04 11:10:50 DEBUG worker 1 processing event 0x1dcf30 2019-04-04 11:10:50 DEBUG PROCESS_EVENT: READ (okta) 2019-04-04 11:10:50 DEBUG im_exec_add_read_event with delay 1000000 2019-04-04 11:10:50 DEBUG got EAGAIN 2019-04-04 11:10:50 DEBUG worker 1 waiting for new event 2019-04-04 11:10:50 DEBUG new event in event_thread [okta:READ] 2019-04-04 11:10:50 DEBUG future event, event thread sleeping 1000000ms in cond_timedwait

Hi, I'm new to nxlog. I have recently installed nxlog in my local system and tried to copy nxlog log file which is available in a default location to another location in my machine. Is this feasible using nxlog?

We are using NXLog to relay logs from ModSecurity to AlienVault. The transfer is working but NXLog is adding time and date to the beginning of every line. This is stopping AlienVault from processing the data properly. Is there a way for us to stop NXLog from modifying the sent logs?

Is there any way of splitting very long log messages in half or smaller portions? We are currently forwarding logs with NXLog to a SIEM system that has a 8kb limit on the messages and what goes beyond that limit is truncated and we don't want that. I tried to read the manual but did not find anything related to my problem. Help please?

I was trying to convert JSON to syslog, okta logs are the source of JSON, but couldn't convert okta logs to syslogs and copy the converted logs to a .txt file as I was getting this: Module in2 got EOF from C:\Users\user\output.txt DEBUG got EOF for C:\Users\user\output.txt. Please help me in resolving this. My nxlog config file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log NoCache TRUE LogLevel DEBUG <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input in2> Module im_file File 'C:\Users\user\output.txt' SavePos TRUE ReadFromLast TRUE PollInterval 1 Exec $Message = $to_json; $SyslogFacilityValue = 22; </Input> <Output out> Module om_file File 'C:\syslog\Sysoutput.txt' Exec to_syslog_bsd(); </Output> <Route r> Path in2 => out </Route>

Good Afternoon. I was hoping someone may be able to assist me with an issue I am having sending my logs from IIS in W3C format to Graylog. The W3C time is by default in UTC. When NXlog is sending the logs to my graylog server it is sending logs that are already 4 hours old because I am in EST, but the IIS logs are in UTC. Is there something I can do in the configuration so NXlog is shipping current logs? <Extension w3c> Module xm_csv Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $time-taken FieldTypes string, string, string, string, string, string, integer, string, string, string, string, string, string, string, string, integer Delimiter ' ' QuoteChar '"' EscapeControl FALSE UndefValue - </Extension> <Input iis> Module im_file File "C:\inetpub\logs\LogFiles\\u_ex*" SavePos TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c-&gt;parse_csv(); \ $EventTime = parsedate($date + &quot; &quot; + $time); \ $SourceName = &quot;Server&quot;; \ $Message = to_json(); \ } </Input> Thanks in advance.

Hi Everyone, Can someone help me to read logs from Sybase DB? We are having two instances of Sybase, One is on Windows and second is on Linux. I want to forward these logs over syslog. Thanks in advance!!!

The nxlog service will not start up - I get an error 1053 The service did not respond to the start or control request in a timely fashion. No log file written to the logging directory so cant troubleshoot any further - nothing in the event logs that helps with any answers. Anyone have any experience with running this on Windows 2000 Using the following in the config: <Extension _syslog> Module xm_syslog </Extension> <Input eventlog> Module im_mseventlog </Input> #define PROCESSORS <Processor p_transform> Module pm_transformer Exec $Hostname=hostname(); OutputFormat syslog_rfc5424 </Processor>

Since there's no support for floating point data types in nxlog, given a log entry that contains numbers with decimal points, is the best option to convert them to fixed point integers? For example, given a field $value = "123.45" (a string) extracted from a log line using regex or xm_kvp, if I go directly to_json, I end up with a string in JSON. I don't see any way to put the value, without quotes, into JSON. Am I correct? One workaround is to convert the value to a fixed point integer, choosing a specific precision. For example, if I were to choose to always store my values * 100, I could do the following: if $value =~ /^(-)?([0-9]+)\.?([0-9]+)?$/ { if not defined($3) or size($3) == 0 $value = integer($2) * 100; else if size($3) == 1 $value = integer($2) * 100 + integer($3) * 10; else $value = integer($2) * 100 + integer(substr($3,0,2)); if $1 == '-' $value = $value * -1; } Given $value = "123.45", "123", "123.4567", "123.4", or "123.", this code will assign the correct, 2-place integer value. Is this the best current approach to converting a string representation of a floating point value to something that will result in a non-quoted value in JSON? Thank you!

From the source code, the version.sh file needed a few changes to make it work on Mac. First, the method of deriving the patch number from git needs to trim leading white space. Before, After: git log --pretty=oneline 2>/dev/null | wc -l git log --pretty=oneline 2>/dev/null | wc -l | tr -d ' ' Second, echo -n is not portable across linux versions. To achieve output without a new-line, use printf instead. Before, After: echo -n "${VERSION}.${VERSION_PATCH}" |sed s/M// printf "${VERSION}.${VERSION_PATCH}" I'm not sure what the sed s/M// was for, but maybe it had to do with the svn codepath, which I did not test.

From this example, in the docs: Name=Mike, Weight=64, Age=24, Pet=dog, Height=172 The sample shows accessing the fields like this, effectively casting certain values to integers: if ( integer($Weight) > integer($Height) - 100 ) However, I am using parse_kvp on data that I don't necessarily know the format of, then converting the data into JSON. In JSON, all of the values are quoted, including numeric ones. I need the numeric fields to be unquoted in JSON. Is there a way to iterate through all fields in an Exec statement? I could test for numeric values and reassign them, casting to integer or float (but I don't think there's a float type) I also thought about transforming the JSON string with s/"([0-9.]+)"/$1/g but this sort of regex is not yet supported in nxlog. Any suggestions how to take Name=Mike, Weight=64, Age=24, Pet=dog, Height=172.5 and get the following JSON without referencing the fields by name? {"Name":"Mike","Weight":64,"Age":24,"Pet":"dog","Height":172.5} (no quotes around numbers)

I'm trying to do something like this, but I'm getting a literal "$1" substituted instead of the value of the capture group. if $data =~ s/(\[[^\]]+\])/"$1"/g log_info($1); $data ends up containing a "$1" while the log_info statement correctly logs the value of captured group $1. Is there a way to use s/// and capture groups in the substitution? I also tried \1 and \\1.

Hello, My organization would like to upgrade our version of NXlog CE from 2.9.1347 to 2.10.21250. We have NXlog installed on hundreds of servers, so performing this process manually is not viable. We have tried ad nauseum to script this process, but we keep running into the same issues. It appears the NXlog can only be uninstalled by the user account that installed it initially. NXlog version 2.9.1347 does not show in Add/Remove Programs for users who did not install the application and there for when you attempt to uninstall it via and MSI call it tries to install it instead. There does not seem to be a completely clean way to uninstall this product. Even running the uninstall from the same user account that installed the application, it leaves traces of the application which cause the entry to remain in Add/Remove Programs. My question... We would like to deploy the latest version of NXlog CE using our existing software deployment utility. This requires us to create a batch script to handle the uninstall and reinstall. Is there a way to totally uninstall NXlog CE version 2.9.1347 via CLI quietly or is there a quiet uninstall utility that we can call prior to installing the new version? Thank you.

I don't believe this is currently possible, but hopefully someone can correct me if it's already implemented. Ability to reorder fields in the raw message that gets sent to output module. The way things currently work any fields that are generated during the processing are added on to the end of the message when it gets forwarded to the output destination. The challenge for me is that I generate both a timestamp and a hostname field which then appear at the end of the message. Both of those fields are important for processing during the ingestion of the data on the remote side. Splunk for example by default only reads a certain limited number of characters into each message in order to find a timestamp and host field. I'd love an ability to reorder (or just move to front of the message) the fields that are generated. JSON templating. For use with another pipeline, we have a requirement for a very specific JSON structure that must wrap each message. It's several levels nested and certain fields have to be present in the right place and in the right order for the event message to be accepted/processed. I tried faking it with json flatten and unflatten functions, but they aren't precise enough. Is there a way to define a specific template that should wrap all the messages before being sent to the output?

parsedate is returning undefined for the string directly from the documentation: "2017-Mar-23 06:38:30.143" If I change the "Mar" to "03", making "2017-03-23 06:38:30.143", it works, returning "2017-02-23 06:38:20" Any insights as to why the sample string from the documentation isn't working appreciated!

Hi everybody, I did upgrade my NXLog client from version 3.2.202 to version 4.2.4216, but windows service can not run. But old version worked normally. It was in Microsoft Windows 10 Enterprise 2016 LTSB. And windows application log contains this event: Faulting application name: nxlog.exe, version: 4.2.4216.0, time stamp: 0x00000000 Faulting module name: libnx-0.dll, version: 0.0.0.0, time stamp: 0x00000000 Exception code: 0xc0000005 Fault offset: 0x000000000001196e Faulting process id: 0x1368 Faulting application start time: 0x01d4cdb54eb121da Faulting application path: C:\nxlog\nxlog.exe Faulting module path: C:\nxlog\libnx-0.dll Report Id: 4b6f1d21-d02e-4502-9c20-f6e0cc5f9637 Faulting package full name: Faulting package-relative application ID: Any ideas? Thanks a lot!