I got the okta add-on as part of a trial, but when I am trying to run the nxlog using the below config it doesn't show any data in the output file. Please advise Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogLevel DEBUG NoCache True <Extension _json> Module xm_json </Extension> <Extension _syslog> Module xm_syslog </Extension> <Input okta> Module im_exec Command C:\Program files (x86)\nxlog-okta\nxlog-okta.exe Exec parse_syslog(); <Exec> parse_syslog(); parse_json($Message); </Exec> </Input> <Output file> Module om_file File 'C:\syslog\o0.log' Exec to_json(); </Output> <Route r> Path okta => file </Route> It has following data in nxlog.log file: 2019-04-04 11:10:50 DEBUG new event in event_thread [okta:READ] 2019-04-04 11:10:50 DEBUG nx_event_to_jobqueue: READ (okta) 2019-04-04 11:10:50 DEBUG event added to jobqueue 2019-04-04 11:10:50 DEBUG no events or no future events, event thread sleeping in condwait 2019-04-04 11:10:50 DEBUG worker 1 got signal for new job 2019-04-04 11:10:50 DEBUG worker 1 processing event 0x1dcf30 2019-04-04 11:10:50 DEBUG PROCESS_EVENT: READ (okta) 2019-04-04 11:10:50 DEBUG im_exec_add_read_event with delay 1000000 2019-04-04 11:10:50 DEBUG got EAGAIN 2019-04-04 11:10:50 DEBUG worker 1 waiting for new event 2019-04-04 11:10:50 DEBUG new event in event_thread [okta:READ] 2019-04-04 11:10:50 DEBUG future event, event thread sleeping 1000000ms in cond_timedwait

Hi, I'm new to nxlog. I have recently installed nxlog in my local system and tried to copy nxlog log file which is available in a default location to another location in my machine. Is this feasible using nxlog?

We are using NXLog to relay logs from ModSecurity to AlienVault. The transfer is working but NXLog is adding time and date to the beginning of every line. This is stopping AlienVault from processing the data properly. Is there a way for us to stop NXLog from modifying the sent logs?

Is there any way of splitting very long log messages in half or smaller portions? We are currently forwarding logs with NXLog to a SIEM system that has a 8kb limit on the messages and what goes beyond that limit is truncated and we don't want that. I tried to read the manual but did not find anything related to my problem. Help please?

I was trying to convert JSON to syslog, okta logs are the source of JSON, but couldn't convert okta logs to syslogs and copy the converted logs to a .txt file as I was getting this: Module in2 got EOF from C:\Users\user\output.txt DEBUG got EOF for C:\Users\user\output.txt. Please help me in resolving this. My nxlog config file: define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log NoCache TRUE LogLevel DEBUG <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input in2> Module im_file File 'C:\Users\user\output.txt' SavePos TRUE ReadFromLast TRUE PollInterval 1 Exec $Message = $to_json; $SyslogFacilityValue = 22; </Input> <Output out> Module om_file File 'C:\syslog\Sysoutput.txt' Exec to_syslog_bsd(); </Output> <Route r> Path in2 => out </Route>

Good Afternoon. I was hoping someone may be able to assist me with an issue I am having sending my logs from IIS in W3C format to Graylog. The W3C time is by default in UTC. When NXlog is sending the logs to my graylog server it is sending logs that are already 4 hours old because I am in EST, but the IIS logs are in UTC. Is there something I can do in the configuration so NXlog is shipping current logs? <Extension w3c> Module xm_csv Fields $date, $time, $s-ip, $cs-method, $cs-uri-stem, $cs-uri-query, $s-port, $cs-username, $c-ip, $csUser-Agent, $cs-Referer, $cs-host, $sc-status, $sc-substatus, $sc-win32-status, $time-taken FieldTypes string, string, string, string, string, string, integer, string, string, string, string, string, string, string, string, integer Delimiter ' ' QuoteChar '"' EscapeControl FALSE UndefValue - </Extension> <Input iis> Module im_file File "C:\inetpub\logs\LogFiles\\u_ex*" SavePos TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ w3c-&gt;parse_csv(); \ $EventTime = parsedate($date + &quot; &quot; + $time); \ $SourceName = &quot;Server&quot;; \ $Message = to_json(); \ } </Input> Thanks in advance.

Hi Everyone, Can someone help me to read logs from Sybase DB? We are having two instances of Sybase, One is on Windows and second is on Linux. I want to forward these logs over syslog. Thanks in advance!!!

The nxlog service will not start up - I get an error 1053 The service did not respond to the start or control request in a timely fashion. No log file written to the logging directory so cant troubleshoot any further - nothing in the event logs that helps with any answers. Anyone have any experience with running this on Windows 2000 Using the following in the config: <Extension _syslog> Module xm_syslog </Extension> <Input eventlog> Module im_mseventlog </Input> #define PROCESSORS <Processor p_transform> Module pm_transformer Exec $Hostname=hostname(); OutputFormat syslog_rfc5424 </Processor>

Since there's no support for floating point data types in nxlog, given a log entry that contains numbers with decimal points, is the best option to convert them to fixed point integers? For example, given a field $value = "123.45" (a string) extracted from a log line using regex or xm_kvp, if I go directly to_json, I end up with a string in JSON. I don't see any way to put the value, without quotes, into JSON. Am I correct? One workaround is to convert the value to a fixed point integer, choosing a specific precision. For example, if I were to choose to always store my values * 100, I could do the following: if $value =~ /^(-)?([0-9]+)\.?([0-9]+)?$/ { if not defined($3) or size($3) == 0 $value = integer($2) * 100; else if size($3) == 1 $value = integer($2) * 100 + integer($3) * 10; else $value = integer($2) * 100 + integer(substr($3,0,2)); if $1 == '-' $value = $value * -1; } Given $value = "123.45", "123", "123.4567", "123.4", or "123.", this code will assign the correct, 2-place integer value. Is this the best current approach to converting a string representation of a floating point value to something that will result in a non-quoted value in JSON? Thank you!

From the source code, the version.sh file needed a few changes to make it work on Mac. First, the method of deriving the patch number from git needs to trim leading white space. Before, After: git log --pretty=oneline 2>/dev/null | wc -l git log --pretty=oneline 2>/dev/null | wc -l | tr -d ' ' Second, echo -n is not portable across linux versions. To achieve output without a new-line, use printf instead. Before, After: echo -n "${VERSION}.${VERSION_PATCH}" |sed s/M// printf "${VERSION}.${VERSION_PATCH}" I'm not sure what the sed s/M// was for, but maybe it had to do with the svn codepath, which I did not test.

From this example, in the docs: Name=Mike, Weight=64, Age=24, Pet=dog, Height=172 The sample shows accessing the fields like this, effectively casting certain values to integers: if ( integer($Weight) > integer($Height) - 100 ) However, I am using parse_kvp on data that I don't necessarily know the format of, then converting the data into JSON. In JSON, all of the values are quoted, including numeric ones. I need the numeric fields to be unquoted in JSON. Is there a way to iterate through all fields in an Exec statement? I could test for numeric values and reassign them, casting to integer or float (but I don't think there's a float type) I also thought about transforming the JSON string with s/"([0-9.]+)"/$1/g but this sort of regex is not yet supported in nxlog. Any suggestions how to take Name=Mike, Weight=64, Age=24, Pet=dog, Height=172.5 and get the following JSON without referencing the fields by name? {"Name":"Mike","Weight":64,"Age":24,"Pet":"dog","Height":172.5} (no quotes around numbers)

I'm trying to do something like this, but I'm getting a literal "$1" substituted instead of the value of the capture group. if $data =~ s/(\[[^\]]+\])/"$1"/g log_info($1); $data ends up containing a "$1" while the log_info statement correctly logs the value of captured group $1. Is there a way to use s/// and capture groups in the substitution? I also tried \1 and \\1.

Hello, My organization would like to upgrade our version of NXlog CE from 2.9.1347 to 2.10.21250. We have NXlog installed on hundreds of servers, so performing this process manually is not viable. We have tried ad nauseum to script this process, but we keep running into the same issues. It appears the NXlog can only be uninstalled by the user account that installed it initially. NXlog version 2.9.1347 does not show in Add/Remove Programs for users who did not install the application and there for when you attempt to uninstall it via and MSI call it tries to install it instead. There does not seem to be a completely clean way to uninstall this product. Even running the uninstall from the same user account that installed the application, it leaves traces of the application which cause the entry to remain in Add/Remove Programs. My question... We would like to deploy the latest version of NXlog CE using our existing software deployment utility. This requires us to create a batch script to handle the uninstall and reinstall. Is there a way to totally uninstall NXlog CE version 2.9.1347 via CLI quietly or is there a quiet uninstall utility that we can call prior to installing the new version? Thank you.

I don't believe this is currently possible, but hopefully someone can correct me if it's already implemented. Ability to reorder fields in the raw message that gets sent to output module. The way things currently work any fields that are generated during the processing are added on to the end of the message when it gets forwarded to the output destination. The challenge for me is that I generate both a timestamp and a hostname field which then appear at the end of the message. Both of those fields are important for processing during the ingestion of the data on the remote side. Splunk for example by default only reads a certain limited number of characters into each message in order to find a timestamp and host field. I'd love an ability to reorder (or just move to front of the message) the fields that are generated. JSON templating. For use with another pipeline, we have a requirement for a very specific JSON structure that must wrap each message. It's several levels nested and certain fields have to be present in the right place and in the right order for the event message to be accepted/processed. I tried faking it with json flatten and unflatten functions, but they aren't precise enough. Is there a way to define a specific template that should wrap all the messages before being sent to the output?

parsedate is returning undefined for the string directly from the documentation: "2017-Mar-23 06:38:30.143" If I change the "Mar" to "03", making "2017-03-23 06:38:30.143", it works, returning "2017-02-23 06:38:20" Any insights as to why the sample string from the documentation isn't working appreciated!

Hi everybody, I did upgrade my NXLog client from version 3.2.202 to version 4.2.4216, but windows service can not run. But old version worked normally. It was in Microsoft Windows 10 Enterprise 2016 LTSB. And windows application log contains this event: Faulting application name: nxlog.exe, version: 4.2.4216.0, time stamp: 0x00000000 Faulting module name: libnx-0.dll, version: 0.0.0.0, time stamp: 0x00000000 Exception code: 0xc0000005 Fault offset: 0x000000000001196e Faulting process id: 0x1368 Faulting application start time: 0x01d4cdb54eb121da Faulting application path: C:\nxlog\nxlog.exe Faulting module path: C:\nxlog\libnx-0.dll Report Id: 4b6f1d21-d02e-4502-9c20-f6e0cc5f9637 Faulting package full name: Faulting package-relative application ID: Any ideas? Thanks a lot!

i want to send my atp log to syslog server with the help of nxlog . i am storing atplog to atp.log file and the data is in json format. I am using below configuration but i am not getting anything on my syslog server which is AV. configuration: # Configuration for converting and sending Windows logs # to AlienVault USM Anywhere. # define ROOT C:\Program Files (x86)\nxlog define OUTPUT_DESTINATION_ADDRESS 10.0.2.4 define OUTPUT_DESTINATION_PORT 514 Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension json> Module xm_json </Extension> <Extension syslog> Module xm_syslog </Extension> <Input internal> Module im_internal </Input> <input atplogfile> Module im_atplogfile </input> <Input ATPLogs_IN> Module im_atplogfile File "C:\temp\log\atplogs-.log" Exec $Message = to_json(); to_syslog_bsd(); </Input> <Output ATPLogs_OUT> Module om_atplogfile File "/var/log/nxlog/syslog.log" Exec to_syslog_bsd(); </Output> ######################################## # Routes # ######################################## <Route 1> Path ATPLogs_IN => ATPLogs_OUT </Route> <Input eventlog> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security"></Select> </Query> </QueryList> Exec if ($EventID == 5156) OR ($EventID == 5158) drop(); </Input> <Output out> Module om_udp Host %OUTPUT_DESTINATION_ADDRESS% Port %OUTPUT_DESTINATION_PORT% Exec $EventTime = integer($EventTime) / 1000000; Exec $Message = to_json(); to_syslog_bsd(); </Output> <Route 1> Path eventlog, internal => out </Route>

Hello, I have recently been trying up a syslog-ng server for various devices and have tried a couple of things for sending Windows Events to the server. Finally decieded that NXLog will do what I need and I have gotten sent some events over without much configuration, but when trying filter within the .conf file, it always fails. I can't really find much good information as to why it might be failing, as it seems that it should be correct.(to me anyway) # Windows Event Log, <Input s_eventlog> Module im_msvistalog Exec if $EventID == 4734 or $EventID == 4624 drop(); Exec $Message = to_json(); </Input> I have narrowed it down to this block, since the log says nxlog failed to start: </Input> without matching <Input> section at C:\Program Files (x86)\nxlog\conf\nxlog.conf:43 Which is where this block ends? I can't really make sense of this, so if anyone has some guidance please tell me.

Getting error: Opsec error. rc=-1 err=-96 Connection error step 4 on the user guide page 706 anyone run into issues pulling certs or suggestions? I can telnet using the port no issues, but not able to pull certs. steps prior were all completed.

Hi, I have the following windows log message which i am trying to parse and remove the \r,\t and \n from the log. with i am using i am unable to parse it and it still shows me the log content as is? input: ACBDEFG 12/03/2015 09:05:13 AM\r\nLogName=Security\r\nSourceName=Microsoft Windows security auditing.\r\nEventCode=4672\r\nEventType=0\r\nType=Information\r\nComputerName=VDEEXCP01.teckcominco.loc\r\nTaskCategory=Special Logon\r\nOpCode=Info\r\nRecordNumber=3259542776\r\nKeywords=Audit Success\r\nMessage=msg_somethinh.\r\r\n\r\r\nSubject:\r\r\n\tSecurity ID:\t\tABCDEFFHG-12345\r\r\n\tAccount Name:\t\ABEDCEDDDD$\r\r\n\tAccount Domain:\t\tABCXDDFFEEFFFF\r\r\n\tLogon ID:\t\t98665svdvdvdv\r\r\n\r\r\nPrivileges:\t\tSeSecurityPrivilege\r\r\n\t\t\tSeBackupPrivilege\r\r\n\t\t\tSeRestorePrivilege\r\r\n\t\t\tSeTakeOwnershipPrivilege\r\r\n\t\t\tSeDebugPrivilege\r\r\n\t\t\tSeSystemEnvironmentPrivilege\r\r\n\t\t\tSeLoadDriverPrivilege\r\r\n\t\t\tSeImpersonatePrivilege\r\r\n\t\t\tSeDelegateSessionUserImpersonatePrivilege output: ACBDEFG 12/03/2018 10:15:13 AM\r\nLogName=Security\r\nSourceName=Microsoft Windows security auditing.\r\nEventCode=4672\r\nEventType=0\r\nType=Information\r\nComputerName=VDEEXCP01.teckcominco.loc\r\nTaskCategory=Special Logon\r\nOpCode=Info\r\nRecordNumber=3259542776\r\nKeywords=Audit Success\r\nMessage=msg_somethinh.\r\r\n\r\r\nSubject:\r\r\n\tSecurity ID:\t\tABCDEFFHG-12345\r\r\n\tAccount Name:\t\ABEDCEDDDD$\r\r\n\tAccount Domain:\t\tABCXDDFFEEFFFF\r\r\n\tLogon ID:\t\t98665svdvdvdv\r\r\n\r\r\nPrivileges:\t\tSeSecurityPrivilege\r\r\n\t\t\tSeBackupPrivilege\r\r\n\t\t\tSeRestorePrivilege\r\r\n\t\t\tSeTakeOwnershipPrivilege\r\r\n\t\t\tSeDebugPrivilege\r\r\n\t\t\tSeSystemEnvironmentPrivilege\r\r\n\t\t\tSeLoadDriverPrivilege\r\r\n\t\t\tSeImpersonatePrivilege\r\r\n\t\t\tSeDelegateSessionUserImpersonatePrivilege I am using the community edition and trying out to parse using the following configuration of nxlog. can you please suggest any changes needed in the conf? Global section User nxlog Group nxlog LogFile /var/log/nxlog/nxlog.log LogLevel INFO #Extension section <Extension multi> Module xm_multiline HeaderLine /^================/ EndLine /^---------------/ </Extension> Modules section <Input in> Module im_tcp HOST 0.0.0.0 Port 1532 #InputType multi # Remove the boundary markers Exec if $raw_event =~ s/========[=]+//g {} Make a single line Exec if $raw_event =~ s/[\r\n]/ /g {} #remove the end directive Exec if $raw_event =~ s/--------[-]+//g {} Exec if $raw_event =~ /^ *$/ drop(); Exec $raw_event = replace($raw_event, &quot;\t&quot;, &quot; &quot;); </Input> <Output out> Module om_file File '/tmp/output' #Exec $raw_event = "--------------------------------------\n" + $raw_event; </Output> Route section <Route r> Path in => out </Route>