I'm getting in syslog info from network devices. It looks like this:

id=scsonicwall sn=18B169F5XXXX time="2020-08-24 19:32:49" fw=64.20.130.54 pri=6 c=1024 gcat=6 m=537 msg="Connection Closed" srcMac=00:01:5c:71:c6:46 src=173.188.249.226:58706:X1 srcZone=Untrusted natSrc=173.188.249.226:58706 dstMac=00:50:56:80:66:a6 dst=10.10.12.5:443:X0 dstZone=Trusted natDst=X.x.x.x:443 proto=tcp/https sent=920 rcvd=3262 spkt=10 rpkt=6 cdur=666 rule="14 (WAN->LAN)" app=11 n=2617279 fw_action="NA" dpi=0

When NXLog is relaying this out to Loggly, it's boogering up the timestamps:

<134>1 2020-12-31T19:00:00.000000-05:00 10.10.12.1 - - - [XXXXXX@41058 tag="windows"] {"MessageSourceAddress":"10.10.12.1","EventReceivedTime":"2020-08-24 20:05:06","SourceModuleName":"udp","SourceModuleType":"im_udp","SyslogFacilityValue":16,"SyslogFacility":"LOCAL0","SyslogSeverityValue":6,"SyslogSeverity":"INFO","SeverityValue":2,"Severity":"INFO","Hostname":"10.10.12.1","EventTime":"2020-12-31 19:00:00","Message":"id=scsonicwall sn=18B169F52958 time=&quot;2020-08-24 20:05:06&quot; fw=X.X.X.X pri=6 c=262144 gcat=6 m=98 msg=&quot;Connection Opened&quot; src=206.74.83.165:61555:X1 natSrc=206.74.83.165:61555 dst=10.10.12.5:443:X0 natDst=X.X.X.X:443 proto=tcp/https sent=52 app=11 n=131486 fw_action=&quot;NA&quot; dpi=0"}

Note the the "time" field internal to the message is correct, but the EventTime and the timestamp at the beginning of the message are completely wrong.

My config is below. What do I need to do to fix this?

This is a sample NXLog configuration file created by Loggly. June 2013

See the nxlog reference manual about the configuration options.

It should be installed locally and is also available

online at https://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html

Please set the ROOT to the folder your nxlog was installed into,

otherwise it will not start.

#define ROOT C:\Program Files\nxlog #define ROOT_STRING C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define ROOT_STRING C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

Include fileop while debugging, also enable in the output module below

<Extension fileop> Module xm_fileop </Extension>

<Extension json> Module xm_json </Extension>

<Extension syslog> Module xm_syslog </Extension>

<Input internal> Module im_internal Exec $Message = to_json(); </Input>

Windows Event Log

#<Input eventlog>

Uncomment im_msvistalog for Windows Vista/2008 and later

#Module im_msvistalog

#Uncomment im_mseventlog for Windows XP/2000/2003 #Module im_mseventlog #Exec $Message = to_json(); #</Input>

<Processor buffer> Module pm_buffer

100Mb disk buffer

MaxSize 102400 Type disk </Processor>

<Input udp> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog_ietf(); Exec $Message = to_json();

</Input>

<Output out> Module om_tcp Host logs-01.loggly.com Port 514

Exec to_syslog_ietf(); $raw_event =~ s/\[NXLOG.*?\]/\[XXXXXXXXXXXX@41058 tag="windows"\]/g;

</Output>

<Route 1> Path udp, internal => buffer => out </Route>

<Output out> Module om_tcp Host 10.xx.xx.10 Port 10514 Exec to_json(); </Output>

<Route 1> Path eventlog,dnslog => out </Route>

Hi There,

When using the “om_tcp” ouput module, is there a tcp timeout related to this? Or does nxlog open a connection and let it sit idle until data is sent?

Many thanks

Craig

Trying to set up NXLog to send syslog info from network devices to Loggly. I can see that it's sending the data and then relaying it to Loggly, but it is NOT manipulating the headers for the inputs received via UDP 514. It does send the Windows information (which I don't even want). I can see the information going out, unchanged using wireshark.

Below is what I've got set up, and I would appreciate any assistance (or maybe a WORKING configuration sample) to straighten this out.

<Input udp> Module im_udp Host 0.0.0.0 Port 514 Exec parse_syslog(); </Input>

<Output out> Module om_tcp Host logs-01.loggly.com Port 514

Exec to_syslog_ietf(); Exec $raw_event =~ s/([.*])//g; $raw_event = replace($raw_event, '{', '[XXXXXXXXXXXXXXXXXXXXXXXX@41058 tag="windows"] {', 1);

#Use the following line for debugging (uncomment the fileop extension above as well) Exec file_write("C:\Program Files (x86)\nxlog\data\nxlog_output.log", $raw_event); </Output> <Route 1> Path udp, internal, eventlog => buffer => out Path udp, internal, eventlog => out </Route>

Hi everyone! I`m trying to delay an event processing for a second, i.e.: if an event has a description I'll delay the output for a period. I found the sleep method in the Documentation, but it's not working as intended. Below is how my input is configured: Module im_file File "%LOGPATH%" ReadFromLast TRUE Recursive FALSE if ($raw_event =~ /UPDATE/) { parse_json(); } else if ($raw_event =~ /DELETE/) { sleep(2000) # wait 2 seconds before processing this event parse_json(); } Is is possible to delay/wait for a few seconds before processing an event? Thank you in advance!

Good day i need help buying the enterprise version, ive sent multiple emails but i still haven't had assistance

Hi I am trying to install nxlog on XP machine but I couldn't troubleshoot the error. Following is the error I am seeing on XP machine:

2020-08-20 15:54:06 WARNING nxlog-ce received a termination request signal, exiting... 2020-08-20 15:54:10 ERROR invalid keyword: Query at C:\Program Files\nxlog\conf\nxlog.conf:37 2020-08-20 15:54:10 ERROR module 'in' has configuration errors, not adding to route '1' at C:\Program Files\nxlog\conf\nxlog.conf:52 2020-08-20 15:54:10 ERROR route 1 is not functional without input modules, ignored at C:\Program Files\nxlog\conf\nxlog.conf:52 2020-08-20 15:54:10 WARNING no routes defined! 2020-08-20 15:54:10 WARNING not starting unused module in 2020-08-20 15:54:10 WARNING not starting unused module out 2020-08-20 15:54:10 INFO nxlog-ce-2.10.2150 started

I cant seem to get my im_file input to find my files. It is from a network mount that is mapped to the Z drive.

This is the actual file: Z:\WebCom\DDIWebCm Log Thursday, August 20, 2020.txt

I have tried to point the input directly to it and even tried wildcards. For example: File "Z:\WebCom\DDIWebCm*.txt" Yields 2020-08-20 13:50:29 ERROR failed to open directory: Z:\WebCom: The system cannot find the path specified.

Any ideas for me? Thanks in advance

Hi

Requirement: Reading logs of multiple servers from a single server

Is there any way to read a network shared log file like "\RemoteServer\Remotelog.log" using nxlog.

I tired using im_file module but not succeed, kindly let me know whether it is possible or not.

BR, Ravi

How can I get the filepath from im_file and pass it as variable in the Output.. as you can see i am using Recursive TRUE, want's to know which file from which directory, current message came from

<Input tst> Module im_file File "/home/*.log" Recursive TRUE </Input>

<Output tst_testfile> Module om_file File "/home/user/nxlogtest" <Exec> $tst_filepath = ??; $Message = $raw_event; to_json(); </Exec> </Output>

<Route tst_out> Path maas => maas_testfile </Route>

Hi all,

I want to send this text-logfiles to a syslog service. Now I´m stuck with xm_kvp, because it manipualtes the data in a way I don´t expect. The data looks as follows:

"srv1","IAS",08/01/2020,10:33:39,1,"123\xyz","123\xyz","dc-aa-94-96-52-70:WLAN-XYZ","f8-aa-4e-24-bc-7c",,,"wlc01","1.1.1.1",13,0,"1.1.1.1","wlc01",,,19,,,2,5,"wlan-xyz-01",0,"311 1 4.4.4.4 03/25/2020 14:12:43 689",,,,,,,,,"5e8427b3/f8:aa:4e:24:bc:7c/133777",,,,,,,,,13,6,,,,"49",,,,,,,,,,,"Wireless-hj",1,,,,


So there is comma separated values, without fieldnames. I want to create KVP values and send it to syslog via UDP. Lets focus on the formatted data.




So this is my code:

#####################
<Extension csv1>
Module xm_csv
Fields $ComputerName, $ServiceName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Callback-Number, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-Vendor, $Client-IP-Address, $Client-Friendly-Name, $Event-Timestamp, $Port-Limit, $NAS-Port-Type, $Connect-Info, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Class, $Session-Timeout, $Idle-Timeout, $Termination-Action, $EAP-Friendly-Name, $Acct-Status-Type, $Acct-Delay-Time, $Acct-Input-Octets, $Acct-Output-Octets, $Acct-Session-Id, $Acct-Authentic, $Acct-Session-Time, $Acct-Input-Packets, $Acct-Output-Packets, $Acct-Terminate-Cause, $Acct-Multi-Ssn-ID, $Acct-Link-Count, $Acct-Interim-Interval, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Client-Endpt, $Tunnel-Server-Endpt, $Acct-Tunnel-Conn, $Tunnel-Pvt-Group-ID, $Tunnel-Assignment-ID, $Tunnel-Preference, $MS-Acct-Auth-Type, $MS-Acct-EAP-Type, $MS-RAS-Version, $MS-RAS-Vendor, $MS-CHAP-Error, $MS-CHAP-Domain, $MS-MPPE-Encryption-Types, $MS-MPPE-Encryption-Policy, $Proxy-Policy-Name, $Provider-Type, $Provider-Name, $Remote-Server-Address, $MS-RAS-Client-Name, $MS-RAS-Client-Version
#EscapeControl FALSE
Delimiter ,
</Extension>
<Extension csv2>
Module xm_csv
Fields $ComputerName, $Record-Date, $Record-Time, $Packet-Type, $User-Name, $Fully-Qualified-Distinguished-Name, $Called-Station-ID, $Calling-Station-ID, $Framed-IP-Address, $NAS-Identifier, $NAS-IP-Address, $NAS-Port, $Client-IP-Address, $Client-Friendly-Name, $Framed-Protocol, $Service-Type, $Authentication-Type, $Policy-Name, $Reason-Code, $Tunnel-Type, $Tunnel-Medium-Type, $Tunnel-Pvt-Group-ID
Delimiter ;
#EscapeControl False
EscapeChar \n
</Extension>
<Extension kvp1>
Module xm_kvp
#Delimiter ''
#ValueQuoteChar "
QuoteMethod All
#KVDelimiter =
EscapeChar \n
KVPDelimiter ;
IncludeHiddenFields False
</Extension>
<Extension kvp2>
Module xm_kvp
KVPDelimiter ;
</Extension>
<Input in>
Module im_file
File "d:\\nxlog\\IN2004.log"
InputType LineBased
PollInterval 1
ReadFromLast FALSE
SavePos FALSE
<Exec>
csv1->parse_csv();
if not defined $number $number = 0;
csv2->to_csv();
kvp1->to_kvp();
delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);
</Exec>
</Input>

#####################

Till csv2->to_csv();, it works fine and the output is as expected. Values are now semicolon seperated and surrounded by quotation marks "". This is what I want.

But know, when kvp1->to_kvp(); is also active, quotes are removed from all values but values with spaces in it. I do not want to change the quotes surrounding the values.

The result looks like this:

EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;ComputerName=srv1;ServiceName=IAS;Record-Date=08/01/2020;Record-Time=10:33:39;Packet-Type=1;User-Name=123\\xyz;Fully-Qualified-Distinguished-Name=123\\xyz;Called-Station-ID=dc-aa-94-96-52-70:WLAN-XYZ;Calling-Station-ID=f8-aa-4e-24-bc-7c;NAS-Identifier=wlc01;NAS-IP-Address=1.1.1.1;NAS-Port=13;Client-Vendor=0;Client-IP-Address=1.1.1.1;Client-Friendly-Name=wlc01;NAS-Port-Type=19;Service-Type=2;Authentication-Type=5;Policy-Name=wlan-xyz-01;Reason-Code=0;Class='311 1 4.4.4.4 03/25/2020 14:12:43 689';Acct-Session-Id=5e8427b3/f8:aa:4e:24:bc:7c/133777;Tunnel-Type=13;Tunnel-Medium-Type=6;Tunnel-Pvt-Group-ID=49;Proxy-Policy-Name=Wireless-hj;Provider-Type=1;number=0;

So does somebody know, why the quotes are beeing removed in general, but only kept for values with spaces in it?

Additionally, I would like to remove these fields: "EventReceivedTime=2020-08-19 17:53:39;SourceModuleName=in;SourceModuleType=im_file;"

by using

delete($EventReceivedTime);
delete($SourceModuleName);
delete($SourceModuleType);


but it is also not working.

Any ideas?



Thanks!

We need to send Cortex Data Lake logs (IETF) to Arcsight (CEF). Will I be able to set up NXLog to do this conversion for me?

Hi All,

I wonder if someone can answer this for me.

According to the documentation, it states that for a UDP client, the localport will be a random high port as per https://nxlog.co/documentation/nxlog-user-guide/om_udp.html

I have a situation where I am sending Zeek logs via UDP through a Google Seesaw load balancer see https://github.com/google/seesaw

The issue I am facing is that each separate log packet / connection from NXLog has the same client source port i.e 41460 in my case.

Tcpdump confirms this

Packet 1 15:55:10.533740 IP (tos 0x0, ttl 64, id 57228, offset 0, flags [DF], proto UDP (17), length 506) 172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 478

Packet 2 15:55:10.534026 IP (tos 0x0, ttl 64, id 57229, offset 0, flags [DF], proto UDP (17), length 847)172.16.4.10.41640 > 172.16.4.166.12210: [udp sum ok] UDP, length 819

Is there a way to get NXLog to use a random client port for each connection?

It looks as if it chooses a random high port when the service is started.

Cheers

Cyberkryption

2020-08-17 16:31:18 INFO nxlog-ce-2.10.2150 started 2020-08-17 16:31:18 ERROR couldn't connect to udp socket on 10.0.20.99:12201; A socket operation was attempted to an unreachable network.

why I am getting this error?How can I solve this?

NXLog-CE version- 2.10.2150

Hi, we provide a SIEM solution for our customers, using AlienVault USM appliance and we are trying to implement/test NXlog for their servers. Has anyone implemented Oracle monitoring on Windows platform in NXlog and could share his experience? Don't seem to find any documentation for it. Many thanks in advance

Internet Explorer Logs Appear to have been deleted after installing nxlog. We received an alert shortly after installing nxlog on our server. After digging, they appear to be temporary log files. We have a very basic configuration with no purging explicitly defined. Is this normal behaviour?

Hi,

in my design, I use NXlog Community Edition servers as proxy collectors in network security zones; all production servers forward their logs to their closest NXlog proxy collector node, which in turn forwards to a SIEM server Output target. My question is: On a such collector node, can I parse the incoming data and if coming from a certain production server Input module instance, e.g. <Input myInput1>, forward only this data to a secondary Output target? The challenge lies in the fact that currently I've only got one collector node per security zone. The individual production server can only forward to the collector in the same zone, otherwise I would have created a separate Output instance and a Route for the particular Input instance to the secondary server.

Good day Family

I have a problem with nxlog on Linux, I am having difficulties pulling records from an oracle DB using the agent. Has anyone done it before?

Please help

Hi again! i want to check this config its ok. I need send a .csv to graylog and the graylog server is not getting messages and I wanted to check that the nxlog configuration was well done. The nxlog log, start without problems

#define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log

<Extension json> Module xm_json </Extension>

<Extension fileop> Module xm_fileop </Extension>

<Extension _syslog> Module xm_syslog </Extension>

<Extension gelf> Module xm_gelf </Extension>

<Extension jira> Module xm_csv Fields $ComputerName,$SID,$Message FieldTypes string,string,string Delimiter"," </Extension>

<Input in> Module im_file File "C:\logs\logs.csv" #ReadFromLast False #Recursive True #SavePos True

</Input>

<Output out> Module om_udp Host 172.28.36.25 Port 12201 #Exec to_syslog_snare(); OutputType GELF </Output>

<Route 1> Path in => out </Route>

Thanks all!

Is it possible to force the agent to send the IP address of the host in the syslog message?

Either send Only the IP or at least ensure the IP is included with the hostname.

You can get microseconds of an DateTime object:

integer microsecond(datetime datetime)
    Return the microsecond part of the time value.

Since this a fraction of a second want it to log:

 second($EventTime) + "." + microsecond(($EventTime)

However this is wrong, when the microseconds fraction is lower then 100000. I need to add leading zeroes.

How do i do this?

Note: I cannot use sprintf("%06d", val) in perl module, since i am on windows.