Hi, I have a log source (Cisco ironport) that sends some multi-line logs via syslog and I want to join only one log when the ICID is the same in the first case and join only one log when the MID is the same in the second case . But I want to remove part of the log. Example below. Is it possible to do this? join when ICID is equal <22>Oct 07 22:16:47 TextMailLogs: Info: New SMTP ICID 929637136 interface Public_SMTP (yy.yy.yy.yy) address xx.xx.xx.xx reverse dns host host@isp.com verified yes <22>Oct 07 22:16:47 TextMailLogs: Info: ICID 929637136 RELAY SG policy match .isp.com SBRS not enabled <22>Oct 07 22:16:47 TextMailLogs: Info: SMTP Auth: (ICID 929637136) succeeded for user: user@isp.com using AUTH mechanism: PLAIN with profile: SMTP_AUTH_CP <22>Oct 07 22:16:48 TextMailLogs: Info: ICID 929637136 lost <22>Oct 07 22:16:48 TextMailLogs: Info: ICID 929637136 close example result: <22>Oct 07 22:16:47 TextMailLogs: Info: New SMTP ICID 929637136 interface Public_SMTP (yy.yy.yy.yy) address xx.xx.xx.xx reverse dns host host@isp.com verified yes RELAY SG policy match .isp.com SBRS not enabled succeeded for user: user@isp.com using AUTH mechanism: PLAIN with profile: SMTP_AUTH_CP lost close join when MID is equal <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 From: <from@isp.com> <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 RID 0 To: <to@isp.com> <22>Apr 12 12:00:36 TextMailLogs: Info: MID 354020393 Subject '=?UTF-8?B?V29tZW7' <22>Apr 12 12:00:36 TextMailLogs: Info: MID 354020393 ready 92209 bytes from <from@isp.com> <22>Apr 12 12:00:53 TextMailLogs: Info: Message finished MID 354020393 done example result: <22>Apr 12 12:00:35 TextMailLogs: Info: MID 354020393 ICID 751911320 From: <from@isp.com> RID 0 To: <to@isp.com> Subject '=?UTF-8?B?V29tZW7' ready 92209 bytes from <from@isp.com> done Thanks

Hey guys, We tested nxlog on a few servers and everything worked fine, no problem at all. Now when we deployed it to more, after 2 hours, the clients just keep disconnecting and reconnecting. I'm really not sure what's happening. We're using a self signed cert, made with opeenssl. Not sure what would you need to identify my problem. If needed I'll copy over the debug log or conf file. Environment is windows. Any help is appreciated!

hi all Fairly simple one i think we send way too many "informational" Messages from a few of our Windows servers to Loggly i want to simply stop sending those "informational" messages any help would be appreciated Dale

Q1. Hello I have one problem about mysql to graylog server i want my db (version:mariadb 5.5 OS centos 7 ) "SELECT eventlog" to my graylog server but my nxlog log have error the message is: ERROR im_dbi failed to execute SQL statement. 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE id > -1 LIMIT 10' at line 1 my nxlog.conf <Extension _syslog> Module xm_syslog </Extension> <Input in> Module im_dbi Driver mysql Option host 127.0.0.1 Option username root Option password librenms Option dbname librenms SQL SELECT eventlog </Input> <Output out> Module om_udp Host 192.168.1.20 Port 10514 Exec to_syslog_snare(); </Output> <Route 1> Path in => out </Route> so how can i do or where are reference about nxlog for mysql(mariadb) ? Q2. can i install nxlog-ce to CentOS8?

How would I go about statically assigning the value of x-forward-for the end of an IIS message that is being read through IM_FIle? I am doing testing of IIS Log files and I want to take a value from an internal system and add the x-forward-for field to the message. I have tried adding it to the Input on im_file Exec $Message = $x_forwarded_for + ' ' + 'x.x.x.x'; I have tried it as $raw_event = $raw_event + 'x.x.x.x' Either the value never shows up or it ends up outside of the message. What I need is for it to be inside the message. Essentially where the time-taken is on this example message I would end up with my static x-forwarded-for entry. #Software: Microsoft Internet Information Services 10.0 #Version: 1.0 #Date: 2017-10-02 17:11:27 #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken 2017-10-02 17:11:27 fe80::b5d8:132c:cec9:daef%6 RPC_IN_DATA /rpc/rpcproxy.dll 1d4026cb-6730-43bf-91eb-df80f41c050f@test.com:6001&CorrelationID=<empty>;&RequestId=11d6a78a-7c34-4f43-9400-ad23b114aa62&cafeReqId=11d6a78a-7c34-4f43-9400-ad23b114aa62; 80 TEST\HealthMailbox418406e fe80::b5d8:132c:cec9:daef%6 MSRPC - 500 0 0 7990 2017-10-02 17:12:57 fe80::a425:345a:7143:3b15%2 POST /powershell clientApplication=ActiveMonitor;PSVersion=5.1.14393.1715 80 - fe80::a425:345a:7143:3b15%2 Microsoft+WinRM+Client - 500 0 0 11279

Using one dedicated NXlog node as master collector that forwards logs to SIEM, I also dump all logs into a file and I rotate that file weekly, keeping 4 weeks worth of logs on the collector; this very clean setup is taken from the documentation and I like it - it works well (see Log rotate in documentation, example 103). Problem is: This consumes much disk, so I'll need to compress the rotated file. Can I add compression to that output to compress the rotate-copy? I'd prefer not using the example 113 in said log-rotation documentation page.

I'm fairly new to this, but wanted to get some pointers. I've been provided with a syslog.conf file, and am trying to determine what events are being logged. Can someone please let me know what I should be looking for? Alternatively if the syslog.conf file doesn't contain this info, where can I find what events are being logged? On the other hand, is there a standard list of event types that are automatically logged?

Hi, I am new to NXLog and I am setting up relay log servers in IP ranges to collect and forward log events from hosts (different OS) with dynamically assigend IP addresses. While doing so, I figured out that the forwarded log events missing important details about the orginating systems and I like to enrich the log events while receiving (accepting ) them with the following details: hostname as SOURCE_HOSTNAME ipv4 address as SOURCE_HOSTIP How is the best practice to do so? Rgds, Thorsten

hi who i can get iis web server logs and then send them to ossim log server?

Hello Guys, Im currently trialing nxlog enterprise version and specifically interested in agentless version of it. While working on it i have come across a blockade which seems to not forward logs from windows server machine to linux windows event collector(nxlog enterprise version is running). Im running nxlog version 5.1.6133 on ubuntu. Here are the details (SERVER details w.r.t im_wseventing module documentation) NAME="Ubuntu" VERSION="18.04.5 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.5 LTS" VERSION_ID="18.04" Below are client details which forwards logs to server above using agentless method OS Name: Microsoft Windows Server 2019 Datacenter OS Version: 10.0.17763 N/A Build 17763 OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Server Im getting following error message while trying to forward logs. Seeing this error in eventviewer under eventlog forwarding plugin. I have followed the documentation and generated certs and certs seems to be valid and working. Log Name: Microsoft-Windows-Forwarding/Operational Source: Microsoft-Windows-Forwarding Date: 10/28/2020 2:37:05 AM Event ID: 105 Task Category: None Level: Error Keywords: User: NETWORK SERVICE Computer: computer_name Description: The forwarder is having a problem communicating with subscription manager at address HTTPS://private_ip_address:5986/wsman/. Error code is 2150858819 and Error Message is 鿰柣ƴ. Event Xml: <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"> <System> <Provider Name="Microsoft-Windows-Forwarding" Guid="{699e309c-e782-4400-98c8-e21d162d7b7b}" /> <EventID>105</EventID> <Version>0</Version> <Level>2</Level> <Task>0</Task> <Opcode>0</Opcode> <Keywords>0x8000000000000000</Keywords> <TimeCreated SystemTime="2020-10-28T02:37:05.265085100Z" /> <EventRecordID>438</EventRecordID> <Correlation ActivityID="{2977fa9f-ac7b-0000-d9fb-77297bacd601}" /> <Execution ProcessID="2732" ThreadID="5668" /> <Channel>Microsoft-Windows-Forwarding/Operational</Channel> <Computer>computer_name</Computer> <Security UserID="S-1-5-20" /> </System> <EventData Name="SubscriptionManagerStatus"> <Data Name="SubscriptionManagerAddress">HTTPS://private_ip_address:5986/wsman/</Data> <Data Name="ErrorCode">2150858819</Data> <Data Name="ErrorMessage">鿰柣ƴ</Data> </EventData> </Event> I have tried even port 5985 and it also faces similar error. I have checked whether the server which is the ubuntu is reachable or not, did GET request using postman api tool.POST request seems to not work which might be the cause which might be behind this problem. Below is my nxlog config <Input wseventing> Module im_wseventing ListenAddr 0.0.0.0 Port 5986 Address https://private_ip_address:5986/wsman HTTPSCertFile %CERTDIR%/server-cert.pem HTTPSCertKeyFile %CERTDIR%/server-key.pem HTTPSCAFile %CERTDIR%/ca-cert.pem <QueryXML> <QueryList> <Computer>*</Computer> <Query Id="0"> <Select Path="Application">*</Select> <Select Path="Security">*</Select> <Select Path="Setup">*</Select> <Select Path="System">*</Select> <Select Path="ForwardedEvents">*</Select> </Query> </QueryList> </QueryXML> </Input> <Output file> Module om_file File "/opt/nxlog/var/log/nxlog/windows_events.log" </Output> <Route route_wsevents> Path wseventing => file </Route> Let me know how do i overcome this issue if possible. Thanks.

Hello windows ---> nxlog-------->graylog&splunk(syslog) how can i do? this conf is right? p.s. graylog 192.168.1.20 splunk 192.168.1.21 ------------------------------------My conf------------------------------------------------------------------ Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_gelf </Extension> <Extension _syslog> Module xm_syslog </Extension> <Input in> Module im_msvistalog </Input> <Output out> Module om_udp Host 192.168.1.20 Port 10554 OutputType GELF </Output> <Output out1> Module om_udp Host 192.168.1.21 Port 10554 Exec to_syslog_snare(); </Output> <Route 1> Path in => out </Route> <Route 2> Path in => out1 </Route>

Hi team, I'm sure this should be easy, but I'm not having much luck finding the answer elsewhere, can any of you help me? So I have (McAfee Firewall) log entries that look like this: Time: 10/23/2020 08:09:36 AM Event: Traffic IP Address: 172.19.0.113 Description: SNMP SERVICE Path: C:\Windows\System32\snmp.exe Message: Allowed Incoming UDP - Source 172.19.0.113 : (52676) Destination 172.23.25.135 : snmp (161) Matched Rule: Adaptive Rule - snmp.exe I've got this being parsed as a multiline log entry with the following: <Extension 5f917c0781064d07c2e8486a-multiline> Module xm_multiline HeaderLine /^Time:.*/ EndLine /^Matched Rule:.*/ <Input 5f917c0781064d07c2e8486a> Module im_file File 'C:\ProgramData\McAfee\Endpoint Security\Logs\FirewallEventMonitor.log' PollInterval 1 SavePos True ReadFromLast True Recursive True RenameCheck False Exec $FileName = file_name(); # Send file name with each message InputType 5f917c0781064d07c2e8486a-multiline So far so good - I get a multiline message come through, but I'd now like to parse it. So my first step is to split out the individual lines - ideally in the above eample I'd split the above message into 7 fields: 'Time' -> '10/23/2020 08:09:36 AM' 'Event' -> 'Traffic' 'IP Address' -> 172.19.0.113 ...etc So I'm guessing there's an Exec section required and some regex work, but I've not managed to get anything to work yet - has anyone else done anything similar here? thanks in advance, Jim

Hope someone can provide some pointers. I have an application that uses mySQL database and it is installed on Windows Server 2019. I just downloaded NXLog CE on the same Windows Server 2019. I am new to this but the goal is to extract a table from mySQL and output it as JSON format so that I can ingest it to another platform. Appreciate if someone can point me to the right direction to make this work. Thanks !

I am using https://nxlog.co/system/files/products/files/348/nxlog-ce-2.10.2150.msi to download the nxlog file. I am using below configuration: This is a sample configuration file. See the nxlog reference manual about the configuration options. It should be installed locally and is also available online at http://nxlog.org/docs/ Please set the ROOT to the folder your nxlog was installed into, otherwise it will not start. #define ROOT C:\Program Files\nxlog define ROOT C:\Program Files (x86)\nxlog define ROOT C:\Program Files (x86)\nxlog Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data LogFile %ROOT%\data\nxlog.log <Extension _json>    Module xm_json </Extension> <Input in> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Security">* </Select> <Select Path="Application">* </Select> <Select Path="Setup">* </Select> <Select Path="System">* </Select> </Query> </QueryList> <Exec> if ($EventID NOT IN (%aisiem%)) drop(); </Exec> </Input> <Output out>    Module om_udp    Host CCE_IP_ADDRESS  Port 5154    Exec to_json(); </Output> <Route 1>    Path in => out </Route> Getting this error: 2020-10-22 19:11:05 WARNING stopping nxlog service 2020-10-22 19:11:05 WARNING nxlog-ce received a termination request signal, exiting... 2020-10-22 19:11:16 ERROR Failed to load module from C:\Program Files (x86)\nxlog\modules\output\om_udp   .dll, Invalid argument; Invalid argument 2020-10-22 19:11:16 ERROR module 'out' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:52 2020-10-22 19:11:16 ERROR route 1>  is not functional without output modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:52 2020-10-22 19:11:16 WARNING no routes defined! 2020-10-22 19:11:16 WARNING not starting unused module in 2020-10-22 19:11:16 INFO nxlog-ce-2.10.2150 started I am using windows 10. Please suggest what should I change to configure successfully.

Hello, I'm trying to query some EventID with a specific SeverityValue in "im_msvistalog", the config is something like this: <Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='System'>*[System[(EventID=6005 or EventID=6008 or EventID=7036)]</Select> </Query> </QueryList> </QueryXML> Exec if $SeverityValue NOT IN (2, 4) drop(); Exec $Message =~ s/(\t|\R)/ /g; I'm trying for a test to output on file, but nothing is outputted. Anyone has some hint? Thanks

Hi everyone, I'm getting this issue of assignment failed at line 215, character 67 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted function 'strftime' failed at line 215, character 66 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted got 'unknown' for first argument of function 'strftime(datetime, string)'. if-else failed at line 208, character 277 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted procedure 'parse_csv' failed at line 208, character 129 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted Not enough fields in CSV input, expected 17, got 0 in input '' Below is the line the error is refering to: Exec $EventTime = strftime($EventTime, '%Y-%m-%d %H:%M:%S, %z'); } Kindly help me out to fix this issue and what is causing the error. Thanks.

Hi guys, I must forwarding some logs, but only of some users. this's my current configuration. Panic Soft #NoFreeOnExit TRUE define ROOT C:\Program Files (x86)\nxlog define CERTDIR %ROOT%\cert define CONFDIR %ROOT%\conf define LOGDIR %ROOT%\data define LOGFILE %LOGDIR%\nxlog.log LogFile %LOGFILE% Moduledir %ROOT%\modules CacheDir %ROOT%\data Pidfile %ROOT%\data\nxlog.pid SpoolDir %ROOT%\data <Extension _syslog> Module xm_syslog </Extension> <Extension _charconv> Module xm_charconv AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32 </Extension> <Extension _exec> Module xm_exec </Extension> <Extension _fileop> Module xm_fileop # Check the size of our log file hourly, rotate if larger than 5MB &lt;Schedule&gt; Every 1 hour Exec if (file_exists('%LOGFILE%') and \ (file_size('%LOGFILE%') &gt;= 5M)) \ file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; # Rotate our log file every week on Sunday at midnight &lt;Schedule&gt; When @weekly Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8); &lt;/Schedule&gt; </Extension> <Input eventlog> Module im_msvistalog <QueryXML> <QueryList> <Query Id='0'> <Select Path='Security'>*[System[(EventID=4720 or EventID=4634 or EventID=4625 or EventID=4624 or EventID=4726)]]</Select> </Query> </QueryList> </QueryXML> Exec $Message = 'EventID: ' + $EventID +' Source: '+ $IpAddress + ' Username: ' + $TargetUserName ; </Input> <Output tcp> Module om_udp Host xx.xxx.xx.xxx Port xxx Exec to_syslog_bsd(); </Output> <Route eventlog_to_tcp> Path eventlog => tcp </Route> can you help me? thank you

Hello, After looking on the community forum i didn't really get the answer i was seeking. I'm sending windows log to a syslog and to a splunk, I got some issue about parsing the Message part as you can see : https://cdn.discordapp.com/attachments/700242491227635714/766300884971159562/unknown.png This is my conf : https://cdn.discordapp.com/attachments/700242491227635714/766301478897451048/unknown.png https://cdn.discordapp.com/attachments/700242491227635714/766301597541335060/unknown.png Maybe this is not the good way to do it, i have to send in syslog because i'm sending it after to an elasticsearch and the splunk. Thank you for your help :)

Hello In the end I have to come to you because I can't find the way to do it and I'm going a bit crazy. I have a record entry from a file. This file sends the data in a disorderly way and I have managed to assign it to variables. Now the only thing I want is to create an output in a file but with the order I need. EJ of log entry : user1 2020 barcelona 12 13:39 12 spaghetti And I want an exit like Place: <variable of place> , Date: <variable of date> , Age :<variable of age> To generate a line like this : Place : barcelona , Date : 2020 , Age : 12 .... This is my config. <Input LOG_IN> Module im_file File "C:\logs\u_ex*" SavePos TRUE <Exec> if $raw_event =~ /(.?)\s"(.?)"\s"(.?)"\s(.?)\s(.?)\s(.?)\s"(.?)"\s"(.?)"\s(.?)\s(.?)\s(.?)\s"(.?)"\s"(.?)"\s(.?)\s(.?)\s"(.?)"/; { $a = $1; $b = $2; $c = $3; $d = $4; $e = $5; $f = $6; $g = $7; $h = $8; $i = $9; $j = $10; $k = $11; $l = $12; $m = $13; $n = $14; $o = $15; $p = $16; } </Exec> </Input> <Output file> Module om_file File "C:\logs\output.log" </Output>

Hi. I'm new to both Graylog and NxLog. I am getting the following errors when trying to add inputs. 2020-10-14 11:48:27 WARNING stopping nxlog service 2020-10-14 11:48:27 WARNING nxlog-ce received a termination request signal, exiting... 2020-10-14 11:48:28 ERROR invalid keyword: Output at C:\Program Files (x86)\nxlog\conf\nxlog.conf:66 2020-10-14 11:48:28 ERROR module 'graylog' has configuration errors, not adding to route 'graylog_route' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:70 2020-10-14 11:48:28 ERROR route graylog_route is not functional without output modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:70 2020-10-14 11:48:28 WARNING no routes defined! 2020-10-14 11:48:28 WARNING not starting unused module win 2020-10-14 11:48:28 WARNING not starting unused module graylog 2020-10-14 11:48:28 INFO nxlog-ce-2.10.2150 started I added the following lines to my config file <Extension _gelf> Module xm_gelf </Extension> <Input win> Module im_msvistalog Query <QueryList> <Query Id="0"> <Select Path="Application"></Select> <Select Path="System"></Select> <Select Path="Security">*</Select> </Query> </QueryList> </Input> <Output graylog> Module om_udp host 172.16.101.33 Port 12201 Output Type GELF </Output> <Route graylog_route> Path win => graylog </Route> Any help is appreciated.