responses
I've got a simple config listening on 514 UDP/TCP and forwarding everything received out to another server for ingest. One of the things I've been having trouble figuring out is how to do simple event aggregation for firewall logs. Ideally it would aggregate over a time window and append the message with a new field containing the count of messages.
I know something like this used to be done via module "pm_norepeat", but I think this is being deprecated, and I'm not aware that it is capable of appending message count to the original message. It seems this should somehow be done using variables going forward.
To add to the complexity, we have two separate firewall types within our environment, (Cisco ASA's and Palo's). Greatly appreciate if anyone can point me in the right direction.
Comments (2)
Ideally the firewall events that match would aggregate over a time window.
This will then ship the new event to the destination defined in the route with the 3 new fields appended. This greatly reduces storage volume when a single event can represent potentially thousands of events.
Thanks for the reply, apologies for the poor formating as im writing this reply from my mobile.
The pm_norepeat feature is being removed in favor of using Variables and Statistical Counters as they can be customized to suit your needs.
The following example section and the one following it may help https://nxlog.co/documentation/nxlog-user-guide/nxlog-language.html#guide_lang_variables
Regards, Nathan