Firewall Event Aggregation

Post a different question
3
responses

I've got a simple config listening on 514 UDP/TCP and forwarding everything received out to another server for ingest. One of the things I've been having trouble figuring out is how to do simple event aggregation for firewall logs. Ideally it would aggregate over a time window and append the message with a new field containing the count of messages.

I know something like this used to be done via module "pm_norepeat", but I think this is being deprecated, and I'm not aware that it is capable of appending message count to the original message. It seems this should somehow be done using variables going forward.

To add to the complexity, we have two separate firewall types within our environment, (Cisco ASA's and Palo's). Greatly appreciate if anyone can point me in the right direction.

AskedMarch 6, 2021 - 12:09am

Answer (1)

Hi,

Correct me if I'm wrong: so you'd like to send you messages in batches, right?

Best regards,
Rafal

AnsweredMarch 9, 2021 - 9:52am

Comments (2)

  • Tenways's picture

    Ideally the firewall events that match would aggregate over a time window.

    1. first firewall log received
    2. Start timer and eggregated event counter.
    3. Next firewall log received, if port, protocol, destination ip and source ip match, increase the count by 1. Drop the newly received event.
    4. Contine step 3 until timer is reached (say 60 seconds).
    5. When timer is reached, add 3 fields to the original event.
      • time of first log received.
      • count of total aggregated events.
      • time of last matching event received

    This will then ship the new event to the destination defined in the route with the 3 new fields appended. This greatly reduces storage volume when a single event can represent potentially thousands of events.

    Thanks for the reply, apologies for the poor formating as im writing this reply from my mobile.

    March 10, 2021 - 9:17am