How a federal agency’s monitoring gaps turned a containable incident into a three-week nightmare
In September 2025, CISA responded to a federal agency breach that security teams could have stopped in hours. Instead, threat actors roamed the network undetected for three weeks.
The damage? Multiple compromised servers, web shells planted across the infrastructure, and a persistent foothold that took significant resources to remediate.
The root cause wasn’t a zero-day exploit or sophisticated malware. According to CISA’s official advisory, the agency had endpoint detection and response (EDR) tools that generated alerts. The problem was simple: those alerts weren’t continuously reviewed.
What happened during those three weeks
Attackers exploited CVE-2024-36401 in GeoServer to gain initial access. With no one actively monitoring the alerts, they had free rein to:
-
Compromise a second GeoServer instance
-
Move laterally to a web server
-
Breach an SQL server
-
Upload web shells for persistent access
Each step likely generated telemetry data. Each compromise probably triggered alerts. But without a system to route critical alerts to the right people at the right time, that data sat unused.
The visibility gap that extended the breach
CISA identified three key failures in their lessons learned:
-
EDR alerts existed but weren’t reviewed continuously
-
Some public-facing systems had no endpoint protection at all
-
Malicious activity went undetected for three weeks
This incident highlights a common challenge: organizations collect enormous amounts of security telemetry, but lack the infrastructure to make that data actionable. Alerts pile up. Critical signals get buried in noise. Security teams can’t distinguish between routine events and active intrusions.
What effective telemetry management looks like
The federal agency’s experience shows why telemetry management matters. With the right pipeline in place, this incident could have unfolded very differently:
- Alert deduplication and prioritization
-
When the first GeoServer exploitation occurred, related alerts could have been grouped and prioritized based on asset criticality. Instead of dozens of individual alerts, the security team would see one high-priority incident: "Public-facing GeoServer showing signs of exploitation."
- Context enrichment
-
Raw alerts about unusual process execution are hard to act on quickly. But when telemetry data includes context—which asset is affected, whether it’s public-facing, what data it accesses, recent vulnerability disclosures—security teams can assess severity in seconds, not hours.
- Intelligent routing
-
Not every alert needs to wake up the entire security team. A well-designed telemetry pipeline routes low-severity events to aggregated dashboards while sending critical alerts directly to on-call personnel through their preferred channels.
- Metrics visibility
-
Traditional event monitoring catches discrete incidents, but metrics data reveals patterns. Unusual authentication times, gradual increases in outbound traffic, or subtle changes in database query patterns can signal compromise before major damage occurs.
The cost of telemetry chaos
Three weeks of undetected access isn’t just a security failure—it’s an operational and financial burden:
-
Extended incident response costs
-
Forensic analysis across multiple compromised systems
-
Potential data exfiltration that may never be fully quantified
-
Regulatory reporting requirements
-
Loss of stakeholder trust
Compare that to the cost of catching the breach on day one. The difference isn’t just technical—it’s measured in weeks of attacker dwell time, number of compromised systems, and ultimately, organizational impact.
Building a better monitoring foundation
The federal agency breach offers clear lessons for any organization managing security telemetry:
- Collect telemetry from all public-facing assets
-
Blind spots are opportunities for attackers. If a system is internet-accessible, it needs monitoring. In the current IT world, that likely means everything from the core network router to the coffee machine in the corner.
- Create a pipeline that processes alerts, not just collects them
-
Raw telemetry data has limited value. The pipeline should enrich, deduplicate, and route alerts based on context and priority.
- Ensure continuous review
-
Alerts only matter if someone sees them. Whether through automation, staffing, or managed services, establish processes that guarantee timely review.
- Use metrics alongside events
-
Events tell you what happened. Metrics tell you what’s changing. Both matter for complete visibility.
Moving forward
The CISA advisory doesn’t name the affected agency, but the lessons apply broadly. Organizations across industries face the same challenge: too much telemetry data, not enough actionable intelligence. Even for seasoned professionals with decades of experience, the volume outpaced response capacity years ago.
A telemetry management pipeline doesn’t just reduce alert fatigue—it fundamentally changes how quickly you can detect and respond to threats. The difference between hours and weeks of attacker access often comes down to whether your monitoring infrastructure can turn raw data into clear, routable, contextual alerts.
If you’re struggling with alert overload or want to improve your security visibility, our team can walk you through how effective telemetry management reduces both noise and risk.
