How NYC Cyber Command built real-time log visibility for the city that never sleeps
With NXLog Platform, NYC Cyber Command brought DNS, Windows, DHCP, AD FS and Centrify AuditTrail logs into a single normalized stream, closed the gaps its SOC had been working around, and gained headroom as log volume grew.
Business Demand
Meet tightening regulatory requirements and improve the city's security posture.
Key Challenges
Collect logs from a mix of DNS, Windows, DHCP, AD FS and Centrify AuditTrail systems, get them to the SOC in real time, and keep up as volume grew.
Results
Wider network visibility, faster incident response, and a log pipeline that scales with the city's infrastructure.
The Challenge
NYC Cyber Command protects the systems that keep New York City running, and those systems do not look alike. DNS servers, Windows servers, DHCP servers, AD FS servers and Centrify AuditTrail all produce logs in different formats and expose them through different mechanisms. Pulling that into one place meant handling each source on its own terms, and the collection tooling NYC3 had in place could not do it.
The gaps showed up where they mattered most. Some logs — DNS among them — could not be extracted and forwarded in a standardized format at all. That left the Security Operations Center working with an incomplete picture, and it held up parser development, because engineers cannot write detection logic against data they cannot get. Both effects landed on the same place: how quickly the team could respond to an incident.
Volume made it harder. As NYC3 extended its coverage across the city's systems, log throughput grew past what the existing collector was built for. The team needed something that would keep pace without adding management overhead every time a new source came online.
The Solution
NYC Cyber Command evaluated the available options and chose NXLog Platform, on the strength of how much of its estate a single agent could cover and how cleanly the data came out the other side.
Two capabilities decided it. First, breadth: NXLog Agent collects from Windows, Red Hat Enterprise Linux, Ubuntu and macOS as well as network devices, and its modular design meant DNS servers, DHCP servers, AD FS and Centrify AuditTrail could each be integrated without a separate tool. Second, syslog: native support let NYC3 normalize and enrich records before forwarding them to its syslog aggregators and SIEM, so downstream tools received data they could parse on arrival.
Rollout was staged to keep NYC3's operations undisturbed. Guided setup in NXLog Platform let the team deploy and configure agents across DNS, Windows, DHCP and Centrify AuditTrail servers in bulk, with little manual work per host. Agents started capturing DNS queries, DHCP lease records and system security events straight away.
With data flowing, the team turned to the shape of it. Records were enriched with hostnames and timestamps and given consistent formatting, which is what made analytics in the SIEM and in NYC3's other tools reliable rather than approximate. Performance tuning followed, to confirm every required log was captured, processed and relayed with minimal latency — the standard real-time operations demand.
Why it Worked
Windows, RHEL, Ubuntu, macOS and network devices, plus DNS, DHCP, AD FS and Centrify AuditTrail
Native ETW collection, DNS debug log parsing and Windows Event Log capture, normalized on the way out
Enrichment and formatting happen before the aggregators and SIEM, so parsers do not have to catch up
Agents deployed and configured across server groups without per-host hand-holding
Results
Centralizing logs from DNS, DHCP and security systems gave NYC Cyber Command a view of network activity it did not have before. Anomalies and suspicious behavior surfaced faster, because the data needed to spot them was in one place and in one format.
Relaying those logs to security tools in real time changed what the SOC could do with them. Analysts worked from current data rather than catching up on it, which shortened the distance between something happening and someone acting on it.
Normalization paid off twice. The same enriched, consistently formatted records that improved detection accuracy also gave NYC3 cleaner material for compliance reporting, so the work done once served both purposes.
The architecture left room to grow. As the city's infrastructure changed, NYC3 could add data sources without rebuilding the pipeline around them — which is what made the deployment durable rather than a point fix.
Wider network visibility
DNS, DHCP and security events centralized in one consistent format
Faster incident response
Real-time relay to security tools, so the SOC acts on current data
Room to scale
New data sources added as the city's infrastructure changed
Cleaner compliance reporting
Normalized, enriched records the team could report from directly
About NYC Cyber Command
New York City Cyber Command was established by Executive Order 28 on 11 July 2017 to direct citywide cyber defense and incident response and to set information security policy across City agencies. The City Council added it to the New York City Charter by unanimous vote in September 2020. In January 2022, Executive Order 3 consolidated it, along with the city's other technology offices, into the NYC Office of Technology and Innovation, where its cyber defense mandate continues. Over its first years it raised cybersecurity maturity across more than 100 city agencies, working with public and private partners to protect the systems and data New Yorkers rely on.
Follow us
© Copyright NXLog Ltd.