The analysis of events produced by various systems and applications can offer insights into the infrastructure health and the operational resilience of an enterprise. From an Infosec perspective, the end-goals are: threat detection, forensics and remediation.
However, we can’t query or analyse data that we haven’t collected in the first place! Before threat hunting and incident response are even possible, security events need to be collected from various sources, parsed, transformed, and then forwarded to data sinks such as security information and event managements (SIEM), security analytics platforms, cloud ecosystems and long term storage.
Incidentally, managing the lifecycle of event records is exactly one of the areas where NXLog excels. NXLog is primarily designed to ensure that all log collection workflows are efficient and reliable end-to-end, no matter what products are employed.
Sharing is caring
As soon as a Security Operations Center (SoC) engineer establishes a way to identify an indicator of compromise (IOC) or any suspicious patterned behaviour in log events, that intelligence needs to be shared with the rest of the InfoSec community. This is done through signatures published in rule repositories or threat sharing platforms such as MISP. Luckily, there is an open standard that can be used by all stakeholders: Sigma.
Sigma signatures are human-readable rules written in YAML format. According to the specification, they include a set of criteria to describe anomalies in log events that should raise concern. Their logic is structured, reproducable and generic enough to be useful regardless of the security products used. In essence, this is similar to how YARA signatures can pinpoint malicious files and how Snort rules can help to discover suspicious activity in raw network traffic.
Like cogs in a wheel
To recap, NXLog is a vendor-neutral tool for collecting and forwarding logs, and Sigma is a vendor-agnostic framework to define, share and reproduce detection logic for those very logs. They complement each other, what a perfect match! The missing piece in this puzzle is the mechanism enabling us to convert generic Sigma rules into search queries for specific platforms.
To achieve that conversion, Sigma offers pySigma, a python library that parses and transforms Sigma rules into queries. Its architecture is decentralised, meaning that it is generic at its core, but product-specific plugins are available to feed and adjust the rules to the target query language syntax, field naming scheme and value representation. It is important to highlight that after the conversion is done, the original detection logic, often linked to Common Vulnerabilities and Exposures (CVE) identifiers, is preserved.
Let’s explore one of the ways NXLog and Sigma can work together.
Sigma rules applied at the SIEM level
After NXLog is configured to collect, transform and forward logs to one or more SIEMs, Sigma rules can be converted to queries tailored to a certain SIEM, by taking advantage of one of the pySigma backends.
Example scenario
A common practice in malware (e.g. Ducktail) is to collect the victim’s public-facing IP address by invoking publicly available API’s (e.g. api.ident.me). This could be part of data harvesting, targeting specific geographical regions (aka. geofencing), or feeding statistics to malicious command and control centres. Those IP lookups could imply a system breach. In order to flag that suspicious behaviour, we’ll use a Sigma rule, part of the public rules repository, that detects queries towards IP lookup services, making sure that any legitimate user activity through browser processes is excluded.
In the following setup, we’ll use the NXLog Enterprise Edition agent to collect the DNS client logs of a Windows host running a Sysmon service, then parse the raw events into fields, transform them into JSON and forward the structured data to a Splunk Indexer via its HTTP Event Collector (HEC) data input. Sysmon events link DNS queries to the process that triggered them, which is why it was preferred in this use case.
-
Step 1: Install the pySigma backend for Splunk
$ pip install pysigma $ pip install sigma-cli $ sigma plugin list $ sigma plugin install sysmon $ sigma plugin install splunk $ sigma list formats splunk $ sigma list pipelines splunk -
Step 2: Download the relevant Sigma threat detection rule and convert it to a Splunk compatible query (SPL) using sigma-cli.
$ wget https://raw.githubusercontent.com/SigmaHQ/sigma/master/rules/windows/dns_query/dns_query_win_susp_ipify.yml $ sigma convert -t splunk -p sysmon -s dns_query_win_susp_ipify.yml Parsing Sigma rules [####################################] 100% EventID=22 QueryName IN ("*api.2ip.ua*", "*api.ipify.org*", "*bot.whatismyipaddress.com*", "*canireachthe.net*", "*checkip.amazonaws.com*", "*checkip.dyndns.org*", "*curlmyip.com*", "*edns.ip-api.com*", "*eth0.me*", "*freegeoip.app*", "*geoipy.com*", "*getip.pro*", "*icanhazip.com*", "*ident.me*", "*ifconfig.io*", "*ifconfig.me*", "*ip-api.com*", "*ip.anysrc.net*", "*ip.tyk.nu*", "*ipaddressworld.com*", "*ipecho.net*", "*ipinfo.io*", "*ipof.in*", "*ipv4.icanhazip.com*", "*ipv4bot.whatismyipaddress.com*", "*ipwho.is*", "*l2.io*", "*myexternalip.com*", "*wgetip.com*", "*whatismyip.akamai.com*", "*wtfismyip.com*") NOT (Image="*\\brave.exe" OR Image IN ("C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe", "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe") OR Image IN ("C:\\Program Files\\Mozilla Firefox\\firefox.exe", "C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe") OR Image IN ("C:\\Program Files (x86)\\Internet Explorer\\iexplore.exe", "C:\\Program Files\\Internet Explorer\\iexplore.exe") OR Image="*\\maxthon.exe" OR Image="C:\\Program Files (x86)\\Microsoft\\EdgeWebView\\Application\\*" OR Image="*\\WindowsApps\\MicrosoftEdge.exe" OR Image IN ("C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe", "C:\\Program Files\\Microsoft\\Edge\\Application\\msedge.exe") OR (Image IN ("C:\\Program Files (x86)\\Microsoft\\EdgeCore\\*", "C:\\Program Files\\Microsoft\\EdgeCore\\*") Image IN ("*\\msedge.exe", "*\\msedgewebview2.exe")) OR Image="*\\opera.exe" OR Image="*\\safari.exe" OR Image="*\\seamonkey.exe" OR Image="*\\vivaldi.exe" OR Image="*\\whale.exe") -
Step 3: Download & Install Sysmon and enable logging of client DNS queries:
> Sysmon.exe -i config-dnsquery.xmlconfig-dnsquery.xml:<Sysmon schemaversion="4.22"> <EventFiltering> <DnsQuery onmatch="exclude"/> </EventFiltering> </Sysmon>More information can be found in the integration guide for DNS query logging.
-
Step 4: Configure Splunk HEC and the output configuration of NXLog
-
Step 5: Add the following NXLog configuration to the Windows Host
nxlog.conf<Extension _json> Module xm_json </Extension> <Extension clean_splunk_fields> Module xm_rewrite Keep time, host, source, sourcetype, event Rename Source, source </Extension> <Input sysmon> Module im_msvistalog <QueryXML> <QueryList> <Query Id="0"> <Select Path="Microsoft-Windows-Sysmon/Operational"> *[System[(EventID='22')]] </Select> </Query> </QueryList> </QueryXML> </Input> <Output splunk> Module om_http URL https://x.y.z.q:8088/services/collector/event AddHeader Authorization: Splunk <token> HTTPSCAFile %CERTDIR%/cacert.pem BatchMode multiline Compression gzip <Exec> # Rename event fields to Splunk taxonomy if $Severity rename_field($Severity, $vendor_severity); if $SeverityValue rename_field($SeverityValue, $severity_id); # Convert all fields to JSON and write to $event field $event = to_json(); # Convert $EventTime to decimal seconds since epoch UTC $time = string(integer($EventTime)); $time =~ /^(?<sec>\d+)(?<ms>\d{6})$/; $time = $sec + "." + $ms; # Set the log source type $sourcetype = "_json"; # Add HEC metadata fields if available if $Hostname $host = $Hostname; if $SourceName $source = $SourceName; # Remove unnecessary fields clean_splunk_fields->process(); # Convert to JSON to_json(); </Exec> </Output> <Route default> Path sysmon => splunk </Route> -
Step 6: Trigger DNS requests
-
Open a browser and make a "legitimate" connection to: https://ifconfig.me
The raw event generated by Sysmon looks as follows:
-
Then run the following Powershell script to get the IP address from ident.me programmatically:
$IP = try { (Invoke-WebRequest -Uri 4.ident.me).Content } catch { (Invoke-WebRequest -Uri 4.tnedi.me).Content } Write-Output "Found public IPv4 $IP"Which produces the following Sysmon event:
-
-
Step 7: Run the query on Splunk and observe the results
As expected, the query to ident.me is flagged as suspicious but the query to ifconfig.me is not:
-
Step 8: Convert the same detection rule for two additional backends:
-
ElasticSearch:
$ sigma plugin list $ sigma plugin install elasticsearch $ sigma convert -t elasticsearch -p sysmon -s dns_query_win_susp_ipify.yml Parsing Sigma rules [####################################] 100% EventID:22 AND ((QueryName:(*api.2ip.ua* OR *api.ipify.org* OR *bot.whatismyipaddress.com* OR *canireachthe.net* OR *checkip.amazonaws.com* OR *checkip.dyndns.org* OR *curlmyip.com* OR *edns.ip\-api.com* OR *eth0.me* OR *freegeoip.app* OR *geoipy.com* OR *getip.pro* OR *icanhazip.com* OR *ident.me* OR *ifconfig.io* OR *ifconfig.me* OR *ip\-api.com* OR *ip.anysrc.net* OR *ip.tyk.nu* OR *ipaddressworld.com* OR *ipecho.net* OR *ipinfo.io* OR *ipof.in* OR *ipv4.icanhazip.com* OR *ipv4bot.whatismyipaddress.com* OR *ipwho.is* OR *l2.io* OR *myexternalip.com* OR *wgetip.com* OR *whatismyip.akamai.com* OR *wtfismyip.com*)) AND (NOT (Image:*\\brave.exe OR (Image:("C\:\\Program\ Files\\Google\\Chrome\\Application\\chrome.exe" OR "C\:\\Program\ Files\ \(x86\)\\Google\\Chrome\\Application\\chrome.exe")) OR (Image:("C\:\\Program\ Files\\Mozilla\ Firefox\\firefox.exe" OR "C\:\\Program\ Files\ \(x86\)\\Mozilla\ Firefox\\firefox.exe")) OR (Image:("C\:\\Program\ Files\ \(x86\)\\Internet\ Explorer\\iexplore.exe" OR "C\:\\Program\ Files\\Internet\ Explorer\\iexplore.exe")) OR Image:*\\maxthon.exe OR (Image:"C\:\\Program\ Files\ \(x86\)\\Microsoft\\EdgeWebView\\Application\\*" OR Image:*\\WindowsApps\\MicrosoftEdge.exe OR (Image:("C\:\\Program\ Files\ \(x86\)\\Microsoft\\Edge\\Application\\msedge.exe" OR "C\:\\Program\ Files\\Microsoft\\Edge\\Application\\msedge.exe"))) OR ((Image:("C\:\\Program\ Files\ \(x86\)\\Microsoft\\EdgeCore\\*" OR "C\:\\Program\ Files\\Microsoft\\EdgeCore\\*")) AND (Image:(*\\msedge.exe OR *\\msedgewebview2.exe))) OR Image:*\\opera.exe OR Image:*\\safari.exe OR Image:*\\seamonkey.exe OR Image:*\\vivaldi.exe OR Image:*\\whale.exe)))If the events are first normalised with NXLog Enterprise Edition to use the ECS Schema, then the query will use slightly different field names:
$ sigma convert -t elasticsearch -p ecs_windows -s dns_query_win_susp_ipify.yml Parsing Sigma rules [####################################] 100% (dns.question.name:(*api.2ip.ua* OR *api.ipify.org* OR *bot.whatismyipaddress.com* OR *canireachthe.net* OR *checkip.amazonaws.com* OR *checkip.dyndns.org* OR *curlmyip.com* OR *edns.ip\-api.com* OR *eth0.me* OR *freegeoip.app* OR *geoipy.com* OR *getip.pro* OR *icanhazip.com* OR *ident.me* OR *ifconfig.io* OR *ifconfig.me* OR *ip\-api.com* OR *ip.anysrc.net* OR *ip.tyk.nu* OR *ipaddressworld.com* OR *ipecho.net* OR *ipinfo.io* OR *ipof.in* OR *ipv4.icanhazip.com* OR *ipv4bot.whatismyipaddress.com* OR *ipwho.is* OR *l2.io* OR *myexternalip.com* OR *wgetip.com* OR *whatismyip.akamai.com* OR *wtfismyip.com*)) AND (NOT (process.executable:*\\brave.exe OR (process.executable:("C\:\\Program\ Files\\Google\\Chrome\\Application\\chrome.exe" OR "C\:\\Program\ Files\ \(x86\)\\Google\\Chrome\\Application\\chrome.exe")) OR (process.executable:("C\:\\Program\ Files\\Mozilla\ Firefox\\firefox.exe" OR "C\:\\Program\ Files\ \(x86\)\\Mozilla\ Firefox\\firefox.exe")) OR (process.executable:("C\:\\Program\ Files\ \(x86\)\\Internet\ Explorer\\iexplore.exe" OR "C\:\\Program\ Files\\Internet\ Explorer\\iexplore.exe")) OR process.executable:*\\maxthon.exe OR (process.executable:"C\:\\Program\ Files\ \(x86\)\\Microsoft\\EdgeWebView\\Application\\*" OR process.executable:*\\WindowsApps\\MicrosoftEdge.exe OR (process.executable:("C\:\\Program\ Files\ \(x86\)\\Microsoft\\Edge\\Application\\msedge.exe" OR "C\:\\Program\ Files\\Microsoft\\Edge\\Application\\msedge.exe"))) OR ((process.executable:("C\:\\Program\ Files\ \(x86\)\\Microsoft\\EdgeCore\\*" OR "C\:\\Program\ Files\\Microsoft\\EdgeCore\\*")) AND (process.executable:(*\\msedge.exe OR *\\msedgewebview2.exe))) OR process.executable:*\\opera.exe OR process.executable:*\\safari.exe OR process.executable:*\\seamonkey.exe OR process.executable:*\\vivaldi.exe OR process.executable:*\\whale.exe)) -
Sentinelone:
$ sigma plugin list $ sigma plugin install sentinelone $ sigma convert -t sentinelone -p sysmon -s dns_query_win_susp_ipify.yml Parsing Sigma rules [####################################] 100% ObjectType = "DNS" AND (EndpointOS = "windows" AND ((DnsRequest containsCIS "api.2ip.ua" OR DnsRequest containsCIS "api.ipify.org" OR DnsRequest containsCIS "bot.whatismyipaddress.com" OR DnsRequest containsCIS "canireachthe.net" OR DnsRequest containsCIS "checkip.amazonaws.com" OR DnsRequest containsCIS "checkip.dyndns.org" OR DnsRequest containsCIS "curlmyip.com" OR DnsRequest containsCIS "edns.ip-api.com" OR DnsRequest containsCIS "eth0.me" OR DnsRequest containsCIS "freegeoip.app" OR DnsRequest containsCIS "geoipy.com" OR DnsRequest containsCIS "getip.pro" OR DnsRequest containsCIS "icanhazip.com" OR DnsRequest containsCIS "ident.me" OR DnsRequest containsCIS "ifconfig.io" OR DnsRequest containsCIS "ifconfig.me" OR DnsRequest containsCIS "ip-api.com" OR DnsRequest containsCIS "ip.anysrc.net" OR DnsRequest containsCIS "ip.tyk.nu" OR DnsRequest containsCIS "ipaddressworld.com" OR DnsRequest containsCIS "ipecho.net" OR DnsRequest containsCIS "ipinfo.io" OR DnsRequest containsCIS "ipof.in" OR DnsRequest containsCIS "ipv4.icanhazip.com" OR DnsRequest containsCIS "ipv4bot.whatismyipaddress.com" OR DnsRequest containsCIS "ipwho.is" OR DnsRequest containsCIS "l2.io" OR DnsRequest containsCIS "myexternalip.com" OR DnsRequest containsCIS "wgetip.com" OR DnsRequest containsCIS "whatismyip.akamai.com" OR DnsRequest containsCIS "wtfismyip.com") AND (NOT (SrcProcImagePath endswithCIS "\brave.exe" OR (SrcProcImagePath In Contains AnyCase ("C:\Program Files\Google\Chrome\Application\chrome.exe","C:\Program Files (x86)\Google\Chrome\Application\chrome.exe")) OR (SrcProcImagePath In Contains AnyCase ("C:\Program Files\Mozilla Firefox\firefox.exe","C:\Program Files (x86)\Mozilla Firefox\firefox.exe")) OR (SrcProcImagePath In Contains AnyCase ("C:\Program Files (x86)\Internet Explorer\iexplore.exe","C:\Program Files\Internet Explorer\iexplore.exe")) OR SrcProcImagePath endswithCIS "\maxthon.exe" OR (SrcProcImagePath startswithCIS "C:\Program Files (x86)\Microsoft\EdgeWebView\Application\" OR SrcProcImagePath endswithCIS "\WindowsApps\MicrosoftEdge.exe" OR (SrcProcImagePath In Contains AnyCase ("C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe","C:\Program Files\Microsoft\Edge\Application\msedge.exe"))) OR ((SrcProcImagePath startswithCIS "C:\Program Files (x86)\Microsoft\EdgeCore\" OR SrcProcImagePath startswithCIS "C:\Program Files\Microsoft\EdgeCore\") AND (SrcProcImagePath endswithCIS "\msedge.exe" OR SrcProcImagePath endswithCIS "\msedgewebview2.exe")) OR SrcProcImagePath endswithCIS "\opera.exe" OR SrcProcImagePath endswithCIS "\safari.exe" OR SrcProcImagePath endswithCIS "\seamonkey.exe" OR SrcProcImagePath endswithCIS "\vivaldi.exe" OR SrcProcImagePath endswithCIS "\whale.exe"))))
-
Conclusion
We’ve shown how a generic Sigma rule can be converted to an actual query that can be applied to events collected by NXLog and forwarded to Splunk, Elasticsearch, and/or SentinelOne. In a future article we’ll explore two additional ways of using Sigma with NXLog: at source (decentralised) and at datasource level using NXLog’s favourite schemaless database: Raijin.
