- Introduction
- Deployment
- Configuration
- OS Support
- Integration
- 42. Amazon Web Services (AWS)
- 43. Apache HTTP Server
- 44. Apache Tomcat
- 45. APC Automatic Transfer Switch
- 46. Apple macOS kernel
- 47. ArcSight Common Event Format (CEF)
- 48. Box
- 49. Brocade Switches
- 50. Browser History Logs
- 51. Check Point
- 52. Cisco ACS
- 53. Cisco ASA
- 54. Cisco FireSIGHT
- 55. Cisco IPS
- 56. Cloud Instance Metadata
- 57. Common Event Expression (CEE)
- 58. Dell EqualLogic
- 59. Dell iDRAC
- 60. Dell PowerVault MD Series
- 61. Devo
- 62. DHCP logs
- 63. DNS Monitoring
- 64. Docker
- 65. Elasticsearch and Kibana
- 66. F5 BIG-IP
- 67. File Integrity Monitoring
- 68. FreeRADIUS
- 69. Graylog
- 70. HP ProCurve
- 71. IBM QRadar SIEM
- 72. Industrial Control Systems
- 73. Linux Audit System
- 74. Linux system logs
- 75. Log Event Extended Format (LEEF)
- 76. McAfee Enterprise Security Manager (ESM)
- 77. McAfee ePolicy Orchestrator
- 78. Microsoft Active Directory Domain Controller
- 79. Microsoft Azure
- 80. Microsoft Azure Event Hubs
- 81. Microsoft Azure Sentinel
- 82. Microsoft Exchange
- 83. Microsoft IIS
- 84. Microsoft SharePoint
- 85. Microsoft SQL Server
- 86. Microsoft System Center Endpoint Protection
- 87. Microsoft System Center Configuration Manager
- 88. Microsoft System Center Operations Manager
- 89. MongoDB
- 90. Nagios Log Server
- 91. Nessus Vulnerability Scanner
- 92. NetApp
- 93. .NET application logs
- 94. Nginx
- 95. Okta
- 96. Osquery
- 97. Postfix
- 98. Promise
- 99. Rapid7 InsightIDR SIEM
- 100. RSA NetWitness
- 101. SafeNet KeySecure
- 102. Salesforce
- 103. Snare
- 104. Snort
- 105. Solarwinds Loggly
- 106. Splunk
- 107. Sumo Logic
- 108. Symantec Endpoint Protection
- 109. Synology DiskStation
- 110. Syslog
- 111. Sysmon
- 112. Ubiquiti UniFi
- 113. VMware vCenter
- 114. Windows AppLocker
- 115. Windows Command Line Auditing
- 116. Windows Event Log
- 117. Windows Firewall
- 118. Windows Group Policy
- 119. Windows Management Instrumentation (WMI)
- 120. Windows PowerShell
- 121. Microsoft Windows Update
- 122. Windows USB auditing
- 123. Zeek (formerly Bro) Network Security Monitor
- Troubleshooting
- Enterprise Edition Reference Manual
- NXLog Manager
- NXLog Add-Ons
123. Zeek (formerly Bro) Network Security Monitor
Zeek formerly known as the Bro Network Security Monitor, is a powerful open source Intrusion Detection System (IDS) and network traffic analysis framework. The Zeek engine captures traffic and converts it to a series of high-level events. These events are then analyzed according to customizable policies. Zeek supports real-time alerts, data logging for further investigation, and automatic program execution for detected anomalies. Zeek is able to analyze different protocols, including HTTP, FTP, SMTP, and DNS; as well as run host and port scans, detect signatures, and discover syn-floods.
NXLog can be configured to collect Zeek logs.
123.1. About Zeek logs
Zeek creates different log files in order to record network activities such as files transferred over the network, SSL sessions, and HTTP requests. By default, Zeek provides 60 different log files.
File | Description |
---|---|
conn.log |
TCP/UDP/ICMP connections |
dhcp.log |
DHCP leases |
dns.log |
DNS activity |
files.log |
Summaries of files transferred over the network |
ftp.log |
FTP activity |
http.log |
HTTP requests and replies |
smtp.log |
SMTP transactions |
ssl.log |
SSL/TLS handshake information |
weird.log |
Unexpected network-level activity |
Zeek produces human-readable logs in a format similar to the W3C log format. Each log file uses a different set of fields.
#separator \x09 #set_separator , #empty_field (empty) #unset_field - #path dns #open 2020-05-27-22-00-01 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id rtt query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs rejected #types time string addr port addr port enum count interval string count string count string count string bool bool bool bool count vector[string] vector[interval] bool 1590634800.248362 C1ggH7liCnwAfLjw9 192.168.1.7 53743 192.168.1.1 53 udp 18876 - 250.255.255.239.in-addr.arpa 1 C_INTERNET 12 PTR 3 NXDOMAIN F F T F 0 - - F 1590634800.259227 C1ggH7liCnwAfLjw9 192.168.1.7 53743 192.168.1.1 53 udp 18876 - 250.255.255.239.in-addr.arpa 1 C_INTERNET 12 PTR 3 NXDOMAIN F F T F 0 - - F 1590634800.274483 CTQxOg2sSOuUO5AZy8 192.168.1.7 47182 192.168.1.1 53 udp 48442 - 7.1.168.192.in-addr.arpa 1 C_INTERNET 12 PTR 3 NXDOMAIN F F T F 0 - - F
For more information about Zeek logging, see the Zeek Manual.
123.2. Parsing Zeek logs
NXLog Enterprise Edition can parse Zeek logs with the xm_w3c module.
Note
|
The following configurations have been tested with Zeek version 3.0.6 LTS. |
This configuration reads Zeek logs from a directory, parses with xm_w3c, and writes out events in JSON format.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Extension _json>
Module xm_json
</Extension>
<Extension w3c_parser>
Module xm_w3c
</Extension>
<Input zeek>
Module im_file
File '/opt/zeek/logs/current/*.log'
InputType w3c_parser
</Input>
<Output zeek_json>
Module om_file
File '/tmp/zeek_logs.json'
Exec to_json();
</Output>
The following output from this configuration represents a sample event logged by Zeek after being parsed by NXLog and converted to JSON format. Spacing and line breaks have been added for readability.
{
"ts": "1590636144.680688",
"uid": "C1InwK3K6fhY6YdvRe",
"id.orig_h": "192.168.1.7",
"id.orig_p": "45500",
"id.resp_h": "35.222.85.5",
"id.resp_p": "80",
"version": "1",
"cipher": "GET",
"curve": "connectivity-check.ubuntu.com",
"server_name": "/",
"resumed": null,
"last_alert": "1.1",
"next_protocol": null,
"established": null,
"cert_chain_fuids": "0",
"client_cert_chain_fuids": "0",
"subject": "204",
"issuer": "No Content",
"client_subject": null,
"client_issuer": null,
"validation_status": "(empty)",
"EventReceivedTime": "2020-05-27T22:22:26.917647-05:00",
"SourceModuleName": "zeek",
"SourceModuleType": "im_file"
}
The xm_w3c module is recommended because it supports reading the field list from the W3C-style log file header. For NXLog Community Edition, the xm_csv module could be used instead to parse Zeek logs. A separate instance of xm_csv must be configured for each log type.
This example has separate xm_csv module instances for the DNS and DHCP log types. Additional CSV parsers could be added for the remaining Zeek log types.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
<Extension csv_parser_dns>
Module xm_csv
Fields ts, uid id.orig_h, id.orig_p, id.resp_h, id.resp_p, proto, \
trans_id, rtt query, qclass, qclass_name, qtype, qtype_name, \
rcode, rcode_name, AA, TC, RD, RA, Z, answers, TTLs, rejected
Delimiter \t
</Extension>
<Extension csv_parser_dhcp>
Module xm_csv
Fields ts, uid, id.orig_h, id.orig_p, id.resp_h, id.resp_p, mac, \
assigned_ip, lease_time, trans_id
Delimiter \t
</Extension>
# xm_fileop provides the `file_basename()` function
<Extension _fileop>
Module xm_fileop
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input zeek>
Module im_file
File '/opt/zeek/spool/zeek/*.log'
<Exec>
if file_basename(file_name()) == 'dhcp.log'
{
csv_parser_dhcp->parse_csv();
}
else if file_basename(file_name()) == 'dns.log'
{
csv_parser_dns->parse_csv();
}
else
{
log_warning('Zeek log type not supported, check configuration');
drop();
}
</Exec>
</Input>
<Output zeek_json>
Module om_file
File '/tmp/ce_zeek_logs.json'
Exec to_json();
</Output>
The following output from this configuration represents a sample event logged by Zeek after being parsed by NXLog and converted to JSON format. Spacing and line breaks have been added for readability.
{
"EventReceivedTime": "2020-05-29 10:55:51",
"SourceModuleName": "zeek",
"SourceModuleType": "im_file",
"ts": "1590767749.877652",
"uid": "CAhAIX1Dl5KFfnhKbi",
"id.orig_h": "192.168.1.7",
"id.orig_p": "42157",
"id.resp_h": "192.168.1.1",
"id.resp_p": "53",
"proto": "udp",
"trans_id": "56765",
"rtt": "0.051801",
"query": "zeek.org",
"qclass": "1",
"qclass_name": "C_INTERNET",
"qtype": "1",
"qtype_name": "A",
"rcode": "0",
"rcode_name": "NOERROR",
"AA": "F",
"TC": "F",
"RD": "T",
"RA": "T",
"Z": "0",
"answers": "192.0.78.212,192.0.78.150",
"TTLs": "60.000000,60.000000",
"rejected": "F"
}