Collecting logs from General Electric CIMPLICITY and sending them to Elastic could be a complex task due to this rather unique combination log sources and destination. In this post, we will take a look at how you can forward log data from GE CIMPLICITY to Elastic using NXLog.
General Electric CIMPLICITY
General Electric CIMPLICITY is a human-machine interface (HMI) and SCADA system solution based on a client-server architecture of servers and viewers. This architecture allows viewers to visualize data and control actions within plants located across the globe. The server’s primary function is to collect and distribute data. A viewer has full access to the data a server has collected once it has connected to that server. The ability to seamlessly network servers and viewers for the purpose of sharing data, configurations, and screens eliminates duplicate work and data. This efficient management of resources facilitates faster access to critical data needed for decision-making. Cimplicity is used in some of the largest manufacturing factories around the world.
Collecting GE CIMPLICITY logs
CIMPLICITY produces a wide variety of logs about its operations. Some of the logs are available through Windows Event Log and network monitoring, but most of the logs are in the format of flat files.
Due to the critical nature and scope of the systems CIMPLICITY controls, there is no room for errors. Its stable, uninterrupted operation is crucial to plant safety. Although CIMPLICITY logs contain valuable information about the systems it controls, the relatively high level of log noise and the lack of a consistent log format present some challenges.
NXLog Enterprise Edition is a lightweight, modular log collection tool, capable of tackling the most demanding cases log collection may pose. Owing to its rich set of features, it can read almost any log format and parse fields to produce structured data for further processing. For these reasons, it is the perfect tool for monitoring and collecting CIMPLICITY logs.
- Logging and Archiving
-
CIMPLICITY provides a database logger which is capable of collecting, analyzing, and creating reports from a variety of ODBC (open database connectivity) complaint databases. You can create, configure, edit tables, and also specify when and what ODBC data source you would like to gather log events from, for any selected process.
- Collecting GE CIMPLICITY logs from Windows Event Log
-
Windows Event Log is the primary logging facility on the Windows platform. The logs CIMPLICITY services generate contain project log files, system log files, and web configuration services logs. Logs can be read and collected using an event id related to CIMPLICITY or by a given source name.
- Collecting GE CIMPLICITY logs from file
-
CIMPLICITY’s file-based logs include project status and system status logs, counters log files, protocol stack trace logs, as well as optional OPC client debug tracing. With CIMPLICITY Log Viewer’s powerful capabilities, you can view project status and system status log files in other formats including CSV, ASCII, or TXT.
- GE CIMPLICITY passive network monitoring
-
NXLog can passively monitor network traffic and generate logs for most network protocols. This ability to log network communication between servers and viewers can provide another valuable log source.
Data normalization and log aggregation are other features that NXLog can provide CIMPLICITY. With NXLog’s ability to collect logs from literally any file, in any format, it is ideally suited for integrating with CIMPLICITY’s wide variety of log types and file formats.
For more information on integrating NXLog with Cimplicity, see the General Electric CIMPLICITY integration guide.
The above mentioned log sources, and the features NXLog provides all play an important role when normalizing logs in order to be accepted by Elastic.
Sending logs to Elastic
Elasticsearch is a search engine and document database for storing, searching, and analyzing log data that you can deploy locally. On the other hand, Elastic Cloud is a SaaS solution that adds value to Elastic with its cloud-native features, such as managed enterprise search, data visualization, and security.
NXLog can integrate with both products by collecting and sending logs or as a relay, aggregating logs it receives from various sources and forwarding them.
- Elasticsearch logs
-
NXLog Enterprise Edition provides the om_elasticsearch output module that supports sending logs in bulk to Elasticsearch. With the NXLog Community Edition, the om_http module sends logs to Elasticsearch for low-volume logging scenarios. Because it sends a request to the Elasticsearch HTTP REST API for each event, HTTP request and response latency limit the maximum logging throughput.
- Elastic Cloud logs
-
The Elasticsearch REST API is used to send logs to the Elastic cloud. For NXLog to connect to the API, it requires an API key, the Elasticsearch endpoint, and the Elastic Cloud CA certificate. These requirements are configured on the main menu under Management > Stack Management.
You can view the log records by logging in to your Elastic Cloud instance. Navigate to Analytics > Discover from the main menu, then select the relevant index pattern to display the data.
For more information on configuring NXLog and sending logs to Elasticsearch and Elastic Cloud respectively, please visit the Elasticsearch and Kibana and Elastic Cloud integration sections in the NXLog User Guide.
