Achieving distributed device situational awareness through cloud-based data management
Database Schema
A fundamental property of NoSQL databases is that they are schema-free, making them particularly suitable for OT systems. Large real-time distributed systems have data schemas that are naturally complex and dynamic.
The industry is largely moving away from the concept of a fixed, single data model (e.g. CORBA) for several reasons -- integration and forward compatibility being two of the most important. In modern OT systems, data schemas must be dynamically discoverable at run-time and must be extensible and/or mutable.
Not only must these schemas be captured by the data management system, they may also be examined for analysis. In other words, meta-data and data may be equally important for situational awareness.
Data Correlation, Querying & Retrieval
Achieving Situational Awareness (SA) requires correlating real-time and historical data. In technical terms, this means running pre-compiled and dynamic queries continuously on the data streams as they are being written to the NoSQL database.
All SA queries are triggered on the occurrence of some real-time event, such as the price change of a security. Once the real-time event occurs the query needs to correlate with historical data to decide what, if any, action to take.
Content Distribution
NoSQL databases have already conquered the world of on-demand content distribution. This is perhaps the most telling by the fact that Netflix uses Apache Cassandra for its streaming service [Izrailevsky, 2011].
The result of queries fall into two categories: alerts and content retrieval. Alerts distribution is trivial in most cases: a single high priority message. In the case of data retrieval, distribution becomes an important factor. As shown in Figure 1 below, the data resulting from a query will need to be retrieved, ordered, and distributed to the consumer in a timely fashion.
Use Case: Power Generation
There are hundreds of windmill farms being constructed around the world. Imagine a windmill farm constructed as a hierarchical distributed system.
At the lowest level of the hierarchy each windmill is in itself a distributed system: it has a vast array of sensors that produce information about current power generation as well as structural and environmental data that is consumed by the windmill to operate safely and efficiently.
This data is also automatically shared with the farm control center. The control center is responsible for maintaining contact with other farms and is linked to two important external systems: the power grid and meteorological systems.
As an example consider an instance where a windmill detects sudden and unexpected strong gusts of wind. The windmill determines that the wind patterns are irregular based on precise measurements over the past week and more general measurements over the past few years.
Potentially harmful, the windmill moves to a fail-safe mode. An alert is sent to a command center, which can then use continuous real-time controls to carefully tune the performance of the other turbines based on each windmill’s current state. The command center can also alert farms that are downwind so that they can adjust performance based.
Another scenario involves real-time pricing updates from energy trading. The power grid and power exchanges provide information on load, demand and pricing of electricity.
If demand and prices are low relative to current output, the farm can automatically redirect power generated to storage or cease production. If demand and prices are high relative to current production, the farm can move to peak productivity and sell stored power. Just like securities trading, there is the potential for designing algorithms to detect beneficial patterns in power generation and distribution.
Meteorological systems can also be based on algorithmic statistical analysis, and can benefit from two-way communication with windmill farms. On one hand farms become important real-time sensor stations for the meteorological data; on the other, the farms are dependent on weather forecasts from the meteorological systems to carefully tune for optimal output.
The algorithms used in both power generation and meteorological systems need timely access to real-time events as well as historical trends. If data is delayed for any reason, the consequences can range from inefficient output that results in lost revenue, to catastrophic failure and loss of infrastructure (and possibly human lives).
Loading comments... Write a comment