Smart Grids Data Management: A Case for Cassandra

The objective of this paper is to present a SMACK based platform for microgrids data storage and management. The platform is being used in a real microgrid, with an infrastructure that monitors and controls 3 buildings within the GECAD - ISEP/IPP campus, while, at the same time, receives and manages data sources coming from different types of buildings from associated partners, to whom intelligent services are being provided. Microgrid data comes in different formats, different rates and with an increasing volume, as the microgrid itself covers more customers and areas. Based on the atual available computational resources, a Big Data platform based on the SMACK stack was implemented and is presented. The Cassandra component of the stack has evolved. AC version 2 is still supported until the version 4 release, and is often still used in production environments. However, a new stable version, version 3, introduces major optimizations in the storage that bring disk space savings. The main focus of this work is on the Data Storage and the formalization of the data mapping in Cassandra version 3, which is contextualized by means of a short example with data coming from the monitoring infrastructure of the microgrid.This work has received funding from EU Horizon 2020 R&D programme under Marie Sklodowska-Curie grant agreement No 641794 (project DREAM-GO), EUREKA - ITEA2 Project FUSE-IT (nr.13023), Project GREEDI (ANI|P2020 17822), FEDER Funds through COMPETE program and National Funds FCT under the UID/EEA/00760/2013 and SFRH/BD/103089/2014.info:eu-repo/semantics/publishedVersio

A Gossip Algorithm based Clock Synchronization Scheme for Smart Grid Applications

The uprising interest in multi-agent based networked system, and the numerous number of applications in the distributed control of the smart grid leads us to address the problem of time synchronization in the smart grid. Utility companies look for new packet based time synchronization solutions with Global Positioning System (GPS) level accuracies beyond traditional packet methods such as Network Time Proto- col (NTP). However GPS based solutions have poor reception in indoor environments and dense urban canyons as well as GPS antenna installation might be costly. Some smart grid nodes such as Phasor Measurement Units (PMUs), fault detection, Wide Area Measurement Systems (WAMS) etc., requires synchronous accuracy as low as 1 ms. On the other hand, 1 sec accuracy is acceptable in management information domain. Acknowledging this, in this study, we introduce gossip algorithm based clock synchronization method among network entities from the decision control and communication point of view. Our method synchronizes clock within dense network with a bandwidth limited environment. Our technique has been tested in different kinds of network topologies- complete, star and random geometric network and demonstrated satisfactory performance

Big Data decision support system

Includes bibliographical references.2022 Fall.Each day, the amount of data produced by sensors, social and digital media, and Internet of Things is rapidly increasing. The volume of digital data is expected to be doubled within the next three years. At some point, it might not be financially feasible to store all the data that is received. Hence, if data is not analyzed as it is received, the information collected could be lost forever. Actionable Intelligence is the next level of Big Data analysis where data is being used for decision making. This thesis document describes my scientific contribution to Big Data Actionable Intelligence generations. Chapter 1 consists of my colleagues and I's contribution in Big Data Actionable Intelligence Architecture. The proven architecture has demonstrated to support real-time actionable intelligence generation using disparate data sources (e.g., social media, satellite, newsfeeds). This work has been published in the Journal of Big Data. Chapter 2 shows my original method to perform real-time detection of moving targets using Remote Sensing Big Data. This work has also been published in the Journal of Big Data and it has received an issuance of a U.S. patent. As the Field-of-View (FOV) in remote sensing continues to expand, the number of targets observed by each sensor continues to increase. The ability to track large quantities of targets in real-time poses a significant challenge. Chapter 3 describes my colleague and I's contribution to the multi-target tracking domain. We have demonstrated that we can overcome real-time tracking challenges when there are large number of targets. Our work was published in the Journal of Sensors

Advanced Wide-Area Monitoring System Design, Implementation, and Application

Wide-area monitoring systems (WAMSs) provide an unprecedented way to collect, store and analyze ultra-high-resolution synchrophasor measurements to improve the dynamic observability in power grids. This dissertation focuses on designing and implementing a wide-area monitoring system and a series of applications to assist grid operators with various functionalities. The contributions of this dissertation are below: First, a synchrophasor data collection system is developed to collect, store, and forward GPS-synchronized, high-resolution, rich-type, and massive-volume synchrophasor data. a distributed data storage system is developed to store the synchrophasor data. A memory-based cache system is discussed to improve the efficiency of real-time situation awareness. In addition, a synchronization system is developed to synchronize the configurations among the cloud nodes. Reliability and Fault-Tolerance of the developed system are discussed. Second, a novel lossy synchrophasor data compression approach is proposed. This section first introduces the synchrophasor data compression problem, then proposes a methodology for lossy data compression, and finally presents the evaluation results. The feasibility of the proposed approach is discussed. Third, a novel intelligent system, SynchroService, is developed to provide critical functionalities for a synchrophasor system. Functionalities including data query, event query, device management, and system authentication are discussed. Finally, the resiliency and the security of the developed system are evaluated. Fourth, a series of synchrophasor-based applications are developed to utilize the high-resolution synchrophasor data to assist power system engineers to monitor the performance of the grid as well as investigate the root cause of large power system disturbances. Lastly, a deep learning-based event detection and verification system is developed to provide accurate event detection functionality. This section introduces the data preprocessing, model design, and performance evaluation. Lastly, the implementation of the developed system is discussed

Studies of Uncertainties in Smart Grid: Wind Power Generation and Wide-Area Communication

This research work investigates the uncertainties in Smart Grid, with special focus on the uncertain wind power generation in wind energy conversion systems (WECSs) and the uncertain wide-area communication in wide-area measurement systems (WAMSs). For the uncertain wind power generation in WECSs, a new wind speed modeling method and an improved WECS control method are proposed, respectively. The modeling method considers the spatial and temporal distributions of wind speed disturbances and deploys a box uncertain set in wind speed models, which is more realistic for practicing engineers. The control method takes maximum power point tracking, wind speed forecasting, and wind turbine dynamics into account, and achieves a balance between power output maximization and operating cost minimization to further improve the overall efficiency of wind power generation. Specifically, through the proposed modeling and control methods, the wind power control problem is developed as a min-max optimal problem and efficiently solved with semi-definite programming. For the uncertain communication delay and communication loss (i.e. data loss) in WAMSs, the corresponding solutions are presented. First, the real-world communication delay is measured and analyzed, and the bounded modeling method for the communication delay is proposed for widearea applications and further applied for system-area and substation-area protection applications, respectively. The proposed bounded modeling method is expected to be an important tool in the planning, design, and operation of time-critical wide-area applications. Second, the real synchronization signal loss and synchrophasor data loss events are measured and analyzed. For the synchronization signal loss, the potential reasons and solutions are explored. For the synchrophasor data loss, a set of estimation methods are presented, including substitution, interpolation, and forecasting. The estimation methods aim to improve the accuracy and availability of WAMSs, and mitigate the effect of communication failure and data loss on wide-area applications

Co-Optimization of Gas-Electricity Integrated Energy Systems Under Uncertainties

In the United States, natural gas-fired generators have gained increasing popularity in recent years due to low fuel cost and emission, as well as the needed large gas reserves. Consequently, it is worthwhile to consider the high interdependency between the gas and electricity networks. In this dissertation, several co-optimization models for the optimal operation and planning of gas-electricity integrated energy systems (IES) are proposed and investigated considering uncertainties from wind power and load demands. For the coordinated operation of gas-electricity IES: 1) an interval optimization based coordinated operating strategy for the gas-electricity IES is proposed to improve the overall system energy efficiency and optimize the energy flow. The gas and electricity infrastructures are modeled in detail and their operation constraints are fully considered. Then, a demand response program is incorporated into the optimization model, and its effects on the IES operation are investigated. Interval optimization is applied to address wind power uncertainty in IES. 2) a stochastic optimal operating strategy for gas-electricity IES is proposed considering N-1 contingencies in both gas and electricity networks. Since gas pipeline contingencies limit the fuel deliverability to gas-fired units, N-1 contingencies in both gas and electricity networks are considered to ensure that the system operation is able to sustain any possible power transmission or gas pipeline failure. Moreover, wind power uncertainty is addressed by stochastic programming. 3) a robust scheduling model is proposed for gas-electricity IES with uncertain wind power considering both gas and electricity N-1 contingencies. The proposed method is robust against wind power uncertainty to ensure that the system can sustain possible N-1 contingency event of gas pipeline or power transmission. Case studies demonstrate the effectiveness of the proposed models. For the co-optimization planning of gas-electricity IES: a two-stage robust optimization model is proposed for expansion co-planning of gas-electricity IES. The proposed model is solved by the column and constraint generation (C&CG) algorithm. The locations and capacities of new gas-fired generators, power transmission lines, and gas pipelines are optimally determined, which is robust against the uncertainties from electric and gas load growth as well as wind power