Real-Time Machine Learning Models To Detect Cyber And Physical Anomalies In Power Systems

A Smart Grid is a cyber-physical system (CPS) that tightly integrates computation and networking with physical processes to provide reliable two-way communication between electricity companies and customers. However, the grid availability and integrity are constantly threatened by both physical faults and cyber-attacks which may have a detrimental socio-economic impact. The frequency of the faults and attacks is increasing every year due to the extreme weather events and strong reliance on the open internet architecture that is vulnerable to cyber-attacks. In May 2021, for instance, Colonial Pipeline, one of the largest pipeline operators in the U.S., transports refined gasoline and jet fuel from Texas up the East Coast to New York was forced to shut down after being attacked by ransomware, causing prices to rise at gasoline pumps across the country. Enhancing situational awareness within the grid can alleviate these risks and avoid their adverse consequences. As part of this process, the phasor measurement units (PMU) are among the suitable assets since they collect time-synchronized measurements of grid status (30-120 samples/s), enabling the operators to react rapidly to potential anomalies. However, it is still challenging to process and analyze the open-ended source of PMU data as there are more than 2500 PMU distributed across the U.S. and Canada, where each of which generates more than 1.5 TB/month of streamed data. Further, the offline machine learning algorithms cannot be used in this scenario, as they require loading and scanning the entire dataset before processing. The ultimate objective of this dissertation is to develop early detection of cyber and physical anomalies in a real-time streaming environment setting by mining multi-variate large-scale synchrophasor data. To accomplish this objective, we start by investigating the cyber and physical anomalies, analyzing their impact, and critically reviewing the current detection approaches. Then, multiple machine learning models were designed to identify physical and cyber anomalies; the first one is an artificial neural network-based approach for detecting the False Data Injection (FDI) attack. This attack was specifically selected as it poses a serious risk to the integrity and availability of the grid; Secondly, we extend this approach by developing a Random Forest Regressor-based model which not only detects anomalies, but also identifies their location and duration; Lastly, we develop a real-time hoeffding tree-based model for detecting anomalies in steaming networks, and explicitly handling concept drifts. These models have been tested and the experimental results confirmed their superiority over the state-of-the-art models in terms of detection accuracy, false-positive rate, and processing time, making them potential candidates for strengthening the grid\u27s security

AN INTELLIGENT PASSIVE ISLANDING DETECTION AND CLASSIFICATION SCHEME FOR A RADIAL DISTRIBUTION SYSTEM

Distributed generation (DG) provides users with a dependable and cost-effective source of electricity. These are directly connected to the distribution system at customer load locations. Integration of DG units into an existing system has significantly high importance due to its innumerable advantages. The high penetration level of distributed generation (DG) provides vast techno-economic and environmental benefits, such as high reliability, reduced total system losses, efficiency, low capital cost, abundant in nature, and low carbon emissions. However, one of the most challenges in microgrids (MG) is the island mode operations of DGs. the effective detection of islanding and rapid DG disconnection is essential to prevent safety problems and equipment damage. The most prevalent islanding protection scheme is based on passive techniques that cause no disruption to the system but have extensive non-detection zones. As a result, the thesis tries to design a simple and effective intelligent passive islanding detection approach using a CatBoost classifier, as well as features collected from three-phase voltages and instantaneous power per phase visible at the DG terminal. This approach enables initial features to be extracted using the Gabor transform (GT) technique. This signal processing (SP) technique illustrates the time-frequency representation of the signal, revealing several hidden features of the processed signals to be the input of the intelligent classifier. A radial distribution system with two DG units was utilized to evaluate the effectiveness of the proposed islanding detection method. The effectiveness of the proposed islanding detection method was verified by comparing its results to those of other methods that use a random forest (RF) or a basic artificial neural network (ANN) as a classifier. This was accomplished through extensive simulations using the DigSILENT Power Factory® software. Several measures are available, including accuracy (F1 Score), the area under the curve (AUC), and training time. The suggested technique has a classification accuracy of 97.1 per cent for both islanded and non-islanded events. However, the RF and ANN classifiers\u27 accuracies for islanding and non-islanding events, respectively, are proven to be 94.23 and 54.8 per cent, respectively. In terms of the training time, the ANN, RF, and CatBoost classifiers have training times of 1.4 seconds, 1.21 seconds, and 0.88 seconds, respectively. The detection time for all methods was less than one cycle. These metrics demonstrate that the suggested strategy is robust and capable of distinguishing between the islanding event and other system disruptions

A Review of Classification Problems and Algorithms in Renewable Energy Applications

Classification problems and their corresponding solving approaches constitute one of the fields of machine learning. The application of classification schemes in Renewable Energy (RE) has gained significant attention in the last few years, contributing to the deployment, management and optimization of RE systems. The main objective of this paper is to review the most important classification algorithms applied to RE problems, including both classical and novel algorithms. The paper also provides a comprehensive literature review and discussion on different classification techniques in specific RE problems, including wind speed/power prediction, fault diagnosis in RE systems, power quality disturbance classification and other applications in alternative RE systems. In this way, the paper describes classification techniques and metrics applied to RE problems, thus being useful both for researchers dealing with this kind of problem and for practitioners of the field

Towards Automated Machine Learning on Imperfect Data for Situational Awareness in Power System

The increasing penetration of renewable energy sources (such as solar and wind) and incoming widespread electric vehicles charging introduce new challenges in the power system. Due to the variability and uncertainty of these sources, reliable and cost-effective operations of the power system rely on high level of situational awareness. Thanks to the wide deployment of sensors (e.g., phasor measurement units (PMUs) and smart meters) and the emerging smart Internet of Things (IoT) sensing devices in the electric grid, large amounts of data are being collected, which provide golden opportunities to achieve high level of situational awareness for reliable and cost-effective grid operations.To better utilize the data, this dissertation aims to develop Machine Learning (ML) methods and provide fundamental understanding and systematic exploitation of ML for situational awareness using large amounts of imperfect data collected in power systems, in order to improve the reliability and resilience of power systems.However, building excellent ML models needs clean, accurate and sufficient training data. The data collected from the real-world power system is of low quality. For example, the data collected from wind farms contains a mixture of ramp and non-ramp as well as the mingle of heterogeneous dynamics data; the data in the transmission grid contains noisy, missing, insufficient and inaccurate timestamp data. Employing ML without considering these distinct features in real-world applications cannot build good ML models. This dissertation aims to address these challenges in two applications, wind generation forecast and power system event classification, by developing ML models in an automated way with less efforts from domain experts, as the cost of processing such large amounts of imperfect data by experts can be prohibitive in practice.First, we take heterogeneous dynamics into consideration, especially for ramp events. A Drifting Streaming Peaks-over-Threshold (DSPOT) enhanced self-evolving neural networks-based short-term wind farm generation forecast is proposed by utilizing dynamic ramp thresholds to separate the ramp and non-ramp events, based on which different neural networks are trained to learn different dynamics of wind farm generation. As the efficacy of the neural networks relies on the quality of training datasets (i.e., the classification accuracy of ramp and non-ramp events), a Bayesian optimization based approach is developed to optimize the parameters of DSPOT to enhance the quality of the training datasets and the corresponding performance of the neural networks. Experimental results show that compared with other forecast approaches, the proposed forecast approach can substantially improve the forecast accuracy, especially for ramp events. Next, we address the challenges of event classification due to the low-quality PMU measurements and event logs. A novel machine learning framework is proposed for robust event classification, which consists of three main steps: data preprocessing, fine-grained event data extraction, and feature engineering. Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e.g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers. Moreover, with the small number of good features, we need much less training data to train a good event classifier, which can address the challenge of insufficient and imbalanced training data, and the training time is negligible compared to neural network based approaches. Based on the proposed framework, we developed a workflow for event classification using the real-world PMU data streaming into the system in real time. Using the proposed framework, robust event classifiers can be efficiently trained based on many off-the-shelf lightweight machine learning models. Numerical experiments using the real-world dataset from the Western Interconnection of the U.S power transmission grid show that the event classifiers trained under the proposed framework can achieve high classification accuracy while being robust against low-quality data. Subsequently, we address the challenge of insufficient training labels. The real-world PMU data is often incomplete and noisy, which can significantly reduce the efficacy of existing machine learning techniques that require high-quality labeled training data. To obtain high-quality event logs for large amounts of PMU measurements, it requires significant efforts from domain experts to maintain the event logs and even hand-label the events, which can be prohibitively costly or impractical in practice. So we develop a weakly supervised machine learning approach that can learn a good event classifier using a few labeled PMU data. The key idea is to learn the labels from unlabeled data using a probabilistic generative model, in order to improve the training of the event classifiers. Experimental results show that even with 95\% of unlabeled data, the average accuracy of the proposed method can still achieve 78.4\%. This provides a promising way for domain experts to maintain the event logs in a less expensive and automated manner. Finally, we conclude the dissertation and discuss future directions

Operation, Monitoring, and Protection of Future Power Systems: Advanced Congestion Forecast and Dynamic State Estimation Applications

The electrical power systems are undergoing drastic changes such as increasing levels of renewable energy sources, energy storage, electrification of energy-efficient loads such as heat pumps and electric vehicles, demand-side resources, etc., in the last decade, and more changes will be followed in the near future. The emergence of digitalization and advanced communication in the case of distribution systems to enhance the performance of the electricity infrastructure also adds further complexities. These changes pose challenges such as increased levels of network congestion, voltage variations, protection mis-operations, increased needs for real-time monitoring, and improved planning practices of the system operators. These challenges will require the development of new paradigms to operate the power grids securely, safely, and economically. This thesis attempted to address those challenges and had the following main contributions:First, the thesis started by presenting a comprehensive assessment framework to address the distribution system operators’ future-readiness and help the distribution system operators to determine the current status of their network infrastructures, business models, and policies and thus identify the pathways for the required developments for the smooth transition towards future intelligent distribution grids.Second, the thesis presents an advanced congestion forecast tool that would support the distribution system operators to forecast and visualize network congestion and voltage variations issues for multiple forecasting horizons ranging from close-to-real time to a day-ahead. The tool is based on a probabilistic power flow that incorporates forecasts of solar photovoltaic production and electricity demand, combined with advanced load models and different operating modes of solar photovoltaic inverters. The tool has been integrated to an existing industrial graded distribution management system via an IoT platform Codex Smart Edge of Atos Worldgrid. The results from case studies demonstrated that the tool performs satisfactorily for both small and large networks and can visualise the cumulative probabilities of network congestion and voltage variations for a variety of forecast horizons as desired by the distribution system operator.Third, a dynamic state estimation-based protection scheme for the transmission lines which does not require complicated relay settings and coordination has been demonstrated using an experimental setup at Chalmers power system laboratory. The scheme makes use of the real-time measurements provided by advanced sensors which are developed by Smart State Technology, The Netherlands. The experimental validations of the scheme have been performed under different fault types and conditions, e.g., unbalanced faults, three-phase faults, high impedance faults, hidden failures, inductive load conditions, etc. The results have shown that the scheme performs adequately in both normal and fault conditions and thus the scheme would work for transmission line protection by avoiding relay coordination and settings issues.Finally, the thesis presents a decentralized dynamic state estimation method for estimating the dynamic states of a transmission line in real-time. This method utilizes the sampled measurements from the local end of a transmission line, and thereafter dynamic state estimation is performed by employing an unscented Kalman filter. The advantage of the method is that the remote end state variables of a transmission line can be estimated using only the local end variables and, hence, the need for communication infrastructure is eliminated. Furthermore, an exact nonlinear model of the transmission line is utilized and the dynamic state estimation of one transmission line is independent of the other lines. These features in turn result in reduced complexity, higher accuracy, and easier implementation of the decentralized estimator. The method is envisioned to have potential applications in transmission line monitoring, control, and protection

Grid Frequency Forecasting in University Campuses using Convolutional LSTM

The modern power grid is facing increasing complexities, primarily stemming from the integration of renewable energy sources and evolving consumption patterns. This paper introduces an innovative methodology that harnesses Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks to establish robust time series forecasting models for grid frequency. These models effectively capture the spatiotemporal intricacies inherent in grid frequency data, significantly enhancing prediction accuracy and bolstering power grid reliability. The research explores the potential and development of individualized Convolutional LSTM (ConvLSTM) models for buildings within a university campus, enabling them to be independently trained and evaluated for each building. Individual ConvLSTM models are trained on power consumption data for each campus building and forecast the grid frequency based on historical trends. The results convincingly demonstrate the superiority of the proposed models over traditional forecasting techniques, as evidenced by performance metrics such as Mean Square Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). Additionally, an Ensemble Model is formulated to aggregate insights from the building-specific models, delivering comprehensive forecasts for the entire campus. This approach ensures the privacy and security of power consumption data specific to each building.Comment: 9 pages, 20 figure

Systematic review of energy theft practices and autonomous detection through artificial intelligence methods

Energy theft poses a significant challenge for all parties involved in energy distribution, and its detection is crucial for maintaining stable and financially sustainable energy grids. One potential solution for detecting energy theft is through the use of artificial intelligence (AI) methods. This systematic review article provides an overview of the various methods used by malicious users to steal energy, along with a discussion of the challenges associated with implementing a generalized AI solution for energy theft detection. In this work, we analyze the benefits and limitations of AI methods, including machine learning, deep learning, and neural networks, and relate them to the specific thefts also analyzing problems arising with data collection. The article proposes key aspects of generalized AI solutions for energy theft detection, such as the use of smart meters and the integration of AI algorithms with existing utility systems. Overall, we highlight the potential of AI methods to detect various types of energy theft and emphasize the need for further research to develop more effective and generalized detection systems, providing key aspects of possible generalized solutions

NILM techniques for intelligent home energy management and ambient assisted living: a review

The ongoing deployment of smart meters and different commercial devices has made electricity disaggregation feasible in buildings and households, based on a single measure of the current and, sometimes, of the voltage. Energy disaggregation is intended to separate the total power consumption into specific appliance loads, which can be achieved by applying Non-Intrusive Load Monitoring (NILM) techniques with a minimum invasion of privacy. NILM techniques are becoming more and more widespread in recent years, as a consequence of the interest companies and consumers have in efficient energy consumption and management. This work presents a detailed review of NILM methods, focusing particularly on recent proposals and their applications, particularly in the areas of Home Energy Management Systems (HEMS) and Ambient Assisted Living (AAL), where the ability to determine the on/off status of certain devices can provide key information for making further decisions. As well as complementing previous reviews on the NILM field and providing a discussion of the applications of NILM in HEMS and AAL, this paper provides guidelines for future research in these topics.Agência financiadora: Programa Operacional Portugal 2020 and Programa Operacional Regional do Algarve 01/SAICT/2018/39578 Fundação para a Ciência e Tecnologia through IDMEC, under LAETA: SFRH/BSAB/142998/2018 SFRH/BSAB/142997/2018 UID/EMS/50022/2019 Junta de Comunidades de Castilla-La-Mancha, Spain: SBPLY/17/180501/000392 Spanish Ministry of Economy, Industry and Competitiveness (SOC-PLC project): TEC2015-64835-C3-2-R MINECO/FEDERinfo:eu-repo/semantics/publishedVersio

Distributed energy resources and the application of AI, IoT, and blockchain in smart grids

Smart grid (SG), an evolving concept in the modern power infrastructure, enables the two-way flow of electricity and data between the peers within the electricity system networks (ESN) and its clusters. The self-healing capabilities of SG allow the peers to become active partakers in ESN. In general, the SG is intended to replace the fossil fuel-rich conventional grid with the distributed energy resources (DER) and pools numerous existing and emerging know-hows like information and digital communications technologies together to manage countless operations. With this, the SG will able to “detect, react, and pro-act” to changes in usage and address multiple issues, thereby ensuring timely grid operations. However, the “detect, react, and pro-act” features in DER-based SG can only be accomplished at the fullest level with the use of technologies like Artificial Intelligence (AI), the Internet of Things (IoT), and the Blockchain (BC). The techniques associated with AI include fuzzy logic, knowledge-based systems, and neural networks. They have brought advances in controlling DER-based SG. The IoT and BC have also enabled various services like data sensing, data storage, secured, transparent, and traceable digital transactions among ESN peers and its clusters. These promising technologies have gone through fast technological evolution in the past decade, and their applications have increased rapidly in ESN. Hence, this study discusses the SG and applications of AI, IoT, and BC. First, a comprehensive survey of the DER, power electronics components and their control, electric vehicles (EVs) as load components, and communication and cybersecurity issues are carried out. Second, the role played by AI-based analytics, IoT components along with energy internet architecture, and the BC assistance in improving SG services are thoroughly discussed. This study revealed that AI, IoT, and BC provide automated services to peers by monitoring real-time information about the ESN, thereby enhancing reliability, availability, resilience, stability, security, and sustainability