Game-Theoretic and Machine-Learning Techniques for Cyber-Physical Security and Resilience in Smart Grid

The smart grid is the next-generation electrical infrastructure utilizing Information and Communication Technologies (ICTs), whose architecture is evolving from a utility-centric structure to a distributed Cyber-Physical System (CPS) integrated with a large-scale of renewable energy resources. However, meeting reliability objectives in the smart grid becomes increasingly challenging owing to the high penetration of renewable resources and changing weather conditions. Moreover, the cyber-physical attack targeted at the smart grid has become a major threat because millions of electronic devices interconnected via communication networks expose unprecedented vulnerabilities, thereby increasing the potential attack surface. This dissertation is aimed at developing novel game-theoretic and machine-learning techniques for addressing the reliability and security issues residing at multiple layers of the smart grid, including power distribution system reliability forecasting, risk assessment of cyber-physical attacks targeted at the grid, and cyber attack detection in the Advanced Metering Infrastructure (AMI) and renewable resources. This dissertation first comprehensively investigates the combined effect of various weather parameters on the reliability performance of the smart grid, and proposes a multilayer perceptron (MLP)-based framework to forecast the daily number of power interruptions in the distribution system using time series of common weather data. Regarding evaluating the risk of cyber-physical attacks faced by the smart grid, a stochastic budget allocation game is proposed to analyze the strategic interactions between a malicious attacker and the grid defender. A reinforcement learning algorithm is developed to enable the two players to reach a game equilibrium, where the optimal budget allocation strategies of the two players, in terms of attacking/protecting the critical elements of the grid, can be obtained. In addition, the risk of the cyber-physical attack can be derived based on the successful attack probability to various grid elements. Furthermore, this dissertation develops a multimodal data-driven framework for the cyber attack detection in the power distribution system integrated with renewable resources. This approach introduces the spare feature learning into an ensemble classifier for improving the detection efficiency, and implements the spatiotemporal correlation analysis for differentiating the attacked renewable energy measurements from fault scenarios. Numerical results based on the IEEE 34-bus system show that the proposed framework achieves the most accurate detection of cyber attacks reported in the literature. To address the electricity theft in the AMI, a Distributed Intelligent Framework for Electricity Theft Detection (DIFETD) is proposed, which is equipped with Benford’s analysis for initial diagnostics on large smart meter data. A Stackelberg game between utility and multiple electricity thieves is then formulated to model the electricity theft actions. Finally, a Likelihood Ratio Test (LRT) is utilized to detect potentially fraudulent meters

Cyber-Physical Power System (CPPS): A Review on Modelling, Simulation, and Analysis with Cyber Security Applications

Cyber-Physical System (CPS) is a new kind of digital technology that increases its attention across academia, government, and industry sectors and covers a wide range of applications like agriculture, energy, medical, transportation, etc. The traditional power systems with physical equipment as a core element are more integrated with information and communication technology, which evolves into the Cyber-Physical Power System (CPPS). The CPPS consists of a physical system tightly integrated with cyber systems (control, computing, and communication functions) and allows the two-way flows of electricity and information for enabling smart grid technologies. Even though the digital technologies monitoring and controlling the electric power grid more efficiently and reliably, the power grid is vulnerable to cybersecurity risk and involves the complex interdependency between cyber and physical systems. Analyzing and resolving the problems in CPPS needs the modelling methods and systematic investigation of a complex interaction between cyber and physical systems. The conventional way of modelling, simulation, and analysis involves the separation of physical domain and cyber domain, which is not suitable for the modern CPPS. Therefore, an integrated framework needed to analyze the practical scenario of the unification of physical and cyber systems. A comprehensive review of different modelling, simulation, and analysis methods and different types of cyber-attacks, cybersecurity measures for modern CPPS is explored in this paper. A review of different types of cyber-attack detection and mitigation control schemes for the practical power system is presented in this paper. The status of the research in CPPS around the world and a new path for recommendations and research directions for the researchers working in the CPPS are finally presented.publishedVersio

Reliability Evaluation and Defense Strategy Development for Cyber-physical Power Systems

With the smart grid initiatives in recent years, the electric power grid is rapidly evolving into a complicated and interconnected cyber-physical system. Unfortunately, the wide deployment of cutting-edge communication, control and computer technologies in the power system, as well as the increasing terrorism activities, make the power system at great risk of attacks from both cyber and physical domains. It is pressing and meaningful to investigate the plausible attack scenarios and develop efficient methods for defending the power system against them. To defend the power grid, it is critical to first study how the attacks could happen and affect the power system, which are the basis for the defense strategy development. Thus, this dissertation quantifies the influence of several typical attacks on power system reliability. Specifically, three representative attack are considered, i.e., intrusion against substations, regional LR attack, and coordinated attacks. For the intrusion against substations, the occurrence frequency of the attack events is modeled based on statistical data and human dynamics; game-theoretical approaches are adopted to model induvial and consecutive attack cases; Monte Carlo simulation is deployed to obtain the desired reliability indices, which incorporates both the attacks and the random failures. For the false data injection attack, a practical regional load redistribution (LR) attack strategy is proposed; the man-in-the-middle (MITM) intrusion process is modeled with a semi-Markov process method; the reliability indices are obtained based on the regional LR attack strategy and the MITM intrusion process using Monte Carlo simulation. For the coordinated attacks, a few typical coordination strategies are proposed considering attacking the current-carrying elements as well as attacking the measurements; a bilevel optimization method is applied to develop the optimal coordination strategy. Further, efficient and effective defense strategies are proposed from the perspectives of power system operation strategy and identification of critical elements. Specially, a robustness-oriented power grid operation strategy is proposed considering the element random failures and the risk of man-made attacks. Using this operation strategy, the power system operation is robust, and can minimize the load loss in case of malicious man-made attacks. Also, a multiple-attack-scenario (MAS) defender-attack-defender model is proposed to identify the critical branches that should be defended when an attack is anticipated but the defender has uncertainty about the capability of the attacker. If those identified critical branches are protected, the expected load loss will be minimal

Reinforcement Learning and Game Theory for Smart Grid Security

This dissertation focuses on one of the most critical and complicated challenges facing electric power transmission and distribution systems which is their vulnerability against failure and attacks. Large scale power outages in Australia (2016), Ukraine (2015), India (2013), Nigeria (2018), and the United States (2011, 2003) have demonstrated the vulnerability of power grids to cyber and physical attacks and failures. These incidents clearly indicate the necessity of extensive research efforts to protect the power system from external intrusion and to reduce the damages from post-attack effects. We analyze the vulnerability of smart power grids to cyber and physical attacks and failures, design different gametheoretic approaches to identify the critical components vulnerable to attack and propose their associated defense strategy, and utilizes machine learning techniques to solve the game-theoretic problems in adversarial and collaborative adversarial power grid environment. Our contributions can be divided into three major parts:Vulnerability identification: Power grid outages have disastrous impacts on almost every aspect of modern life. Despite their inevitability, the effects of failures on power grids’ performance can be limited if the system operator can predict and identify the vulnerable elements of power grids. To enable these capabilities we study machine learning algorithms to identify critical power system elements adopting a cascaded failure simulator as a threat and attack model. We use generation loss, time to reach a certain percentage of line outage/generation loss, number of line outages, etc. as evaluation metrics to evaluate the consequences of threat and attacks on the smart power grid.Adversarial gaming in power system: With the advancement of the technologies, the smart attackers are deploying different techniques to supersede the existing protection scheme. In order to defend the power grid from these smart attackers, we introduce an adversarial gaming environment using machine learning techniques which is capable of replicating the complex interaction between the attacker and the power system operators. The numerical results show that a learned defender successfully narrows down the attackers’ attack window and reduce damages. The results also show that considering some crucial factors, the players can independently execute actions without detailed information about each other.Deep learning for adversarial gaming: The learning and gaming techniques to identify vulnerable components in the power grid become computationally expensive for large scale power systems. The power system operator needs to have the advanced skills to deal with the large dimensionality of the problem. In order to aid the power system operator in finding and analyzing vulnerability for large scale power systems, we study a deep learning technique for adversary game which is capable of dealing with high dimensional power system state space with less computational time and increased computational efficiency. Overall, the results provided in this dissertation advance power grids’ resilience and security by providing a better understanding of the systems’ vulnerability and by developing efficient algorithms to identify vulnerable components and appropriate defensive strategies to reduce the damages of the attack

Understanding Deregulated Retail Electricity Markets in the Future: A Perspective from Machine Learning and Optimization

On top of Smart Grid technologies and new market mechanism design, the further deregulation of retail electricity market at distribution level will play a important role in promoting energy system transformation in a socioeconomic way. In today’s retail electricity market, customers have very limited ”energy choice,” or freedom to choose different types of energy services. Although the installation of distributed energy resources (DERs) has become prevalent in many regions, most customers and prosumers who have local energy generation and possible surplus can still only choose to trade with utility companies.They either purchase energy from or sell energy surplus back to the utilities directly while suffering from some price gap. The key to providing more energy trading freedom and open innovation in the retail electricity market is to develop new consumer-centric business models and possibly a localized energy trading platform. This dissertation is exactly pursuing these ideas and proposing a holistic localized electricity retail market to push the next-generation retail electricity market infrastructure to be a level playing field, where all customers have an equal opportunity to actively participate directly. This dissertation also studied and discussed opportunities of many emerging technologies, such as reinforcement learning and deep reinforcement learning, for intelligent energy system operation. Some improvement suggestion of the modeling framework and methodology are included as well.Ph.D.College of Engineering & Computer ScienceUniversity of Michigan-Dearbornhttps://deepblue.lib.umich.edu/bitstream/2027.42/145686/1/Tao Chen Final Dissertation.pdfDescription of Tao Chen Final Dissertation.pdf : Dissertatio

Real-Time Machine Learning Models To Detect Cyber And Physical Anomalies In Power Systems

A Smart Grid is a cyber-physical system (CPS) that tightly integrates computation and networking with physical processes to provide reliable two-way communication between electricity companies and customers. However, the grid availability and integrity are constantly threatened by both physical faults and cyber-attacks which may have a detrimental socio-economic impact. The frequency of the faults and attacks is increasing every year due to the extreme weather events and strong reliance on the open internet architecture that is vulnerable to cyber-attacks. In May 2021, for instance, Colonial Pipeline, one of the largest pipeline operators in the U.S., transports refined gasoline and jet fuel from Texas up the East Coast to New York was forced to shut down after being attacked by ransomware, causing prices to rise at gasoline pumps across the country. Enhancing situational awareness within the grid can alleviate these risks and avoid their adverse consequences. As part of this process, the phasor measurement units (PMU) are among the suitable assets since they collect time-synchronized measurements of grid status (30-120 samples/s), enabling the operators to react rapidly to potential anomalies. However, it is still challenging to process and analyze the open-ended source of PMU data as there are more than 2500 PMU distributed across the U.S. and Canada, where each of which generates more than 1.5 TB/month of streamed data. Further, the offline machine learning algorithms cannot be used in this scenario, as they require loading and scanning the entire dataset before processing. The ultimate objective of this dissertation is to develop early detection of cyber and physical anomalies in a real-time streaming environment setting by mining multi-variate large-scale synchrophasor data. To accomplish this objective, we start by investigating the cyber and physical anomalies, analyzing their impact, and critically reviewing the current detection approaches. Then, multiple machine learning models were designed to identify physical and cyber anomalies; the first one is an artificial neural network-based approach for detecting the False Data Injection (FDI) attack. This attack was specifically selected as it poses a serious risk to the integrity and availability of the grid; Secondly, we extend this approach by developing a Random Forest Regressor-based model which not only detects anomalies, but also identifies their location and duration; Lastly, we develop a real-time hoeffding tree-based model for detecting anomalies in steaming networks, and explicitly handling concept drifts. These models have been tested and the experimental results confirmed their superiority over the state-of-the-art models in terms of detection accuracy, false-positive rate, and processing time, making them potential candidates for strengthening the grid\u27s security

Bibliographical review on cyber attacks from a control oriented perspective

This paper presents a bibliographical review of definitions, classifications and applications concerning cyber attacks in networked control systems (NCSs) and cyber-physical systems (CPSs). This review tackles the topic from a control-oriented perspective, which is complementary to information or communication ones. After motivating the importance of developing new methods for attack detection and secure control, this review presents security objectives, attack modeling, and a characterization of considered attacks and threats presenting the detection mechanisms and remedial actions. In order to show the properties of each attack, as well as to provide some deeper insight into possible defense mechanisms, examples available in the literature are discussed. Finally, open research issues and paths are presented.Peer ReviewedPostprint (author's final draft

Secure authentication and key agreement via abstract multi-agent interaction

Authentication and key agreement are the foundation for secure communication over the Internet. Authenticated Key Exchange (AKE) protocols provide methods for communicating parties to authenticate each other, and establish a shared session key by which they can encrypt messages in the session. Within the category of AKE protocols, symmetric AKE protocols rely on pre-shared master keys for both services. These master keys can be transformed after each session in a key-evolving scheme to provide the property of forward secrecy, whereby the compromise of master keys does not allow for the compromise of past session keys. This thesis contributes a symmetric AKE protocol named AMI (Authentication via Multi-Agent Interaction). The AMI protocol is a novel formulation of authentication and key agreement as a multi-agent system, where communicating parties are treated as autonomous agents whose behavior within the protocol is governed by private agent models used as the master keys. Parties interact repeatedly using their behavioral models for authentication and for agreeing upon a unique session key per communication session. These models are evolved after each session to provide forward secrecy. The security of the multi-agent interaction process rests upon the difficulty of modeling an agent's decisions from limited observations about its behavior, a long-standing problem in AI research known as opponent modeling. We conjecture that it is difficult to efficiently solve even by a quantum computer, since the problem is fundamentally one of missing information rather than computational hardness. We show empirically that the AMI protocol achieves high accuracy in correctly identifying legitimate agents while rejecting different adversarial strategies from the security literature. We demonstrate the protocol's resistance to adversarial agents which utilize random, replay, and maximum-likelihood estimation (MLE) strategies to bypass the authentication test. The random strategy chooses actions randomly without attempting to mimic a legitimate agent. The replay strategy replays actions previously observed by a legitimate client. The MLE strategy estimates a legitimate agent model using previously observed interactions, as an attempt to solve the opponent modeling problem. This thesis also introduces a reinforcement learning approach for efficient multi-agent interaction and authentication. This method trains an authenticating server agent's decision model to take effective probing actions which decrease the number of interactions in a single session required to successfully reject adversarial agents. We empirically evaluate the number of interactions required for a trained server agent to reject an adversarial agent, and show that using the optimized server leads to a much more sample-efficient interaction process than a server agent selecting actions by a uniform-random behavioral policy. Towards further research on and adoption of the AMI protocol for authenticated key-exchange, this thesis also contributes an open-source application written in Python, PyAMI. PyAMI consists of a multi-agent system where agents run on separate virtual machines, and communicate over low-level network sockets using TCP. The application supports extending the basic client-server setting to a larger multi-agent system for group authentication and key agreement, providing two such architectures for different deployment scenarios