Reinforcement Learning and Game Theory for Smart Grid Security

This dissertation focuses on one of the most critical and complicated challenges facing electric power transmission and distribution systems which is their vulnerability against failure and attacks. Large scale power outages in Australia (2016), Ukraine (2015), India (2013), Nigeria (2018), and the United States (2011, 2003) have demonstrated the vulnerability of power grids to cyber and physical attacks and failures. These incidents clearly indicate the necessity of extensive research efforts to protect the power system from external intrusion and to reduce the damages from post-attack effects. We analyze the vulnerability of smart power grids to cyber and physical attacks and failures, design different gametheoretic approaches to identify the critical components vulnerable to attack and propose their associated defense strategy, and utilizes machine learning techniques to solve the game-theoretic problems in adversarial and collaborative adversarial power grid environment. Our contributions can be divided into three major parts:Vulnerability identification: Power grid outages have disastrous impacts on almost every aspect of modern life. Despite their inevitability, the effects of failures on power grids’ performance can be limited if the system operator can predict and identify the vulnerable elements of power grids. To enable these capabilities we study machine learning algorithms to identify critical power system elements adopting a cascaded failure simulator as a threat and attack model. We use generation loss, time to reach a certain percentage of line outage/generation loss, number of line outages, etc. as evaluation metrics to evaluate the consequences of threat and attacks on the smart power grid.Adversarial gaming in power system: With the advancement of the technologies, the smart attackers are deploying different techniques to supersede the existing protection scheme. In order to defend the power grid from these smart attackers, we introduce an adversarial gaming environment using machine learning techniques which is capable of replicating the complex interaction between the attacker and the power system operators. The numerical results show that a learned defender successfully narrows down the attackers’ attack window and reduce damages. The results also show that considering some crucial factors, the players can independently execute actions without detailed information about each other.Deep learning for adversarial gaming: The learning and gaming techniques to identify vulnerable components in the power grid become computationally expensive for large scale power systems. The power system operator needs to have the advanced skills to deal with the large dimensionality of the problem. In order to aid the power system operator in finding and analyzing vulnerability for large scale power systems, we study a deep learning technique for adversary game which is capable of dealing with high dimensional power system state space with less computational time and increased computational efficiency. Overall, the results provided in this dissertation advance power grids’ resilience and security by providing a better understanding of the systems’ vulnerability and by developing efficient algorithms to identify vulnerable components and appropriate defensive strategies to reduce the damages of the attack

BSCSML: Design of an Efficient Bioinspired Security &Privacy Model for Cyber Physical System using Machine Learning

With the increasing prevalence of Smart Grid Cyber Physical Systems with Advanced Metering Infrastructure (SG CPS AMI), securing their internal components has become one of the paramount concerns. Traditional security mechanisms have proven to be insufficient in defending against sophisticated attacks. Bioinspired security and privacy models have emerged as promising solutions due to their stochastic solutions. This paper proposes a novel bio-inspired security and privacy model for SG CPS AMI that utilizes machine learning to strengthen their security levels. The proposed model is inspired by the hybrid Grey Wolf Teacher Learner based Optimizer (GWTLbO) Method’s ability to detect and respond to threats in real-time deployments. The GWTLbO Model also ensures higher privacy by selecting optimal methods between k-privacy, t-closeness & l-diversity depending upon contextual requirements. This study improves system accuracy and efficiency under diverse attacks using machine learning techniques. The method uses supervised learning to teach the model to recognize known attack trends and uncontrolled learning to spot unknown attacks. Our model was tested using real-time IoT device data samples. The model identified Zero-Day Attacks, Meter Bypass, Flash Image Manipulation, and Buffer-level attacks. The proposed model detects and responds to attacks with high accuracy and low false-positive rates. In real-time operations, the proposed model can handle huge volumes of data efficiently. The bioinspired security and privacy model secures CPS efficiently and is scalable for various cases. Machine learning techniques can improve the security and secrecy of these systems and revolutionize defense against different attacks

Risk analysis beyond vulnerability and resilience - characterizing the defensibility of critical systems

A common problem in risk analysis is to characterize the overall security of a system of valuable assets (e.g., government buildings or communication hubs), and to suggest measures to mitigate any hazards or security threats. Currently, analysts typically rely on a combination of indices, such as resilience, robustness, redundancy, security, and vulnerability. However, these indices are not by themselves sufficient as a guide to action; for example, while it is possible to develop policies to decrease vulnerability, such policies may not always be cost-effective. Motivated by this gap, we propose a new index, defensibility. A system is considered defensible to the extent that a modest investment can significantly reduce the damage from an attack or disruption. To compare systems whose performance is not readily commensurable (e.g., the electrical grid vs. the water-distribution network, both of which are critical, but which provide distinct types of services), we defined defensibility as a dimensionless index. After defining defensibility quantitatively, we illustrate how the defensibility of a system depends on factors such as the defender and attacker asset valuations, the nature of the threat (whether intelligent and adaptive, or random), and the levels of attack and defense strengths and provide analytical results that support the observations arising from the above illustrations. Overall, we argue that the defensibility of a system is an important dimension to consider when evaluating potential defensive investments, and that it can be applied in a variety of different contexts.Comment: 36 pages; Keywords: Risk Analysis, Defensibility, Vulnerability, Resilience, Counter-terroris

An Energy Aware and Secure MAC Protocol for Tackling Denial of Sleep Attacks in Wireless Sensor Networks

Wireless sensor networks which form part of the core for the Internet of Things consist of resource constrained sensors that are usually powered by batteries. Therefore, careful energy awareness is essential when working with these devices. Indeed,the introduction of security techniques such as authentication and encryption, to ensure confidentiality and integrity of data, can place higher energy load on the sensors. However, the absence of security protection c ould give room for energy drain attacks such as denial of sleep attacks which have a higher negative impact on the life span ( of the sensors than the presence of security features. This thesis, therefore, focuses on tackling denial of sleep attacks from two perspectives A security perspective and an energy efficiency perspective. The security perspective involves evaluating and ranking a number of security based techniques to curbing denial of sleep attacks. The energy efficiency perspective, on the other hand, involves exploring duty cycling and simulating three Media Access Control ( protocols Sensor MAC, Timeout MAC andTunableMAC under different network sizes and measuring different parameters such as the Received Signal Strength RSSI) and Link Quality Indicator ( Transmit power, throughput and energy efficiency Duty cycling happens to be one of the major techniques for conserving energy in wireless sensor networks and this research aims to answer questions with regards to the effect of duty cycles on the energy efficiency as well as the throughput of three duty cycle protocols Sensor MAC ( Timeout MAC ( and TunableMAC in addition to creating a novel MAC protocol that is also more resilient to denial of sleep a ttacks than existing protocols. The main contributions to knowledge from this thesis are the developed framework used for evaluation of existing denial of sleep attack solutions and the algorithms which fuel the other contribution to knowledge a newly developed protocol tested on the Castalia Simulator on the OMNET++ platform. The new protocol has been compared with existing protocols and has been found to have significant improvement in energy efficiency and also better resilience to denial of sleep at tacks Part of this research has been published Two conference publications in IEEE Explore and one workshop paper

On the Use of Reinforcement Learning for Attacking and Defending Load Frequency Control

The electric grid is an attractive target for cyberattackers given its critical nature in society. With the increasing sophistication of cyberattacks, effective grid defense will benefit from proactively identifying vulnerabilities and attack strategies. We develop a deep reinforcement learning-based method that recognizes vulnerabilities in load frequency control, an essential process that maintains grid security and reliability. We demonstrate how our method can synthesize a variety of attacks involving false data injection and load switching, while specifying the attack and threat models - providing insight into potential attack strategies and impact. We discuss how our approach can be employed for testing electric grid vulnerabilities. Moreover our method can be employed to generate data to inform the design of defense strategies and develop attack detection methods. For this, we design and compare a (deep learning-based) supervised attack detector with an unsupervised anomaly detector to highlight the benefits of developing defense strategies based on identified attack strategies