Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Comprehensive Survey and Taxonomies of False Injection Attacks in Smart Grid: Attack Models, Targets, and Impacts

Smart Grid has rapidly transformed the centrally controlled power system into a massively interconnected cyber-physical system that benefits from the revolutions happening in the communications (e.g. 5G) and the growing proliferation of the Internet of Things devices (such as smart metres and intelligent electronic devices). While the convergence of a significant number of cyber-physical elements has enabled the Smart Grid to be far more efficient and competitive in addressing the growing global energy challenges, it has also introduced a large number of vulnerabilities culminating in violations of data availability, integrity, and confidentiality. Recently, false data injection (FDI) has become one of the most critical cyberattacks, and appears to be a focal point of interest for both research and industry. To this end, this paper presents a comprehensive review in the recent advances of the FDI attacks, with particular emphasis on 1) adversarial models, 2) attack targets, and 3) impacts in the Smart Grid infrastructure. This review paper aims to provide a thorough understanding of the incumbent threats affecting the entire spectrum of the Smart Grid. Related literature are analysed and compared in terms of their theoretical and practical implications to the Smart Grid cybersecurity. In conclusion, a range of technical limitations of existing false data attack research is identified, and a number of future research directions is recommended.Comment: Double-column of 24 pages, prepared based on IEEE Transaction articl

Application of Game Theory to Improve the Defense of the Smart Grid

This thesis presents the development and evaluation of a distributed agent based system using reputation based trust and game theoretic techniques to improve the defense of the future smart grid from cyber-attack and equipment malfunctions. Future smart grid capabilities promise to leverage network technologies to revolutionize the production, transmission, distribution and consumption of electrical power. However, the internet like communication also increase the power grid\u27s vulnerability to cyber-attack. This thesis uses simulation linking power systems with communication networks to demonstrate the benefits of a Distributed Decision Making Communication Enable Special Protection System (SPS) using reputation based trust and game theory to protect the power grid from malicious and non-malicious malfunctions. The simulations show that a distributed approach to SPS load shedding successfully maintains power grid stability after an electrical disturbance while using reputation based trust to defend the load shedding from cyber-attack and equipment malfunction. Additional simulations demonstrate the application of game theory to defend the SPS load shedding process when available resources prevent the monitoring and defense of every part of the power grid. The demonstrated capability increases the resiliency of the power grid by preventing uncontrolled blackouts through detection and mitigation of attacks, improving the system\u27s reliability

Reliability Evaluation and Defense Strategy Development for Cyber-physical Power Systems

With the smart grid initiatives in recent years, the electric power grid is rapidly evolving into a complicated and interconnected cyber-physical system. Unfortunately, the wide deployment of cutting-edge communication, control and computer technologies in the power system, as well as the increasing terrorism activities, make the power system at great risk of attacks from both cyber and physical domains. It is pressing and meaningful to investigate the plausible attack scenarios and develop efficient methods for defending the power system against them. To defend the power grid, it is critical to first study how the attacks could happen and affect the power system, which are the basis for the defense strategy development. Thus, this dissertation quantifies the influence of several typical attacks on power system reliability. Specifically, three representative attack are considered, i.e., intrusion against substations, regional LR attack, and coordinated attacks. For the intrusion against substations, the occurrence frequency of the attack events is modeled based on statistical data and human dynamics; game-theoretical approaches are adopted to model induvial and consecutive attack cases; Monte Carlo simulation is deployed to obtain the desired reliability indices, which incorporates both the attacks and the random failures. For the false data injection attack, a practical regional load redistribution (LR) attack strategy is proposed; the man-in-the-middle (MITM) intrusion process is modeled with a semi-Markov process method; the reliability indices are obtained based on the regional LR attack strategy and the MITM intrusion process using Monte Carlo simulation. For the coordinated attacks, a few typical coordination strategies are proposed considering attacking the current-carrying elements as well as attacking the measurements; a bilevel optimization method is applied to develop the optimal coordination strategy. Further, efficient and effective defense strategies are proposed from the perspectives of power system operation strategy and identification of critical elements. Specially, a robustness-oriented power grid operation strategy is proposed considering the element random failures and the risk of man-made attacks. Using this operation strategy, the power system operation is robust, and can minimize the load loss in case of malicious man-made attacks. Also, a multiple-attack-scenario (MAS) defender-attack-defender model is proposed to identify the critical branches that should be defended when an attack is anticipated but the defender has uncertainty about the capability of the attacker. If those identified critical branches are protected, the expected load loss will be minimal

Cyber Defense Remediation in Energy Delivery Systems

The integration of Information Technology (IT) and Operational Technology (OT) in Cyber-Physical Systems (CPS) has resulted in increased efficiency and facilitated real-time information acquisition, processing, and decision making. However, the increase in automation technology and the use of the internet for connecting, remote controlling, and supervising systems and facilities has also increased the likelihood of cybersecurity threats that can impact safety of humans and property. There is a need to assess cybersecurity risks in the power grid, nuclear plants, chemical factories, etc. to gain insight into the likelihood of safety hazards. Quantitative cybersecurity risk assessment will lead to informed cyber defense remediation and will ensure the presence of a mitigation plan to prevent safety hazards. In this dissertation, using Energy Delivery Systems (EDS) as a use case to contextualize a CPS, we address key research challenges in managing cyber risk for cyber defense remediation. First, we developed a platform for modeling and analyzing the effect of cyber threats and random system faults on EDS\u27s safety that could lead to catastrophic damages. We developed a data-driven attack graph and fault graph-based model to characterize the exploitability and impact of threats in EDS. We created an operational impact assessment to quantify the damages. Finally, we developed a strategic response decision capability that presents optimal mitigation actions and policies that balance the tradeoff between operational resilience (tactical risk) and strategic risk. Next, we addressed the challenge of management of tactical risk based on a prioritized cyber defense remediation plan. A prioritized cyber defense remediation plan is critical for effective risk management in EDS. Due to EDS\u27s complexity in terms of the heterogeneous nature of blending IT and OT and Industrial Control System (ICS), scale, and critical processes tasks, prioritized remediation should be applied gradually to protect critical assets. We proposed a methodology for prioritizing cyber risk remediation plans by detecting and evaluating critical EDS nodes\u27 paths. We conducted evaluation of critical nodes characteristics based on nodes\u27 architectural positions, measure of centrality based on nodes\u27 connectivity and frequency of network traffic, as well as the controlled amount of electrical power. The model also examines the relationship between cost models of budget allocation for removing vulnerabilities on critical nodes and their impact on gradual readiness. The proposed cost models were empirically validated in an existing network ICS test-bed computing nodes criticality. Two cost models were examined, and although varied, we concluded the lack of correlation between types of cost models to most damageable attack path and critical nodes readiness. Finally, we proposed a time-varying dynamical model for the cyber defense remediation in EDS. We utilize the stochastic evolutionary game model to simulate the dynamic adversary of cyber-attack-defense. We leveraged the Logit Quantal Response Dynamics (LQRD) model to quantify real-world players\u27 cognitive differences. We proposed the optimal decision making approach by calculating the stable evolutionary equilibrium and balancing defense costs and benefits. Case studies on EDS indicate that the proposed method can help the defender predict possible attack action, select the related optimal defense strategy over time, and gain the maximum defense payoffs. We also leveraged software-defined networking (SDN) in EDS for dynamical cyber defense remediation. We presented an approach to aid the selection security controls dynamically in an SDN-enabled EDS and achieve tradeoffs between providing security and Quality of Service (QoS). We modeled the security costs based on end-to-end packet delay and throughput. We proposed a non-dominated sorting based multi-objective optimization framework which can be implemented within an SDN controller to address the joint problem of optimizing between security and QoS parameters by alleviating time complexity at O(MN2). The M is the number of objective functions, and N is the population for each generation, respectively. We presented simulation results that illustrate how data availability and data integrity can be achieved while maintaining QoS constraints