Reinforcement Learning Based Penetration Testing of a Microgrid Control Algorithm

Microgrids (MGs) are small-scale power systems which interconnect distributed energy resources and loads within clearly defined regions. However, the digital infrastructure used in an MG to relay sensory information and perform control commands can potentially be compromised due to a cyberattack from a capable adversary. An MG operator is interested in knowing the inherent vulnerabilities in their system and should regularly perform Penetration Testing (PT) activities to prepare for such an event. PT generally involves looking for defensive coverage blindspots in software and hardware infrastructure, however the logic in control algorithms which act upon sensory information should also be considered in PT activities. This paper demonstrates a case study of PT for an MG control algorithm by using Reinforcement Learning (RL) to uncover malicious input which compromises the effectiveness of the controller. Through trial-and-error episodic interactions with a simulated MG, we train an RL agent to find malicious input which reduces the effectiveness of the MG controller

IoT-Based Cyber-Physical Communication Architecture: Challenges and Research Directions

In order to provide intelligent services, the Internet of Things (IoT) facilitates millions of smart cyber-physical devices to be enabled with network connectivity to sense, collect, process, and exchange information. Unfortunately, the traditional communication infrastructure is vulnerable to cyber attacks and link failures, so it is a challenging task for the IoT to explore these applications. In order to begin research and contribute into the IoT-based cyber-physical digital world, one will need to know the technical challenges and research opportunities. In this study, several key technical challenges and requirements for the IoT communication systems are identified. Basically, privacy, security, intelligent sensors/actuators design, low cost and complexity, universal antenna design, and friendly smart cyber-physical system design are the main challenges for the IoT implementation. Finally, the authors present a diverse set of cyber-physical communication system challenges such as practical implementation, distributed state estimation, real-time data collection, and system identification, which are the major issues require to be addressed in implementing an efficient and effective IoT communication system

Resilient Consensus Control Design for DC Microgrids against False Data Injection Attacks Using a Distributed Bank of Sliding Mode Observers

This paper investigates the problem of false data injection attack (FDIA) detection in microgrids. The grid under study is a DC microgrid with distributed boost converters, where the false data are injected into the voltage data so as to investigate the effect of attacks. The proposed algorithm uses a bank of sliding mode observers that estimates the states of the neighbor agents. Each agent estimates the neighboring states and, according to the estimation and communication data, the detection mechanism reveals the presence of FDIA. The proposed control scheme provides resiliency to the system by replacing the conventional consensus rule with attack-resilient ones. In order to evaluate the efficiency of the proposed method, a real-time simulation with eight agents has been performed. Moreover, a verification experimental test with three boost converters has been utilized to confirm the simulation results. It is shown that the proposed algorithm is able to detect FDI attacks and it protects the consensus deviation against FDI attacks

Resilient Consensus Control Design for DC Microgrids against False Data Injection Attacks Using a Distributed Bank of Sliding Mode Observers

This paper investigates the problem of false data injection attack (FDIA) detection in microgrids. The grid under study is a DC microgrid with distributed boost converters, where the false data are injected into the voltage data so as to investigate the effect of attacks. The proposed algorithm uses a bank of sliding mode observers that estimates the states of the neighbor agents. Each agent estimates the neighboring states and, according to the estimation and communication data, the detection mechanism reveals the presence of FDIA. The proposed control scheme provides resiliency to the system by replacing the conventional consensus rule with attack-resilient ones. In order to evaluate the efficiency of the proposed method, a real-time simulation with eight agents has been performed. Moreover, a verification experimental test with three boost converters has been utilized to confirm the simulation results. It is shown that the proposed algorithm is able to detect FDI attacks and it protects the consensus deviation against FDI attacks

Smart grid state estimation and its applications to grid stabilization

University of Technology Sydney. Faculty of Engineering and Information Technology.The smart grid is expected to modernize the current electricity grid by commencing a new set of technologies and services that can make the electricity networks more secure, automated, cooperative and sustainable. The smart grid can integrate multiple distributed energy resources (DERs) into the main grid. The need for DERs is expected to become more important in the future smart grid due to the global warming and energy problems. Basically, the smart grid can spread the intelligence of the energy distribution and control system from the central unit to long-distance remote areas, thus enabling accurate state estimation and wide-area real-time monitoring of these intermittent energy sources. Reliable state estimation is a key technique to fulfil the control requirement and hence is an enabler for the automation of power grids. Driven by these motivations, this research explores the problem of state estimation and stabilization taking disturbances, cyber attacks and packet losses into consideration for the smart grid. The first contribution of this dissertation is to develop a least square based Kalman filter (KF) algorithm for state estimation, and an optimal feedback control framework for stabilizing the microgrid states. To begin with, the environment-friendly renewable microgrid incorporating multiple DERs is modelled to obtain discrete-time state-space linear equations where sensors are deployed to obtain system state information. The proposed smart grid communication system provides an opportunity to address the state regulation challenge by offering two-way communication links for microgrid information collection, estimation and stabilization. Interestingly, the developed least square based centralised KF algorithm is able to estimate the system states properly even at the beginning of the dynamic process, and the proposed H2 based optimal feedback controller is able to stabilize the microgrid states in a fairly short time. Unfortunately, the smart grid is susceptible to malicious cyber attacks, which can create serious technical, economic, social and control problems in power network operations. In contrast to the traditional cyber attack minimization techniques, this study proposes a recursive systematic convolutional (RSC) code and KF based method in the context of smart grids. The proposed RSC code is used to add redundancy in the microgrid states, and the log maximum a-posterior is used to recover the state information which is affected by random noises and cyber attacks. Once the estimated states are obtained, a semidefinite programming (SDP) based optimal feedback controller is proposed to regulate the system states. Test results show that the proposed approach can accurately mitigate the cyber attacks and properly estimate as well as regulate the system states. The other significant contribution of this dissertation is to develop an adaptive-then-combine distributed dynamic approach for monitoring the grid under lossy communication links between wind turbines and the energy management system. Based on the mean squared error principle, an adaptive approach is proposed to estimate the local state information. The global estimation is designed by combining local estimation results with weighting factors, which are calculated by minimizing the estimation error covariances based on SDP. Afterwards, the convergence analysis indicates that the estimation error is gradually decreased, so the estimated state converges to the actual state. The efficacy of the developed approach is verified using the wind turbine and IEEE 6-bus distribution system. Furthermore, the distribution power sub-systems are usually interconnected to each other, so this research investigates the interconnected optimal filtering problem for distributed dynamic state estimation considering packet losses. The optimal local and neighbouring gains are computed to reach a consensus estimation after exchanging their information with the neighbouring estimators. Then the convergence of the developed algorithm is theoretically proved. Afterwards, a distributed controller is designed based on the SDP approach. Simulation results demonstrate the accuracy of the developed approaches. The penultimate contribution of this dissertation is to develop a distributed state estimation algorithm for interconnected power systems that only needs a consensus step. After modelling the interconnected synchronous generators, the optimal gain is determined to obtain a distributed state estimation. The consensus of the developed approach is proved based on the Lyapunov theory. From the circuit and system point of view, the proposed framework is useful for designing a practical energy management system as it has less computational complexity and provides accurate estimation results. The distributed state estimation algorithm is further modified by considering different observation matrices with both local and consensus steps. The optimal local gain is computed after minimizing the mean squared error between the true and estimated states. The consensus gain is determined by a convex optimization process with a given local gain. Moreover, the convergence of the proposed scheme is analysed after stacking all the estimation error dynamics. The efficacy of the developed approach is demonstrated using the environment-friendly renewable microgrid and IEEE 30-bus power system. Overall, the findings, theoretical development and analysis of this research represent a comprehensive source of information for smart grid state estimation and stabilization schemes, and will shed light on green smart energy management systems and monitoring centre design in future smart grid implementations. It is worth pointing out that the aforementioned contributions are very important in the smart grid community as communication impairments have a significant impact on grid stability and the distributed strategies can reduce communication burden and offer a sparse communication network

State Estimation Fusion for Linear Microgrids over an Unreliable Network

Microgrids should be continuously monitored in order to maintain suitable voltages over time. Microgrids are mainly monitored remotely, and their measurement data transmitted through lossy communication networks are vulnerable to cyberattacks and packet loss. The current study leverages the idea of data fusion to address this problem. Hence, this paper investigates the effects of estimation fusion using various machine-learning (ML) regression methods as data fusion methods by aggregating the distributed Kalman filter (KF)-based state estimates of a linear smart microgrid in order to achieve more accurate and reliable state estimates. This unreliability in measurements is because they are received through a lossy communication network that incorporates packet loss and cyberattacks. In addition to ML regression methods, multi-layer perceptron (MLP) and dependent ordered weighted averaging (DOWA) operators are also employed for further comparisons. The results of simulation on the IEEE 4-bus model validate the effectiveness of the employed ML regression methods through the RMSE, MAE and R-squared indices under the condition of missing and manipulated measurements. In general, the results obtained by the Random Forest regression method were more accurate than those of other methods.This research was partially funded by public research projects of Spanish Ministry of Science and Innovation, references PID2020-118249RB-C22 and PDC2021-121567-C22 - AEI/10.13039/ 501100011033, and by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors, reference EPUC3M17