Intelligent Novel Methods for Identifying Critical Components and Their Combinations for Hypothesized Cyber-physical Attacks Against Electric Power Grids

As a revolutionary change to the traditional power grid, the smart grid is expected to introduce a myriad of noteworthy benefits by integrating the advanced information and communication technologies in terms of system costs, reliability, environmental impacts, operational flexibility, etc. However, the wider deployment of cyber networks in the power grid will bring about important issues on power system cyber security. Meanwhile, the power grid is becoming more vulnerable to various physical attacks due to vandalism and probable terrorist attacks. In an envisioned smart grid environment, attackers have more entry points to various parts of the power grid for launching a well-planned and highly destructive attack in a coordinated manner. Thus, it is important to address the smart grid cyber-physical security issues in order to strengthen the robustness and resiliency of the smart grid in the face of various adverse events. One key step of this research topic is to efficiently identify the vulnerable parts of the smart grid. In this thesis, from the perspective of smart grid cyber-physical security, three critical component combination identification methods are proposed to reveal the potential vulnerability of the smart grid. First, two performance indices based critical component combination recognition methods are proposed for more effectively identifying the critical component combinations in the multi-component attack scenarios. The optimal selection of critical components is determined according to the criticality of the components, which can be modeled by various performance indices. Further, the space-pruning based enumerative search strategy is investigated to comprehensively and effectively identify critical combinations of multiple same or different types of components. The pruned search space is generated based on the criticality of potential target component which is obtained from low-order enumeration data. Specifically, the combinatorial line-generator attack strategy is investigated by exploring the strategy for attacking multiple different types of components. Finally, an effective, novel approach is proposed for identifying critical component combinations, which is termed search space conversion and reduction strategy based intelligent search method (SCRIS). The conversion and reduction of the search space is achieved based on the criticality of the components which is obtained from an efficient sampling method. The classic intelligent search algorithm, Particle Swarm Optimization (PSO), is improved and deployed for more effectively identifying critical component combinations. MATLAB is used as the simulation platform in this study. The IEEE 30, 39, 118 and Polish 2383-bus systems are adopted for verifying the effectiveness of the proposed attack strategies. According to the simulation results, the proposed attack strategies turn out to be effective and computationally efficient. This thesis can provide some useful insight into vulnerability identification in a smart grid environment, and defensive strategies can be developed in view of this work to prevent malicious coordinated multi-component attacks which may initiate cascading failures in a cyber-physical environment

Reinforcement learning for efficient network penetration testing

Penetration testing (also known as pentesting or PT) is a common practice for actively assessing the defenses of a computer network by planning and executing all possible attacks to discover and exploit existing vulnerabilities. Current penetration testing methods are increasingly becoming non-standard, composite and resource-consuming despite the use of evolving tools. In this paper, we propose and evaluate an AI-based pentesting system which makes use of machine learning techniques, namely reinforcement learning (RL) to learn and reproduce average and complex pentesting activities. The proposed system is named Intelligent Automated Penetration Testing System (IAPTS) consisting of a module that integrates with industrial PT frameworks to enable them to capture information, learn from experience, and reproduce tests in future similar testing cases. IAPTS aims to save human resources while producing much-enhanced results in terms of time consumption, reliability and frequency of testing. IAPTS takes the approach of modeling PT environments and tasks as a partially observed Markov decision process (POMDP) problem which is solved by POMDP-solver. Although the scope of this paper is limited to network infrastructures PT planning and not the entire practice, the obtained results support the hypothesis that RL can enhance PT beyond the capabilities of any human PT expert in terms of time consumed, covered attacking vectors, accuracy and reliability of the outputs. In addition, this work tackles the complex problem of expertise capturing and re-use by allowing the IAPTS learning module to store and re-use PT policies in the same way that a human PT expert would learn but in a more efficient way

A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work

A Review of the Family of Artificial Fish Swarm Algorithms: Recent Advances and Applications

The Artificial Fish Swarm Algorithm (AFSA) is inspired by the ecological behaviors of fish schooling in nature, viz., the preying, swarming, following and random behaviors. Owing to a number of salient properties, which include flexibility, fast convergence, and insensitivity to the initial parameter settings, the family of AFSA has emerged as an effective Swarm Intelligence (SI) methodology that has been widely applied to solve real-world optimization problems. Since its introduction in 2002, many improved and hybrid AFSA models have been developed to tackle continuous, binary, and combinatorial optimization problems. This paper aims to present a concise review of the family of AFSA, encompassing the original ASFA and its improvements, continuous, binary, discrete, and hybrid models, as well as the associated applications. A comprehensive survey on the AFSA from its introduction to 2012 can be found in [1]. As such, we focus on a total of {\color{blue}123} articles published in high-quality journals since 2013. We also discuss possible AFSA enhancements and highlight future research directions for the family of AFSA-based models.Comment: 37 pages, 3 figure

Automatic programming methodologies for electronic hardware fault monitoring

This paper presents three variants of Genetic Programming (GP) approaches for intelligent online performance monitoring of electronic circuits and systems. Reliability modeling of electronic circuits can be best performed by the Stressor - susceptibility interaction model. A circuit or a system is considered to be failed once the stressor has exceeded the susceptibility limits. For on-line prediction, validated stressor vectors may be obtained by direct measurements or sensors, which after pre-processing and standardization are fed into the GP models. Empirical results are compared with artificial neural networks trained using backpropagation algorithm and classification and regression trees. The performance of the proposed method is evaluated by comparing the experiment results with the actual failure model values. The developed model reveals that GP could play an important role for future fault monitoring systems.This research was supported by the International Joint Research Grant of the IITA (Institute of Information Technology Assessment) foreign professor invitation program of the MIC (Ministry of Information and Communication), Korea