Achieving High Renewable Energy Integration in Smart Grids with Machine Learning

The integration of high levels of renewable energy into smart grids is crucial for achieving a sustainable and efficient energy infrastructure. However, this integration presents significant technical and operational challenges due to the intermittent nature and inherent uncertainty of renewable energy sources (RES). Therefore, the energy storage system (ESS) has always been bound to renewable energy, and its charge and discharge control has become an important part of the integration. The addition of RES and ESS comes with their complex control, communication, and monitor capabilities, which also makes the grid more vulnerable to attacks, brings new challenges to the cybersecurity. A large number of works have been devoted to the optimization integration of the RES and ESS system to the traditional grid, along with combining the ESS scheduling control with the traditional Optimal Power Flow (OPF) control. Cybersecurity problem focusing on the RES integrated grid has also gradually aroused researchers’ interest. In recent years, machine learning techniques have emerged in different research field including optimizing renewable energy integration in smart grids. Reinforcement learning (RL), which trains agent to interact with the environment by making sequential decisions to maximize the expected future reward, is used as an optimization tool. This dissertation explores the application of RL algorithms and models to achieve high renewable energy integration in smart grids. The research questions focus on the effectiveness, benefits of renewable energy integration to individual consumers and electricity utilities, applying machine learning techniques in optimizing the behaviors of the ESS and the generators and other components in the grid. The objectives of this research are to investigate the current algorithms of renewable energy integration in smart grids, explore RL algorithms, develop novel RL-based models and algorithms for optimization control and cybersecurity, evaluate their performance through simulations on real-world data set, and provide practical recommendations for implementation. The research approach includes a comprehensive literature review to understand the challenges and opportunities associated with renewable energy integration. Various optimization algorithms, such as linear programming (LP), dynamic programming (DP) and various RL algorithms, such as Deep Q-Learning (DQN) and Deep Deterministic Policy Gradient (DDPG), are applied to solve problems during renewable energy integration in smart grids. Simulation studies on real-world data, including different types of loads, solar and wind energy profiles, are used to evaluate the performance and effectiveness of the proposed machine learning techniques. The results provide insights into the capabilities and limitations of machine learning in solving the optimization problems in the power system. Compared with traditional optimization tools, the RL approach has the advantage of real-time implementation, with the cost being the training time and unguaranteed model performance. Recommendations and guidelines for practical implementation of RL algorithms on power systems are provided in the appendix

Achieving High Renewable Energy Integration in Smart Grids with Machine Learning

The integration of high levels of renewable energy into smart grids is crucial for achieving a sustainable and efficient energy infrastructure. However, this integration presents significant technical and operational challenges due to the intermittent nature and inherent uncertainty of renewable energy sources (RES). Therefore, the energy storage system (ESS) has always been bound to renewable energy, and its charge and discharge control has become an important part of the integration. The addition of RES and ESS comes with their complex control, communication, and monitor capabilities, which also makes the grid more vulnerable to attacks, brings new challenges to the cybersecurity. A large number of works have been devoted to the optimization integration of the RES and ESS system to the traditional grid, along with combining the ESS scheduling control with the traditional Optimal Power Flow (OPF) control. Cybersecurity problem focusing on the RES integrated grid has also gradually aroused researchers’ interest. In recent years, machine learning techniques have emerged in different research field including optimizing renewable energy integration in smart grids. Reinforcement learning (RL), which trains agent to interact with the environment by making sequential decisions to maximize the expected future reward, is used as an optimization tool. This dissertation explores the application of RL algorithms and models to achieve high renewable energy integration in smart grids. The research questions focus on the effectiveness, benefits of renewable energy integration to individual consumers and electricity utilities, applying machine learning techniques in optimizing the behaviors of the ESS and the generators and other components in the grid. The objectives of this research are to investigate the current algorithms of renewable energy integration in smart grids, explore RL algorithms, develop novel RL-based models and algorithms for optimization control and cybersecurity, evaluate their performance through simulations on real-world data set, and provide practical recommendations for implementation. The research approach includes a comprehensive literature review to understand the challenges and opportunities associated with renewable energy integration. Various optimization algorithms, such as linear programming (LP), dynamic programming (DP) and various RL algorithms, such as Deep Q-Learning (DQN) and Deep Deterministic Policy Gradient (DDPG), are applied to solve problems during renewable energy integration in smart grids. Simulation studies on real-world data, including different types of loads, solar and wind energy profiles, are used to evaluate the performance and effectiveness of the proposed machine learning techniques. The results provide insights into the capabilities and limitations of machine learning in solving the optimization problems in the power system. Compared with traditional optimization tools, the RL approach has the advantage of real-time implementation, with the cost being the training time and unguaranteed model performance. Recommendations and guidelines for practical implementation of RL algorithms on power systems are provided in the appendix

State of the art of cyber-physical systems security: An automatic control perspective

Cyber-physical systems are integrations of computation, networking, and physical processes. Due to the tight cyber-physical coupling and to the potentially disrupting consequences of failures, security here is one of the primary concerns. Our systematic mapping study sheds light on how security is actually addressed when dealing with cyber-physical systems from an automatic control perspective. The provided map of 138 selected studies is defined empirically and is based on, for instance, application fields, various system components, related algorithms and models, attacks characteristics and defense strategies. It presents a powerful comparison framework for existing and future research on this hot topic, important for both industry and academia

Secure Data Management and Transmission Infrastructure for the Future Smart Grid

Power grid has played a crucial role since its inception in the Industrial Age. It has evolved from a wide network supplying energy for incorporated multiple areas to the largest cyber-physical system. Its security and reliability are crucial to any country’s economy and stability [1]. With the emergence of the new technologies and the growing pressure of the global warming, the aging power grid can no longer meet the requirements of the modern industry, which leads to the proposal of ‘smart grid’. In smart grid, both electricity and control information communicate in a massively distributed power network. It is essential for smart grid to deliver real-time data by communication network. By using smart meter, AMI can measure energy consumption, monitor loads, collect data and forward information to collectors. Smart grid is an intelligent network consists of many technologies in not only power but also information, telecommunications and control. The most famous structure of smart grid is the three-layer structure. It divides smart grid into three different layers, each layer has its own duty. All these three layers work together, providing us a smart grid that monitor and optimize the operations of all functional units from power generation to all the end-customers [2]. To enhance the security level of future smart grid, deploying a high secure level data transmission scheme on critical nodes is an effective and practical approach. A critical node is a communication node in a cyber-physical network which can be developed to meet certain requirements. It also has firewalls and capability of intrusion detection, so it is useful for a time-critical network system, in other words, it is suitable for future smart grid. The deployment of such a scheme can be tricky regarding to different network topologies. A simple and general way is to install it on every node in the network, that is to say all nodes in this network are critical nodes, but this way takes time, energy and money. Obviously, it is not the best way to do so. Thus, we propose a multi-objective evolutionary algorithm for the searching of critical nodes. A new scheme should be proposed for smart grid. Also, an optimal planning in power grid for embedding large system can effectively ensure every power station and substation to operate safely and detect anomalies in time. Using such a new method is a reliable method to meet increasing security challenges. The evolutionary frame helps in getting optimum without calculating the gradient of the objective function. In the meanwhile, a means of decomposition is useful for exploring solutions evenly in decision space. Furthermore, constraints handling technologies can place critical nodes on optimal locations so as to enhance system security even with several constraints of limited resources and/or hardware. The high-quality experimental results have validated the efficiency and applicability of the proposed approach. It has good reason to believe that the new algorithm has a promising space over the real-world multi-objective optimization problems extracted from power grid security domain. In this thesis, a cloud-based information infrastructure is proposed to deal with the big data storage and computation problems for the future smart grid, some challenges and limitations are addressed, and a new secure data management and transmission strategy regarding increasing security challenges of future smart grid are given as well

Secure Data Management and Transmission Infrastructure for the Future Smart Grid

Power grid has played a crucial role since its inception in the Industrial Age. It has evolved from a wide network supplying energy for incorporated multiple areas to the largest cyber-physical system. Its security and reliability are crucial to any country’s economy and stability [1]. With the emergence of the new technologies and the growing pressure of the global warming, the aging power grid can no longer meet the requirements of the modern industry, which leads to the proposal of ‘smart grid’. In smart grid, both electricity and control information communicate in a massively distributed power network. It is essential for smart grid to deliver real-time data by communication network. By using smart meter, AMI can measure energy consumption, monitor loads, collect data and forward information to collectors. Smart grid is an intelligent network consists of many technologies in not only power but also information, telecommunications and control. The most famous structure of smart grid is the three-layer structure. It divides smart grid into three different layers, each layer has its own duty. All these three layers work together, providing us a smart grid that monitor and optimize the operations of all functional units from power generation to all the end-customers [2]. To enhance the security level of future smart grid, deploying a high secure level data transmission scheme on critical nodes is an effective and practical approach. A critical node is a communication node in a cyber-physical network which can be developed to meet certain requirements. It also has firewalls and capability of intrusion detection, so it is useful for a time-critical network system, in other words, it is suitable for future smart grid. The deployment of such a scheme can be tricky regarding to different network topologies. A simple and general way is to install it on every node in the network, that is to say all nodes in this network are critical nodes, but this way takes time, energy and money. Obviously, it is not the best way to do so. Thus, we propose a multi-objective evolutionary algorithm for the searching of critical nodes. A new scheme should be proposed for smart grid. Also, an optimal planning in power grid for embedding large system can effectively ensure every power station and substation to operate safely and detect anomalies in time. Using such a new method is a reliable method to meet increasing security challenges. The evolutionary frame helps in getting optimum without calculating the gradient of the objective function. In the meanwhile, a means of decomposition is useful for exploring solutions evenly in decision space. Furthermore, constraints handling technologies can place critical nodes on optimal locations so as to enhance system security even with several constraints of limited resources and/or hardware. The high-quality experimental results have validated the efficiency and applicability of the proposed approach. It has good reason to believe that the new algorithm has a promising space over the real-world multi-objective optimization problems extracted from power grid security domain. In this thesis, a cloud-based information infrastructure is proposed to deal with the big data storage and computation problems for the future smart grid, some challenges and limitations are addressed, and a new secure data management and transmission strategy regarding increasing security challenges of future smart grid are given as well

The State-of-the-Art Survey on Optimization Methods for Cyber-physical Networks

Cyber-Physical Systems (CPS) are increasingly complex and frequently integrated into modern societies via critical infrastructure systems, products, and services. Consequently, there is a need for reliable functionality of these complex systems under various scenarios, from physical failures due to aging, through to cyber attacks. Indeed, the development of effective strategies to restore disrupted infrastructure systems continues to be a major challenge. Hitherto, there have been an increasing number of papers evaluating cyber-physical infrastructures, yet a comprehensive review focusing on mathematical modeling and different optimization methods is still lacking. Thus, this review paper appraises the literature on optimization techniques for CPS facing disruption, to synthesize key findings on the current methods in this domain. A total of 108 relevant research papers are reviewed following an extensive assessment of all major scientific databases. The main mathematical modeling practices and optimization methods are identified for both deterministic and stochastic formulations, categorizing them based on the solution approach (exact, heuristic, meta-heuristic), objective function, and network size. We also perform keyword clustering and bibliographic coupling analyses to summarize the current research trends. Future research needs in terms of the scalability of optimization algorithms are discussed. Overall, there is a need to shift towards more scalable optimization solution algorithms, empowered by data-driven methods and machine learning, to provide reliable decision-support systems for decision-makers and practitioners

Blockchain-Based Water-Energy Transactive Management with Spatial-Temporal Uncertainties

Water resources are vital to the energy conversion process but few efforts have been devoted to the joint optimization problem which is fundamentally critical to the water-energy nexus for small-scale or remote energy systems (e.g., energy hubs). Traditional water and energy trading mechanisms depend on centralized authorities and cannot preserve security and privacy effectively. Also, their transaction process cannot be verified and is subject to easy tampering and frequent exposures to cyberattacks, forgery, and network failures. Toward that end, water-energy hubs (WEHs) offers a promising way to analyse water-energy nexus for greater resource utilization efficiency. We propose a two-stage blockchain-based transactive management method for multiple, interconnected WEHs. Our method considers peer-topeer (P2P) trading and demand response, and leverages blockchain to create a secure trading environment. It features auditing and resource transaction record management via system aggregators enabled by a consortium blockchain, and entails spatial-temporal distributionally robust optimization (DRO) for renewable generation and load uncertainties. A spatial-temporal ambiguity set is incorporated in DRO to characterize the spatial-temporal dependencies of the uncertainties in distributed renewable generation and load demand. We conduct a simulation-based evaluation that includes robust optimization and the moment-based DRO as benchmarks. The results reveal that our method is consistently more effective than both benchmarks. Key findings include i) our method reduces conservativeness with lower WEH trading and operation costs, and achieves important performance improvements by up to 6.1%; and ii) our method is efficient and requires 18.7% less computational time than the moment-based DRO. Overall, this study contributes to the extant literature by proposing a novel two-stage blockchain-based WEH transaction method, developing a realistic spatialtemporal ambiguity set to effectively hedge against the uncertainties for distributed renewable generation and load demand, and producing empirical evidence suggesting its greater effectiveness and values than several prevalent methods.</p

Blockchain-Based Water-Energy Transactive Management with Spatial-Temporal Uncertainties

Water resources are vital to the energy conversion process but few efforts have been devoted to the joint optimization problem which is fundamentally critical to the water-energy nexus for small-scale or remote energy systems (e.g., energy hubs). Traditional water and energy trading mechanisms depend on centralized authorities and cannot preserve security and privacy effectively. Also, their transaction process cannot be verified and is subject to easy tampering and frequent exposures to cyberattacks, forgery, and network failures. Toward that end, water-energy hubs (WEHs) offers a promising way to analyse water-energy nexus for greater resource utilization efficiency. We propose a two-stage blockchain-based transactive management method for multiple, interconnected WEHs. Our method considers peer-topeer (P2P) trading and demand response, and leverages blockchain to create a secure trading environment. It features auditing and resource transaction record management via system aggregators enabled by a consortium blockchain, and entails spatial-temporal distributionally robust optimization (DRO) for renewable generation and load uncertainties. A spatial-temporal ambiguity set is incorporated in DRO to characterize the spatial-temporal dependencies of the uncertainties in distributed renewable generation and load demand. We conduct a simulation-based evaluation that includes robust optimization and the moment-based DRO as benchmarks. The results reveal that our method is consistently more effective than both benchmarks. Key findings include i) our method reduces conservativeness with lower WEH trading and operation costs, and achieves important performance improvements by up to 6.1%; and ii) our method is efficient and requires 18.7% less computational time than the moment-based DRO. Overall, this study contributes to the extant literature by proposing a novel two-stage blockchain-based WEH transaction method, developing a realistic spatialtemporal ambiguity set to effectively hedge against the uncertainties for distributed renewable generation and load demand, and producing empirical evidence suggesting its greater effectiveness and values than several prevalent methods.</p