Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

An Improved Imperialist Competitive Algorithm based on a new assimilation strategy

Meta-heuristic algorithms inspired by the natural processes are part of the optimization algorithms that they have been considered in recent years, such as genetic algorithm, particle swarm optimization, ant colony optimization, Firefly algorithm. Recently, a new kind of evolutionary algorithm has been proposed that it is inspired by the human sociopolitical evolution process. This new algorithm has been called Imperialist Competitive Algorithm (ICA). The ICA is a population-based algorithm where the populations are represented by countries that are classified as colonies or imperialists. This paper is going to present a modified ICA with considerable accuracy, referred to here as ICA2. The ICA2 is tested with six well-known benchmark functions. Results show high accuracy and avoidance of local optimum traps to reach the minimum global optimal.Three important policies are in the ICA, and assimilation policy is the most important of them. This research focuses on an assimilation policy in the ICA to propose a meta-heuristic optimization algorithm for optimizing function with high accuracy and avoiding to trap in local optima rather than using original ICA by a new assimilation strategy

On Proportionate and Truthful International Alliance Contributions: An Analysis of Incentive Compatible Cost Sharing Mechanisms to Burden Sharing

Burden sharing within an international alliance is a contentious topic, especially in the current geopolitical environment, that in practice is generally imposed by a central authority\u27s perception of its members\u27 abilities to contribute. Instead, we propose a cost sharing mechanism such that burden shares are allocated to nations based on their honest declarations of the alliance\u27s worth. Specifically, we develop a set of multiobjective nonlinear optimization problem formulations that respectively impose Bayesian Incentive Compatible (BIC), Strategyproof (SP), and Group Strategyproof (GSP) mechanisms based on probabilistic inspection efforts and deception penalties that are budget balanced and in the core. Any feasible solution to these problems corresponds to a single stage Bayesian stochastic game wherein a collectively honest declaration is a Bayes-Nash equilibrium, a Nash Equilibrium in dominant strategies, or a collusion resistant Nash equilibrium, respectively, but the optimal solution considers the alliance\u27s central authority preferences. Each formulation is shown to be a nonconvex optimization problem. The solution quality and computational effort required for three heuristic algorithms as well as the BARON global solver are analyzed to determine the superlative solution methodology for each problem. The Pareto fronts associated with each multiobjective optimization problem are examined to determine the tradeoff between inspection frequency and penalty severity required to obtain truthfulness under stronger assumptions. Memory limitations are examined to ascertain the size of alliances for which the proposed methodology can be utilized. Finally, a full block design experiment considering the clustering of available alliance valuations and the member nations\u27 probability distributions therein is executed on an intermediate-sized alliance motivated by the South American alliance UNASUR

Solution Techniques for Large-Scale Optimization Problems on the Transmission Grid

In this thesis, we are interested in solution techniques and primal heuristics for several large-scale optimization problems on the transmission grid. While some of these problems have been studied for a long time, none of the techniques proposed previously allowed them to be solved exactly on large-scale networks, rendering them of little use in practice. We will present methodology which yields high quality solutions on large networks. In Chapter 2, we consider the DC optimal transmission switching (DCOTS) problem. In this problem, we simultaneously optimize the grid topology (i.e., choose which lines are on and off) and the generator dispatch, using a DC optimal power flow (DCOPF) model. It is well-known that transmission switching is an affordable way to reduce congestion in the grid, reducing generation costs. However, DCOTS has so far been a prohibitively difficult model to solve, particularly given that dispatch problems are solved every 5 to 10 minutes for most independent system operators. We present a data-driven approach which assumes that DCOTS has been solved to optimality offline for a variety of demand profiles. We then use the k-nearest neighbors (KNN) method as a primal heuristic, directly mapping from demand profiles to topologies. This scales well since the computational time is dominated by the time to solve k linear programs. We find that we can generate high-quality primal solutions within the time constraints imposed by real-time operations. In addition, we find that defining the feature space for KNN differently can also yield equally good results: In particular, using dual information from the DCOPF problem can be effective. In Chapter 3, we propose a scalable lower bound for a worst-case attack on transmission grid relays. This is a bilevel interdiction problem in which an attacker first targets relays within an attack budget, compromising all components controlled by the relays he chooses, and aiming to maximize load shed. Then, a defender redispatches the generators, solving a DCOPF model and minimizing the load shed. Since the inner problem is convex, it is possible to dualize it, resulting in a mixed-integer single-level reformulation. However, the difficulty arises in linearizing this reformulation. Without bounds on the dual variables of DCOPF, this is not possible to do exactly. Prior literature has used heuristic bounds on the duals. However, in addition to providing a lower bound only, this comes at a computational cost: The more conservatively the bounds are chosen, the larger the big-M values are in the resulting mixed-integer programming (MIP) formulation. This worsens the continuous relaxation and makes it increasingly difficult for even commercial solvers. Instead, we propose using a different lower bound: We relax DCOPF to capacitated network flow, dropping the constraints corresponding to Ohm's law. We show that, on uncongested networks, the injections we get from solving this relaxation are a good approximation of those from solving DCOPF. We can again dualize this problem, creating a single-level formulation. The duals of the relaxed defender problem are bounded in absolute value by 1, meaning we can linearize the single-level formulation and solve it exactly. Furthermore this MIP scales extremely well when solving with a commercial solver. Last, in Chapter 4, we present methodology to solve a trilevel interdiction problem where the inner bilevel problem is the worst-case relay attack problem from Chapter 3. In this problem, we optimize the design of the Supervisory Control and Data Acquisition (SCADA) network controlling the transmission grid in order to minimize the impact of a worst-case cyberattack. Specifically, we decide where the cyber networks in the SCADA system should be subdivided, with communication limited between these subdivisions, a technique called network segmentation. The resulting problem is a trilevel interdiction model with pure integer first and second player problems and a convex third-player problem. We show that it can be solved for large-scale power networks using a covering decomposition approach in which we iteratively fix a network design and generate a worst-case attack. We then find a new design that makes all the generated attacks infeasible. When there is no such design, then the design corresponding to the least-damaging attack generated so far is optimal. We show empirically that this method is scalable for large power networks and moderate network designer and attacker budgets.Ph.D

Application of heat pumps and thermal storage systems for improved control and performance of microgrids

The high penetration of renewable energy sources (RES), in particular, the rooftop photovoltaic (PV) systems in power systems, causes rapid ramps in power generation to supply load during peak-load periods. Residential and commercial buildings have considerable potential for providing load exibility by exploiting energy-e_cient devices like ground source heat pump (GSHP). The proper integration of PV systems with the GSHP could reduce power demand from demand-side. This research provides a practical attempt to integrate PV systems and GSHPs e_ectively into buildings and the grid. The multi-directional approach in this work requires an optimal control strategy to reduce energy cost and provide an opportunity for power trade-o_ or feed-in in the electricity market. In this study, some optimal control models are developed to overcome both the operational and technical constraints of demand-side management (DSM) and for optimum integration of RES. This research focuses on the development of an optimal real-time thermal energy management system for smart homes to respond to DR for peak-load shifting. The intention is to manage the operation of a GSHP to produce the desired amount of thermal energy by controlling the volume and temperature of the stored water in the thermal energy storage (TES) while optimising the operation of the heat distributors to control indoor temperature. This thesis proposes a new framework for optimal sizing design and real-time operation of energy storage systems in a residential building equipped with a PV system, heat pump (HP), and thermal and electrical energy storage systems. The results of this research demonstrate to rooftop PV system owners that investment in combined TSS and battery can be more profitable as this system can minimise life cycle costs. This thesis also presents an analysis of the potential impact of residential HP systems into reserve capacity market. This research presents a business aggregate model for controlling residential HPs (RHPs) of a group of houses that energy aggregators can utilise to earn capacity credits. A control strategy is proposed based on a dynamic aggregate RHPs coupled with TES model and predicting trading intervals capacity requirements through forecasting demand and non-scheduled generation. RHPs coupled with TES are optimised to provide DSM reserve capacity. A rebound effect reduction method is proposed that reduces the peak rebound RHPs power

Integration of renewable energy into Nigerian power systems

Many countries are advancing down the road of electricity privatization, deregulation, and competition as a solution to their growing electricity demand and other challenges posed by the monopolistic nature of the existing structure. Presently, Nigeria has a supply deficit of electricity as a result of the growing demand. This imbalance has negatively affected the economy of the country and the social-economic well-being of the population. Hence, there is an urgent need to reform the power sector for greater efficiency and better performance. The objectives of the reform are to meet the growing power demand by increasing the electric power generation and also by increasing competitiveness through the participation of more private sector entities. The renewable energy integration is one way of increasing the electricity generation in the country in order to cater for the growing demand adequately. Examples of the renewable energy that is available in the country include wind, geothermal, solar and hydro. They are considered to be environmentally friendly, replenishable and do not contribute to the climate change phenomena. The country presently generates the bulk of its electricity from both thermal (85%) and hydroelectric (15%) power plants. While electricity generation from the thermal power stations constitutes the largest share of greenhouse emission, this is mostly from burning coal and natural gas. The effect of this high proportion of greenhouse emission causes climate change which is referred to as a variation in the climate system statistical properties over a long period of time. It has been observed that many of the activities of human beings are contributory factors to the release of these greenhouse gases (GHG). But, as the traditional sources of energy continue to threaten the present and future existence on the planet earth, it is, therefore, imperative to increase the integration of the variable renewable energy sources in a sustainable and eco-friendly manner over a long period of time. The variability and the uncertainties of the renewable energy source's output, present a major challenge in the design of an efficient electricity market in a deregulated environment. The system deregulation and the use of renewable sources for the generation of electricity are major changes presently being experienced in power system. In a deregulated power system, the integration of renewable generation and its penetration affects both the physical and the economic operations. The main focus of this research is on the integration of wind energy into Nigerian power systems. Up till now, research on the availability of the wind energy and its economic impacts has been limited in Nigeria. Generally, the previous study of wind energy availability in Nigeria has been limited in scope. The wind energy assessment study has not been detailed enough to be able to ascertain the wind energy potential of the country. To cope with this shortcoming, a detailed statistical wind modeling and forecasting methodology have been used in this thesis to determine the amount of extractable wind energy in six selected locations in Nigeria using historical wind speed data for 30 years. The accuracy test of the statistical models was also carried using the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Chi-Square methods to determine the inherent error margin in the modeling and analysis. It is found that the error margin of the evaluations falls within the expected permissible tolerance range. For a more detailed wind assessment study of the Nigeria weather, the seasonal variation of the weather conditions as it affects the wind speed and availability during the two major seasons of dry and rainy was considered. A Self-Adaptive Differential Evolution (SADE) was used to solve the economic load dispatch problem that considers the valve-point effects and the transmission losses subject to many constraints. The results obtained were compared with those obtained using the "standard" Differential Evolution (DE), Genetic Algorithm (GA), and traditional Gradient Descent method. The results of the SADE obtained when compared with the GA, DE, and Gradient descent show the superiority of SADE over all the other methods. The research work shows that the wind energy is available in commercial quantity for generation of electricity in Nigeria. And, if tapped would help reduce the gap between the demand and supply of electricity in the country. It was also demonstrated that the wind energy integration into the power systems affects the generators total production cost

Hiroshima University Research and Technology Guide 2012 Version : Physical Science & Engineering

II Environment/Energy III Design and Manufacturing IV Material/Device V Mechanical Engineering VI Civil Engineering/Architecture VII Computer Science, Information, Communication and System Engineering VIII Measurement & Control/Scientific Analyse

Optimisation, Optimal Control and Nonlinear Dynamics in Electrical Power, Energy Storage and Renewable Energy Systems

The electrical power system is undergoing a revolution enabled by advances in telecommunications, computer hardware and software, measurement, metering systems, IoT, and power electronics. Furthermore, the increasing integration of intermittent renewable energy sources, energy storage devices, and electric vehicles and the drive for energy efficiency have pushed power systems to modernise and adopt new technologies. The resulting smart grid is characterised, in part, by a bi-directional flow of energy and information. The evolution of the power grid, as well as its interconnection with energy storage systems and renewable energy sources, has created new opportunities for optimising not only their techno-economic aspects at the planning stages but also their control and operation. However, new challenges emerge in the optimization of these systems due to their complexity and nonlinear dynamic behaviour as well as the uncertainties involved.This volume is a selection of 20 papers carefully made by the editors from the MDPI topic “Optimisation, Optimal Control and Nonlinear Dynamics in Electrical Power, Energy Storage and Renewable Energy Systems”, which was closed in April 2022. The selected papers address the above challenges and exemplify the significant benefits that optimisation and nonlinear control techniques can bring to modern power and energy systems

Statistical Computational Topology and Geometry for Understanding Data

Here we describe three projects involving data analysis which focus on engaging statistics with the geometry and/or topology of the data. The first project involves the development and implementation of kernel density estimation for persistence diagrams. These kernel densities consider neighborhoods for every feature in the center diagram and gives to each feature an independent, orthogonal direction. The creation of kernel densities in this realm yields a (previously unavailable) full characterization of the (random) geometry of a dataspace or data distribution. In the second project, cohomology is used to guide a search for kidney exchange cycles within a kidney paired donation pool. The same technique also produces a score function that helps to predict a patient-donor pair\u27s a priori advantage within a donation pool. The resulting allocation of cycles is determined to be equitable according to a strict analysis of the allocation distribution. In the last project, a previously formulated metric between surfaces called continuous Procrustes distance (CPD) is applied to species discrimination in fossils. This project involves both the application and a rigorous comparison of the metric with its primary competitor, discrete Procrustes distance. Besides comparing the separation power of discrete and continuous Procrustes distances, the effect of surface resolution on CPD is investigated in this study