Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Using Optimality Theory and Reference Points to Improve the Diversity and Convergence of a Fuzzy-Adaptive Multi-Objective Particle Swarm Optimizer

Particle Swarm Optimization (PSO) has received increasing attention from the evolutionary optimization research community in the last twenty years. PSO is a metaheuristic approach based on collective intelligence obtained by emulating the swarming behavior of bees. A number of multi-objective variants of the original PSO algorithm that extend its applicability to optimization problems with conflicting objectives have also been developed; these multi-objective PSO (MOPSO) algorithms demonstrate comparable performance to other state-of-the-art metaheuristics. The existence of multiple optimal solutions (Pareto-optimal set) in optimization problems with conflicting objectives is not the only challenge posed to an optimizer, as the latter needs to be able to identify and preserve a well-distributed set of solutions during the search of the decision variable space. Recent attempts by evolutionary optimization researchers to incorporate mathematical convergence conditions into genetic algorithm optimizers have led to the derivation of a point-wise proximity measure, which is based on the solution of the achievement scalarizing function (ASF) optimization problem with a complementary slackness condition that quantifies the violation of the Karush-Kuhn-Tucker necessary conditions of optimality. In this work, the aforementioned KKT proximity measure is incorporated into the original Adaptive Coevolutionary Multi-Objective Swarm Optimizer (ACMOPSO) in order to monitor the convergence of the sub-swarms towards the Pareto-optimal front and provide feedback to Mamdani-type fuzzy logic controllers (FLCs) that are utilized for online adaptation of the algorithmic parameters. The proposed Fuzzy-Adaptive Multi-Objective Optimization Algorithm with the KKT proximity measure (FAMOPSOkkt) utilizes a set of reference points to cluster the computed nondominated solutions. These clusters interact with their corresponding sub-swarms to provide the swarm leaders and are also utilized to manage the external archive of nondominated solutions. The performance of the proposed algorithm is evaluated on benchmark problems chosen from the multi-objective optimization literature and compared to the performance of state-of-the-art multi-objective optimization algorithms with similar features

A compromise-based particle swarm optimization algorithm for solving Bi-level programming problems with fuzzy parameters

© 2015 IEEE. Bi-level programming has arisen to handle decentralized decision-making problems that feature interactive decision entities distributed throughout a bi-level hierarchy. Fuzzy parameters often appear in such a problem in applications and this is called a fuzzy bi-level programming problem. Since the existing approaches lack universality in solving such problems, this study aims to develop a particle swarm optimization (PSO) algorithm to solve fuzzy bi-level programming problems in the linear and nonlinear versions. In this paper, we first present a general fuzzy bi-level programming problem and discuss related theoretical properties based on a fuzzy number ranking method commonly used. A PSO algorithm is then developed to solve the fuzzy bi-level programming problem based on different compromised selections by decision entities on the feasible degree for constraint conditions under fuzziness. Lastly, an illustrative numerical example and two benchmark examples are adopted to state the effectiveness of the compromise-based PSO algorithm

A survey of swarm intelligence for dynamic optimization: algorithms and applications

Swarm intelligence (SI) algorithms, including ant colony optimization, particle swarm optimization, bee-inspired algorithms, bacterial foraging optimization, firefly algorithms, fish swarm optimization and many more, have been proven to be good methods to address difficult optimization problems under stationary environments. Most SI algorithms have been developed to address stationary optimization problems and hence, they can converge on the (near-) optimum solution efficiently. However, many real-world problems have a dynamic environment that changes over time. For such dynamic optimization problems (DOPs), it is difficult for a conventional SI algorithm to track the changing optimum once the algorithm has converged on a solution. In the last two decades, there has been a growing interest of addressing DOPs using SI algorithms due to their adaptation capabilities. This paper presents a broad review on SI dynamic optimization (SIDO) focused on several classes of problems, such as discrete, continuous, constrained, multi-objective and classification problems, and real-world applications. In addition, this paper focuses on the enhancement strategies integrated in SI algorithms to address dynamic changes, the performance measurements and benchmark generators used in SIDO. Finally, some considerations about future directions in the subject are given

08051 Abstracts Collection -- Theory of Evolutionary Algorithms

From Jan. 27, 2008 to Feb. 1, 2008, the Dagstuhl Seminar 08051 ``Theory of Evolutionary Algorithms\u27\u27 was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

A review of optimization techniques in spacecraft flight trajectory design

For most atmospheric or exo-atmospheric spacecraft flight scenarios, a well-designed trajectory is usually a key for stable flight and for improved guidance and control of the vehicle. Although extensive research work has been carried out on the design of spacecraft trajectories for different mission profiles and many effective tools were successfully developed for optimizing the flight path, it is only in the recent five years that there has been a growing interest in planning the flight trajectories with the consideration of multiple mission objectives and various model errors/uncertainties. It is worth noting that in many practical spacecraft guidance, navigation and control systems, multiple performance indices and different types of uncertainties must frequently be considered during the path planning phase. As a result, these requirements bring the development of multi-objective spacecraft trajectory optimization methods as well as stochastic spacecraft trajectory optimization algorithms. This paper aims to broadly review the state-of-the-art development in numerical multi-objective trajectory optimization algorithms and stochastic trajectory planning techniques for spacecraft flight operations. A brief description of the mathematical formulation of the problem is firstly introduced. Following that, various optimization methods that can be effective for solving spacecraft trajectory planning problems are reviewed, including the gradient-based methods, the convexification-based methods, and the evolutionary/metaheuristic methods. The multi-objective spacecraft trajectory optimization formulation, together with different class of multi-objective optimization algorithms, is then overviewed. The key features such as the advantages and disadvantages of these recently-developed multi-objective techniques are summarised. Moreover, attentions are given to extend the original deterministic problem to a stochastic version. Some robust optimization strategies are also outlined to deal with the stochastic trajectory planning formulation. In addition, a special focus will be given on the recent applications of the optimized trajectory. Finally, some conclusions are drawn and future research on the development of multi-objective and stochastic trajectory optimization techniques is discussed

Search based software engineering: Trends, techniques and applications

© ACM, 2012. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version is available from the link below.In the past five years there has been a dramatic increase in work on Search-Based Software Engineering (SBSE), an approach to Software Engineering (SE) in which Search-Based Optimization (SBO) algorithms are used to address problems in SE. SBSE has been applied to problems throughout the SE lifecycle, from requirements and project planning to maintenance and reengineering. The approach is attractive because it offers a suite of adaptive automated and semiautomated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives. This article provides a review and classification of literature on SBSE. The work identifies research trends and relationships between the techniques applied and the applications to which they have been applied and highlights gaps in the literature and avenues for further research.EPSRC and E

Evolutionary Dynamic Multi-Objective Optimisation : A survey

Peer reviewedPostprin