Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

A PARETO-FRONTIER ANALYSIS OF PERFORMANCE TRENDS FOR SMALL REGIONAL COVERAGE LEO CONSTELLATION SYSTEMS

As satellites become smaller, cheaper, and quicker to manufacture, constellation systems will be an increasingly attractive means of meeting mission objectives. Optimizing satellite constellation geometries is therefore a topic of considerable interest. As constellation systems become more achievable, providing coverage to specific regions of the Earth will become more common place. Small countries or companies that are currently unable to afford large and expensive constellation systems will now, or in the near future, be able to afford their own constellation systems to meet their individual requirements for small coverage regions. The focus of this thesis was to optimize constellation geometries for small coverage regions with the constellation design limited between 1-6 satellites in a Walker-delta configuration, at an altitude of 200-1500km, and to provide remote sensing coverage with a minimum ground elevation angle of 60 degrees. Few Pareto-frontiers have been developed and analyzed to show the tradeoffs among various performance metrics, especially for this type of constellation system. The performance metrics focus on geometric coverage and include revisit time, daily visibility time, constellation altitude, ground elevation angle, and the number of satellites. The objective space containing these performance metrics were characterized for 5 different regions at latitudes of 0, 22.5, 45, 67.5, and 90 degrees. In addition, the effect of minimum ground elevation angle was studied on the achievable performance of this type of constellation system. Finally, the traditional Walker-delta pattern constraint was relaxed to allow for asymmetrical designs. These designs were compared to see how the Walker-delta pattern performs compared to a more relaxed design space. The goal of this thesis was to provide both a framework as well as obtain and analyze Pareto-frontiers for constellation performance relating to small regional coverage LEO constellation systems. This work provided an in-depth analysis of the trends in both the design and objective space of the obtained Pareto-frontiers. A variation on the εNSGA-II algorithm was utilized along with a MATLAB/STK interface to produce these Pareto-frontiers. The εNSGA-II algorithm is an evolutionary algorithm that was developed by Kalyanmoy Deb to solve complex multi-objective optimization problems. The algorithm used in this study proved to be very efficient at obtaining various Pareto-frontiers. This study was also successful in characterizing the design and solution space surrounding small LEO remote sensing constellation systems providing small regional coverage

An Algorithmic Framework for Multiobjective Optimization

Multiobjective (MO) optimization is an emerging field which is increasingly being encountered in many fields globally. Various metaheuristic techniques such as differential evolution (DE), genetic algorithm (GA), gravitational search algorithm (GSA), and particle swarm optimization (PSO) have been used in conjunction with scalarization techniques such as weighted sum approach and the normal-boundary intersection (NBI) method to solve MO problems. Nevertheless, many challenges still arise especially when dealing with problems with multiple objectives (especially in cases more than two). In addition, problems with extensive computational overhead emerge when dealing with hybrid algorithms. This paper discusses these issues by proposing an alternative framework that utilizes algorithmic concepts related to the problem structure for generating efficient and effective algorithms. This paper proposes a framework to generate new high-performance algorithms with minimal computational overhead for MO optimization

樹状突起ニューロン計算および差分進化アルゴリズムに関する研究

富山大学・富理工博甲第118号・陳瑋・2017/03/23富山大学201

Single and Multiobjective Optimal Reactive Power Dispatch Based on Hybrid Artificial Physics–Particle Swarm Optimization

The optimal reactive power dispatch (ORPD) problem represents a noncontinuous, nonlinear, highly constrained optimization problem that has recently attracted wide research investigation. This paper presents a new hybridization technique for solving the ORPD problem based on the integration of particle swarm optimization (PSO) with artificial physics optimization (APO). This hybridized algorithm is tested and verified on the IEEE 30, IEEE 57, and IEEE 118 bus test systems to solve both single and multiobjective ORPD problems, considering three main aspects. These aspects include active power loss minimization, voltage deviation minimization, and voltage stability improvement. The results prove that the algorithm is effective and displays great consistency and robustness in solving both the single and multiobjective functions while improving the convergence performance of the PSO. It also shows superiority when compared with results obtained from previously reported literature for solving the ORPD problem

Multi-objective trajectory optimization of Space Maneuver Vehicle using adaptive differential evolution and modified game theory

Highly constrained trajectory optimization for Space Manoeuvre Vehicles (SMV) is a challenging problem. In practice, this problem becomes more difficult when multiple mission requirements are taken into account. Because of the nonlinearity in the dynamic model and even the objectives, it is usually hard for designers to generate a compromised trajectory without violating strict path and box constraints. In this paper, a new multi-objective SMV optimal control model is formulated and parameterized using combined shooting-collocation technique. A modified game theory approach, coupled with an adaptive differential evolution algorithm, is designed in order to generate the pareto front of the multi-objective trajectory optimization problem. In addition, to improve the quality of obtained solutions, a control logic is embedded in the framework of the proposed approach. Several existing multi-objective evolutionary algorithms are studied and compared with the proposed method. Simulation results indicate that without driving the solution out of the feasible region, the proposed method can perform better in terms of convergence ability and convergence speed than its counterparts. Moreover, the quality of the pareto set generated using the proposed method is higher than other multi-objective evolutionary algorithms, which means the newly proposed algorithm is more attractive for solving multi-criteria SMV trajectory planning problem

On the Application of Multiobjective Optimization to Software Development Process and Antenna Designing

Esta tesis doctoral, presentada como compendio de artículos, explora los beneficios prácticos del uso combinado de la optimización multi-objetivo con aplicaciones de simulación. En esta tesis, con un caracter de aplicación, se aportan ideas prácticas sobre cómo combinar meta-heurísticas aplicadas a la optimización de problemas con herramientas y técnicas de simulación. La simulación permite estudiar problemas complejos antes de implementarlos en el mundo real. Los problemas de optimización son de los más complicados de resolver. Involucran 3 o más variables y en muchos casos no pueden ser resueltos matemáticamente. La simulación permite modelar el problema, pero son una ayuda insuficiente a la hora de encontrar las mejores soluciones a dicho problema. En estos casos, el trabajo conjunto de la herramienta de simulación con metaheurísticas de optimización permiten abordar estos problemas con costes computacionales razonables, obteniendo resultados muy cercanos al óptimo. Debe tenerse en cuenta que las soluciones de los problemas multiobjetivo contienen un conjunto de variables donde habitualmente mejorar (optimizar) una variable, suponga empeorar (hacer menos óptima) otra(s). Por tanto, lo deseable es encontrar un conjunto de soluciones donde cada variable se optimiza teniendo en cuenta el posible impacto negativo en el resto de variables. A ese conjunto de soluciones, se le suele conocer como el Frente de Pareto Óptimo. Esta tesis presenta dos problemas reales, complejos y pertenecientes a campos totalmente distintos, que han sido resueltos de forma existosa, aplicando la misma técnica: Simulación combinada con optimización multiobjetivo. Esta tesis comienza presentando un caso de técnicas de optimización multiobjetivo a través de la simulación para ayudar a los directores de proyectos de software a encontrar las mejores configuraciones para los proyectos basados ITIL (Information Technology Infrastructure Library), de manera que se optimicen las estimaciones de calendario para un proyecto determinado, el tiempo y la productividad. Los datos de gestión de proyectos pueden obtenerse mediante simulación, por ejemplo, para optimizar el número de recursos utilizados en cada fase de la vida del proyecto. También se presenta otro caso de estudio sobre la forma en que la optimización de la simulación puede ayudar en el diseño de cualquier tipo de antena. En este caso de estudio, el objetivo es lograr una antena helicoidal, de doble banda, lo más compacta posible, para la telemetría, el seguimiento y el control (TTC) de los satélites. En los satélites es esencial reducir el volumen y el peso de los dispositivos instalados, manteniendo al mismo tiempo los requisitos de funcionamiento. Adicionalmente, esta tesis realiza un aporte teórico proponiendo un nuevo algoritmo llamado MNDS (Merge Non-Dominated Sorting) que mejora el rendimiento de los algoritmos de optimización multi-objectivo basados en el cálculo del Pareto Front

AN IMPROVED BARE-BONES PARTICLE SWARM ALGORITHM FOR MULTI-OBJECTIVE OPTIMIZATION WITH APPLICATION TO THE ENGINEERING STRUCTURES

In this paper, an improved bare-bones multi-objective particle swarm algorithm is proposed to solve the multi-objective size optimization problems with non-linearity and constraints in structural design and optimization. Firstly, the development of particle individual guide and the randomness of gravity factor are increased by modifying the updated form of particle position. Then, the combination of spatial grid density and congestion distance ranking is used to maintain the external archive, which is divided into two parts: feasible solution set and infeasible solution set. Next, the global best positions are determined by increasing the probability allocation strategy which varies with time. The algorithmic complexity is given and the performance of solution ability, convergence and constraint processing are analyzed through standard test functions and compared with other algorithms. Next, as a case study, a support frame of triangle track wheel is optimized by the BB-MOPSO and improved BB-MOPSO. The results show that the improved algorithm improves the cross-region exploration, optimal solution distribution and convergence of the bare-bones particle swarm optimization algorithm, which can effectively solve the multi-objective size optimization problem with non-linearity and constraints