Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Meta-heuristic algorithms in car engine design: a literature survey

Meta-heuristic algorithms are often inspired by natural phenomena, including the evolution of species in Darwinian natural selection theory, ant behaviors in biology, flock behaviors of some birds, and annealing in metallurgy. Due to their great potential in solving difficult optimization problems, meta-heuristic algorithms have found their way into automobile engine design. There are different optimization problems arising in different areas of car engine management including calibration, control system, fault diagnosis, and modeling. In this paper we review the state-of-the-art applications of different meta-heuristic algorithms in engine management systems. The review covers a wide range of research, including the application of meta-heuristic algorithms in engine calibration, optimizing engine control systems, engine fault diagnosis, and optimizing different parts of engines and modeling. The meta-heuristic algorithms reviewed in this paper include evolutionary algorithms, evolution strategy, evolutionary programming, genetic programming, differential evolution, estimation of distribution algorithm, ant colony optimization, particle swarm optimization, memetic algorithms, and artificial immune system

A convergence acceleration operator for multiobjective optimisation

A novel multiobjective optimisation accelerator is introduced that uses direct manipulation in objective space together with neural network mappings from objective space to decision space. This operator is a portable component that can be hybridized with any multiobjective optimisation algorithm. The purpose of this Convergence Acceleration Operator (CAO) is to enhance the search capability and the speed of convergence of the host algorithm. The operator acts directly in objective space to suggest improvements to solutions obtained by a multiobjective evolutionary algorithm (MOEA). These suggested improved objective vectors are then mapped into decision variable space and tested. The CAO is incorporated with two leading MOEAs, the Non-Dominated Sorting Genetic Algorithm (NSGA-II) and the Strength Pareto Evolutionary Algorithm (SPEA2) and tested. Results show that the hybridized algorithms consistently improve the speed of convergence of the original algorithm whilst maintaining the desired distribution of solutions

Modeling and Optimization of M-cresol Isopropylation for Obtaining N-thymol: Combining a Hybrid Artificial Neural Network with a Genetic Algorithm

The application of a hybrid framework based on the combination, artificial neural network-genetic algorithm (ANN-GA), for n-thymol synthesis modeling and optimization has been developed. The effects of molar ratio propylene/cresol (X1), catalyst mass (X2) and temperature (X3) on n-thymol selectivity Y1 and m-cresol conversion Y2 were studied. A 3-8-2 ANN model was found to be very suitable for reaction modeling. The multiobjective optimization, led to optimal operating conditions (0.55 ≤X1≤0.77; 1.773 g ≤ X2 ≤1.86 g; 289.74 °C ≤ X3 ≤291.33 °C) representing good solutions for obtaining high n-thymol selectivity and high m-cresol conversion. This optimal zone corresponded to n-thymol selectivity and m-cresol conversion ranging respectively in the interval [79.3; 79.5]% and [13.4 %; 23.7]%. These results were better than those obtained with a sequential method based on experimental design for which, optimum conditions led to n-thymol selectivity and m-cresol conversion values respectively equal to 67%and 11%. The hybrid method ANN-GA showed its ability to solve complex problems with a good fitting

Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs)

Recently, increasing works have proposed to drive evolutionary algorithms using machine learning models. Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality. To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial networks (GANs). At each generation of the proposed algorithm, the parent solutions are first classified into real and fake samples to train the GANs; then the offspring solutions are sampled by the trained GANs. Thanks to the powerful generative ability of the GANs, our proposed algorithm is capable of generating promising offspring solutions in high-dimensional decision space with limited training data. The proposed algorithm is tested on 10 benchmark problems with up to 200 decision variables. Experimental results on these test problems demonstrate the effectiveness of the proposed algorithm

Evolutionary model type selection for global surrogate modeling

Due to the scale and computational complexity of currently used simulation codes, global surrogate (metamodels) models have become indispensable tools for exploring and understanding the design space. Due to their compact formulation they are cheap to evaluate and thus readily facilitate visualization, design space exploration, rapid prototyping, and sensitivity analysis. They can also be used as accurate building blocks in design packages or larger simulation environments. Consequently, there is great interest in techniques that facilitate the construction of such approximation models while minimizing the computational cost and maximizing model accuracy. Many surrogate model types exist ( Support Vector Machines, Kriging, Neural Networks, etc.) but no type is optimal in all circumstances. Nor is there any hard theory available that can help make this choice. In this paper we present an automatic approach to the model type selection problem. We describe an adaptive global surrogate modeling environment with adaptive sampling, driven by speciated evolution. Different model types are evolved cooperatively using a Genetic Algorithm ( heterogeneous evolution) and compete to approximate the iteratively selected data. In this way the optimal model type and complexity for a given data set or simulation code can be dynamically determined. Its utility and performance is demonstrated on a number of problems where it outperforms traditional sequential execution of each model type

Backpropagation Neural Network Based on Local Search Strategy and Enhanced Multi-objective Evolutionary Algorithm for Breast Cancer Diagnosis

The role of intelligence techniques is becoming more significant in detecting and diagnosis of medical data. However, the performance of such methods is based on the algorithms or technique. In this paper, we develop an intelligent technique using multiobjective evolutionary method hybrid with a local search approach to enhance the backpropagation neural network. First, we enhance the famous multiobjective evolutionary algorithms, which is a non-dominated sorting genetic algorithm (NSGA-II). Then, we hybrid the enhanced algorithm with the local search strategy to ensures the acceleration of the convergence speed to the non-dominated front. In addition, such hybridization get the solutions achieved are well spread over it. As a result of using a local search method the quality of the Pareto optimal solutions are increased and all individuals in the population are enhanced. The key notion of the proposed algorithm was to  show a new technique to settle automaticly artificial neural network design problem. The empirical results generated by the proposed intelligent technique evaluated by applying to the breast cancer dataset and emphasize the capability of the proposed algorithm to improve the results. The network size and accuracy results of the proposed method are better than the previous methods. Therefore, the method is then capable of finding a proper number of hidden neurons and error rates of the BP algorithm