Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

A simplex-like search method for bi-objective optimization

We describe a new algorithm for bi-objective optimization, similar to the Nelder Mead simplex algorithm, widely used for single objective optimization. For diferentiable bi-objective functions on a continuous search space, internal Pareto optima occur where the two gradient vectors point in opposite directions. So such optima may be located by minimizing the cosine of the angle between these vectors. This requires a complex rather than a simplex, so we term the technique the \cosine seeking complex". An extra beneft of this approach is that a successful search identifes the direction of the effcient curve of Pareto points, expediting further searches. Results are presented for some standard test functions. The method presented is quite complicated and space considerations here preclude complete details. We hope to publish a fuller description in another place

An exact method for a discrete multiobjective linear fractional optimization

Integer linear fractional programming problem with multiple objective MOILFP is an important field of research and has not received as much attention as did multiple objective linear fractional programming. In this work, we develop a branch and cut algorithm based on continuous fractional optimization, for generating the whole integer efficient solutions of the MOILFP problem. The basic idea of the computation phase of the algorithm is to optimize one of the fractional objective functions, then generate an integer feasible solution. Using the reduced gradients of the objective functions, an efficient cut is built and a part of the feasible domain not containing efficient solutions is truncated by adding this cut. A sample problem is solved using this algorithm, and the main practical advantages of the algorithm are indicated.multiobjective programming, integer programming, linear fractional programming, branch and cut

Approximating Pareto frontier using a hybrid line search approach

This is the post-print version of the final paper published in Information Sciences. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2010 Elsevier B.V.The aggregation of objectives in multiple criteria programming is one of the simplest and widely used approach. But it is well known that this technique sometimes fail in different aspects for determining the Pareto frontier. This paper proposes a new approach for multicriteria optimization, which aggregates the objective functions and uses a line search method in order to locate an approximate efficient point. Once the first Pareto solution is obtained, a simplified version of the former one is used in the context of Pareto dominance to obtain a set of efficient points, which will assure a thorough distribution of solutions on the Pareto frontier. In the current form, the proposed technique is well suitable for problems having multiple objectives (it is not limited to bi-objective problems) and require the functions to be continuous twice differentiable. In order to assess the effectiveness of this approach, some experiments were performed and compared with two recent well known population-based metaheuristics namely ParEGO and NSGA II. When compared to ParEGO and NSGA II, the proposed approach not only assures a better convergence to the Pareto frontier but also illustrates a good distribution of solutions. From a computational point of view, both stages of the line search converge within a short time (average about 150 ms for the first stage and about 20 ms for the second stage). Apart from this, the proposed technique is very simple, easy to implement and use to solve multiobjective problems.CNCSIS IDEI 2412, Romani