Preventing premature convergence and proving the optimality in evolutionary algorithms

http://ea2013.inria.fr//proceedings.pdfInternational audienceEvolutionary Algorithms (EA) usually carry out an efficient exploration of the search-space, but get often trapped in local minima and do not prove the optimality of the solution. Interval-based techniques, on the other hand, yield a numerical proof of optimality of the solution. However, they may fail to converge within a reasonable time due to their inability to quickly compute a good approximation of the global minimum and their exponential complexity. The contribution of this paper is a hybrid algorithm called Charibde in which a particular EA, Differential Evolution, cooperates with a Branch and Bound algorithm endowed with interval propagation techniques. It prevents premature convergence toward local optima and outperforms both deterministic and stochastic existing approaches. We demonstrate its efficiency on a benchmark of highly multimodal problems, for which we provide previously unknown global minima and certification of optimality

Improved sampling of the pareto-front in multiobjective genetic optimizations by steady-state evolution: a Pareto converging genetic algorithm

Previous work on multiobjective genetic algorithms has been focused on preventing genetic drift and the issue of convergence has been given little attention. In this paper, we present a simple steady-state strategy, Pareto Converging Genetic Algorithm (PCGA), which naturally samples the solution space and ensures population advancement towards the Pareto-front. PCGA eliminates the need for sharing/niching and thus minimizes heuristically chosen parameters and procedures. A systematic approach based on histograms of rank is introduced for assessing convergence to the Pareto-front, which, by definition, is unknown in most real search problems. We argue that there is always a certain inheritance of genetic material belonging to a population, and there is unlikely to be any significant gain beyond some point; a stopping criterion where terminating the computation is suggested. For further encouraging diversity and competition, a nonmigrating island model may optionally be used; this approach is particularly suited to many difficult (real-world) problems, which have a tendency to get stuck at (unknown) local minima. Results on three benchmark problems are presented and compared with those of earlier approaches. PCGA is found to produce diverse sampling of the Pareto-front without niching and with significantly less computational effort

A reliable hybrid solver for nonconvex optimization

International audienceNonconvex and highly multimodal optimization problems represent a challenge both for stochastic and deterministic global optimization methods. The former (metaheuristics) usually achieve satisfactory solutions but cannot guarantee global optimality, while the latter (generally based on a spatial branch and bound scheme [1], an exhaustive and non-uniform partitioning method) may struggle to converge toward a global minimum within reasonable time. The partitioning process is exponential in the number of variables, which prevents the resolution of large instances. The performances of the solvers even dramatically deteriorate when using reliable techniques, namely techniques that cope with rounding errors.In this paper, we present a fully reliable hybrid algorithm named Charibde (Cooperative Hybrid Algorithm using Reliable Interval-Based methods and Dierential Evolution) [2] that reconciles stochastic and deterministic techniques. An Evolutionary Algorithm (EA) cooperates with intervalbased techniques to accelerate convergence toward the global minimum and prove the optimality of the solution with user-defined precision. Charibde may be used to solve continuous, nonconvex, constrained or bound-constrained problems involving factorable functions

Certified Global Minima for a Benchmark of Difficult Optimization Problems

PreprintWe provide the global optimization community with new optimality proofs for 6 deceptive benchmark functions (5 bound-constrained functions and one nonlinearly constrained problem). These highly multimodal nonlinear test problems are among the most challenging benchmark functions for global optimization solvers; some have not been solved even with approximate methods. The global optima that we report have been numerically certified using Charibde (Vanaret et al., 2013), a hybrid algorithm that combines an Evolutionary Algorithm and interval-based methods. While metaheuristics generally solve large problems and provide sufficiently good solutions with limited computation capacity, exact methods are deemed unsuitable for difficult multimodal optimization problems. The achievement of new optimality results by Charibde demonstrates that reconciling stochastic algorithms and numerical analysis methods is a step forward into handling problems that were up to now considered unsolvable. We also provide a comparison with state-of-the-art solvers based on mathematical programming methods and population based metaheuristics, and show that Charibde, in addition to being reliable, is highly competitive with the best solvers on the given test functions

Combining Interval Methods with Evolutionary Algorithms for Global Optimization

International audienceReliable global optimization is dedicated to solving problems to optimality in the presence of rounding errors. The most successful approaches for achieving a numerical proof of optimality in global optimization are mainly exhaustive interval-based algorithms that interleave pruning and branching steps. The Interval Branch & Prune (IBP) framework has been widely studied and has benefitted from the development of refutation methods and filtering algorithms stemming from the Interval Analysis and Interval Constraint Programming communities. In a minimization problem, refutation consists in discarding subdomains of the search-space where a lower bound of the objective function is worse than the best known solution. It is therefore crucial: i) to compute a sharp lower bound of the objective function on a given subdomain; ii) to find a good approximation (an upper bound) of the global minimum. Many techniques aim at narrowing the pessimistic enclosures of interval arithmetic (centered forms, convex relaxation, local monotonicity, etc.) and will not be discussed in further detail. State-of-the-art solvers are generally integrative methods, that is they embed local optimization algorithms (BFGS, LP, interior points) to compute an upper bound of the global minimum over each subspace. In this presentation, we propose a cooperative approach in which interval methods collaborate with Evolutionary Algorithms (EA) on aglobalscale. EA are stochastic algorithms in which a population of individuals (candidate solutions) iteratively evolves in the search-space to reach satisfactory solutions. They make no assumption on the objective function and are equipped with nature-inspired operators that help individuals escape from local minima. EA are thus particularly suitable for highly multimodal nonconvex problems. In our approach, the EA and the IBP algorithm run in parallel and exchange bounds and solutions through shared memory: the EA updates the best known upper bound of the global minimum to enhance the pruning, while the IBP updates the population of the EA when a better solution is found. We show that this cooperation scheme prevents premature convergence toward local minima and accelerates the convergence of the interval method. Our hybrid algorithm also exploits a geometric heuristic to select the next subdomain to be processed, that compares well with standard heuristics (best first, largest first). We provide new optimality results for a benchmark of difficult multimodal problems (Michalewicz, Egg Holder, Rana, Keane functions). We also certify the global minimum of the (open) Lennard-Jones cluster problem for 5 atoms. Finally we present an aeronautical application to solve conflicts between aircraft

A Multi objective Approach to Evolving Artificial Neural Networks for Coronary Heart Disease Classification

The optimisation of the accuracy of classifiers in pattern recognition is a complex problem that is often poorly understood. Whilst numerous techniques exist for the optimisa- tion of weights in artificial neural networks (e.g. the Widrow-Hoff least mean squares algorithm and back propagation techniques), there do not exist any hard and fast rules for choosing the structure of an artificial neural network - in particular for choosing both the number of the hidden layers used in the network and the size (in terms of number of neurons) of those hidden layers. However, this internal structure is one of the key factors in determining the accuracy of the classification. This paper proposes taking a multi-objective approach to the evolutionary design of artificial neural networks using a powerful optimiser based around the state-of-the-art MOEA/D- DRA algorithm and a novel method of incorporating decision maker preferences. In contrast to previous approaches, the novel approach outlined in this paper allows the intuitive consideration of trade-offs between classification objectives that are frequently present in complex classification problems but are often ignored. The effectiveness of the proposed multi-objective approach to evolving artificial neural networks is then shown on a real-world medical classification problem frequently used to benchmark classification method

An Analysis of Particle Swarm Optimizers

Many scientific, engineering and economic problems involve the optimisation of a set of parameters. These problems include examples like minimising the losses in a power grid by finding the optimal configuration of the components, or training a neural network to recognise images of people's faces. Numerous optimisation algorithms have been proposed to solve these problems, with varying degrees of success. The Particle Swarm Optimiser (PSO) is a relatively new technique that has been empirically shown to perform well on many of these optimisation problems. This thesis presents a theoretical model that can be used to describe the long-term behaviour of the algorithm. An enhanced version of the Particle Swarm Optimiser is constructed and shown to have guaranteed convergence on local minima. This algorithm is extended further, resulting in an algorithm with guaranteed convergence on global minima. A model for constructing cooperative PSO algorithms is developed, resulting in the introduction of two new PSO-based algorithms. Empirical results are presented to support the theoretical properties predicted by the various models, using synthetic benchmark functions to investigate specific properties. The various PSO-based algorithms are then applied to the task of training neural networks, corroborating the results obtained on the synthetic benchmark functions.Thesis (PhD)--University of Pretoria, 2007.Computer ScienceUnrestricte

Multiobjective genetic programming for financial portfolio management in dynamic environments

Multiobjective (MO) optimisation is a useful technique for evolving portfolio optimisation solutions that span a range from high-return/high-risk to low-return/low-risk. The resulting Pareto front would approximate the risk/reward Efficient Frontier [Mar52], and simplifies the choice of investment model for a given client’s attitude to risk. However, the financial market is continuously changing and it is essential to ensure that MO solutions are capturing true relationships between financial factors and not merely over fitting the training data. Research on evolutionary algorithms in dynamic environments has been directed towards adapting the algorithm to improve its suitability for retraining whenever a change is detected. Little research focused on how to assess and quantify the success of multiobjective solutions in unseen environments. The multiobjective nature of the problem adds a unique feature to be satisfied to judge robustness of solutions. That is, in addition to examining whether solutions remain optimal in the new environment, we need to ensure that the solutions’ relative positions previously identified on the Pareto front are not altered. This thesis investigates the performance of Multiobjective Genetic Programming (MOGP) in the dynamic real world problem of portfolio optimisation. The thesis provides new definitions and statistical metrics based on phenotypic cluster analysis to quantify robustness of both the solutions and the Pareto front. Focusing on the critical period between an environment change and when retraining occurs, four techniques to improve the robustness of solutions are examined. Namely, the use of a validation data set; diversity preservation; a novel variation on mating restriction; and a combination of both diversity enhancement and mating restriction. In addition, preliminary investigation of using the robustness metrics to quantify the severity of change for optimum tracking in a dynamic portfolio optimisation problem is carried out. Results show that the techniques used offer statistically significant improvement on the solutions’ robustness, although not on all the robustness criteria simultaneously. Combining the mating restriction with diversity enhancement provided the best robustness results while also greatly enhancing the quality of solutions

The enhanced best performance algorithm for global optimization with applications.

Doctor of Philosophy in Computer Science. University of KwaZulu-Natal, Durban, 2016.Abstract available in PDF file