Running Up Those Hills: Multi-Modal Search with the Niching Migratory Multi-Swarm Optimiser

Copyright © 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.2014 IEEE Congress on Evolutionary Computation, Beijing, China, 6 - 11 July 2014The codebase for this paper, containing the NMMSO algorithm, is at https://github.com/fieldsend/ieee_cec_2014_nmmsoWe present a new multi-modal evolutionary optimiser, the niching migratory multi-swarm optimiser (NMMSO), which dynamically manages many particle swarms. These sub-swarms are concerned with optimising separate local modes, and employ measures to allow swarm elements to migrate away from their parent swarm if they are identified as being in the vicinity of a separate peak, and to merge swarms together if they are identified as being concerned with the same peak. We employ coarse peak identification to facilitate the mode identification required. Swarm members are not constrained to particular sub- regions of the parameter space, however members are initialised in the vicinity of a swarm’s local mode estimate. NMMSO is shown to cope with a range of problem types, and to produce results competitive with the state-of-the-art on the CEC 2013 multi-modal optimisation competition test problems, providing new benchmark results in the field

Region-based memetic algorithm with archive for multimodal optimisation.

In this paper we propose a specially designed memetic algorithm for multimodal optimisation problems. The proposal uses a niching strategy, called region-based niching strategy, that divides the search space in predefined and indexable hypercubes with decreasing size, called regions. This niching technique allows our proposal to keep high diversity in the population, and to keep the most promising regions in an external archive. The most promising solutions are improved with a local search method and also stored in the archive. The archive is used as an index to effiently prevent further exploration of these areas with the evolutionary algorithm. The resulting algorithm, called Region-based Memetic Algorithm with Archive, is tested on the benchmark proposed in the special session and competition on niching methods for multimodal function optimisation of the Congress on Evolutionary Computation in 2013. The results obtained show that the region-based niching strategy is more efficient than the classical niching strategy called clearing and that the use of the archive as restrictive index significantly improves the exploration efficiency of the algorithm. The proposal achieves better exploration and accuracy than other existing techniques

Using an adaptive collection of local evolutionary algorithms for multi-modal problems

The codebase for this paper, containing LSEA_EA algorithm, is available at https://github.com/fieldsend/soft_computing_2014_lsea_eaMulti-modality can cause serious problems for many optimisers, often resulting convergence to sub-optimal modes. Even when this is not the case, it is often useful to locate and memorise a range of modes in the design space. This is because “optimal" decision parameter combinations may not actually be feasible when moving from a mathematical model emulating the real problem, to engineering an actual solution, making a range of disparate modal solutions of practical use. This paper builds upon our work on the use of a collection of localised search algorithms for niche/mode discovery which we presented at UKCI 2013 when using a collection of surrogate models to guide mode search. Here we present the results of using a collection of exploitative local evolutionary algorithms (EAs) within the same general framework. The algorithm dynamically adjusts its population size according to the number of regions it encounters that it believes contain a mode, and uses localised EAs to guide the mode exploitation. We find that using a collection of localised EAs, which have limited communication with each other, produces competitive results with the current state-of-the-art multimodal optimisation approaches on the CEC 2013 benchmark functions

Control de diversidad en algoritmos genéticos utilizando estrategias multimodales

An optimization process is a kind of process that systematically comes up with solutions that are better than a previous solution used before. Optimization algorithms are used to find solutions which are optimal or near-optimal with respect to some goals, to evaluate design tradeoffs, to assess control systems, to find patterns in data, and to find the optimum values (local or global) of mathematical functions. A genetic algorithm is one of the optimization techniques. In this way, a heuristic search that is inspired by Charles Darwin’s theory of natural evolution. This algorithm reflects the process of natural selection where the fittest individuals are selected for reproduction in order to produce offspring of the next generation which are population algorithms that emulate behavior similar to Darwinian natural selection. Taking into account these issues, this article shows the performance of a genetic algorithm designed, which allows to find several minimums within a function from the control of population diversity. To perform the tests, the algorithm with four different functions was used, with the particularity of having several minima with the same value. Proposed strategy was compared with a conventional genetic algorithm, the result was the conventional one can only find some of the minimums of the function and sometimes only one, while the proposal finds most of the minimumsLa búsqueda de la mejor solución posible a un problema se realiza con procesos de optimización, explorando los valores de los parámetros para los que cierta función objetivo tiene un valor óptimo (local o global). Entre las técnicas de optimización se encuentran los algoritmos genéticos, los cuales son de tipo poblacional o que emulan un comportamiento similar al de la selección natural Darwiniana. Este artículo muestra el desempeño de un algoritmo genético que permite encontrar varios mínimos dentro de una función a partir del control de diversidad de la población. Para realizar las pruebas se utilizó el algoritmo con cuatro diferentes funciones, con la particularidad de tener varios mínimos con el mismo valor. Se comparó esta estrategia propuesta con un algoritmo genético convencional, encontrándose que el convencional solo puede hallar algunos de los mínimos de la función —y en ocasiones solo uno— en tanto que la propuesta encuentra la mayoría de los mínimo

Structural optimization using evolutionary multimodal and bilevel optimization techniques

This research aims to investigate the multimodal properties of structural optimization using techniques from the field of evolutionary computation, specifically niching and bilevel techniques. Truss design is a well-known structural optimization problem which has important practical applications in many fields. Truss design problems are typically multimodal by nature, meaning that it offers multiple equally good design solutions with respect to the topology and/or sizes of the members, but they are evaluated to have similar or equally good objective function values. From a practical standpoint, it is desirable to find as many alternative designs as possible, rather than finding a single design, as often practiced. Niching is an intuitive way of finding multiple optimal solutions in a single optimization run. Literature shows that existing niching methods are largely designed for handling continuous optimization problems. There does not exist a well-studied niching method for constrained discrete optimization problems like truss design problems. In addition, there are no well-defined multimodal discrete benchmark problems that can be used to evaluate the reliability and robustness of such a niching method. This thesis fills the identified research gaps by means of five major contributions. In the first contribution, we design a test suite for producing a diverse set of challenging multimodal discrete benchmark problems, which can be used for evaluating the discrete niching methods. In the second contribution, we develop a binary speciation-based PSO (B-SPSO) niching method using the concept of speciation in nature along with the binary PSO (BPSO). The results show that the proposed multimodal discrete benchmark problems are useful for the evaluation of the discrete niching methods like B-SPSO. In light of this study, a time-varying transfer function based binary PSO (TVT-BPSO) is developed for the B-SPSO which is the third contribution of this thesis. We propose this TVT-BPSO for maintaining a better balance between exploration/exploitation during the search process of the BPSO. The results show that the TVT-BPSO outperforms the state-of-the-art discrete optimization methods on the large-scale 0-1 knapsack problems. The fourth contribution is to consider and formulate the truss design problem as a bilevel optimization problem. With this new formulation, truss topology can be optimized in the upper level, at the same time the size of that truss topology can be optimized in the lower level. The proposed bilevel formulation is a precursor to the development of a bilevel niching method (Bi-NM) which constitutes the fifth contribution of this thesis. The proposed Bi-NM method performs niching at the upper level and a local search at the lower level to further refine the solutions. Extensive empirical studies are carried out to examine the accuracy, robustness, and efficiency of the proposed bilevel niching method in finding multiple topologies and their size solutions. Our results confirm that the proposed bilevel niching method is superior in all these three aspects over the state-of-the-art methods on several low to high-dimensional truss design problems

Efficient Algorithms for Computationally Expensive Multifidelity Optimization Problems

Multifidelity optimization problems refer to a class of problems where one is presented with a physical system or mathematical model that can be represented in different levels of fidelity. The term “fidelity” refers to the accuracy of representation, where higher fidelity estimates are more accurate and expensive, while lower fidelity estimates are inaccurate, albeit cheaper. Most common iterative solvers such as those employed in computational fluid dynamics (CFD), finite element analysis (FEA), computational electromagnetics (CEM) etc. can be run with different fine/course meshes or residual error thresholds to yield estimates in various fidelities. In the event an optimization exercise requires their use, it is possible to invoke analysis in various fidelities for different solutions during the course of search. Multifidelity optimization algorithms are the special class of algorithms that are able to deal with analysis in various levels of fidelity. In this thesis, two novel multifidelity optimization algorithms have been developed. The first is to deal with bilevel optimization problems and the second is to deal with robust optimization problems involving iterative solvers. Bilevel optimization problems are particularly challenging as the optimum of an upper level (UL) problem is sought subject to the optimality of a nested lower level (LL) problem. Due to the inherent nested nature, naive implementations consume very significant number of UL and LL evaluations. The proposed multifidelity approach controls the rigour of LL optimization exercise for any given UL solution during the course of search as opposed to undertaking exhaustive LL optimization for every UL solution. Robust optimization problems are yet another class of problems where numerous solutions need to be assessed since the intent is to identify solutions that have both good performance and is also insensitive to unavoidable perturbations in the variable values. Computing the latter metric requires evaluation of numerous solutions in the vicinity of the given solution and not all solutions are worthy of such computation. The proposed multifidelity approach considers pre-converged simulations as lower fidelity estimates and uses them to reduce the computational overhead. While multi-objective optimization problems have long been in existence, there has been limited attempts in the past to deal with problems where the objectives can be independently computed. For example, the weight of a structure and the maximum stress in the structure are two objectives that can be independently computed. For such classes of problems, an efficient algorithm should ideally evaluate either one or both objectives as opposed of always evaluating both objectives. A novel algorithm is introduced that is capable of selectively evaluating the objectives of the infill solutions. The approach exploits principles of non-dominance and sparse subset selection to facilitate decomposition and through maximization of probabilistic dominance (PD) measure, identifies the infill solutions. Thereafter, for each of these infill solutions, one or more objectives are evaluated based on evaluation status of its closest neighbor and the probability of improvement along each objective. Finally, there has been significant research interest in recent years to develop efficient algorithms to deal with multimodal, multi-objective optimization problems (MMOPs). Such problems are particulatly challenging as there is a need to identify well distributed and well converged solutions in the objective space along with diverse solutions in the variable space. Existing algorithms for MMOPs still require prohibitive number of function evaluations (often in several thousands). The algorithms are typically embedded with sophisticated, customized mechanisms that require additional parameters to manage the diversity and convergence in the variable and the objective spaces. A steady-state evolutionary algorithm is introduced in this thesis for solving MMOPs, with a simple design and no additional user-defined parameters that need tuning. All the developments listed above have been studied using well established benchmarks and real-world examples. The results have been compared with existing state-of-the-art approaches to substantiate the benefits

Meta-optimization of Bio-inspired Techniques for Object Recognition

Il riconoscimento di oggetti consiste nel trovare automaticamente un oggetto all'interno di un'immagine o in una sequenza video. Questo compito è molto importante in molti campi quali diagnosi mediche, assistenza di guida avanzata, visione artificiale, sorveglianza, realtà aumentata. Tuttavia, questo compito può essere molto impegnativo a causa di artefatti (dovuti al sistema di acquisizione, all'ambiente o ad altri effetti ottici quali prospettiva, variazioni di illuminazione, etc.) che possono influenzare l'aspetto anche di oggetti facili da identificare e ben definiti . Una possibile tecnica per il riconoscimento di oggetti consiste nell'utilizzare approcci basati su modello: in questo scenario viene creato un modello che rappresenta le proprietà dell'oggetto da individuare; poi, vengono generate possibili ipotesi sul posizionamento dell'oggetto, e il modello viene trasformato di conseguenza, fino a trovare la migliore corrispondenza con l'aspetto reale dell'oggetto. Per generare queste ipotesi in maniera intelligente, è necessario un buon algoritmo di ottimizzazione. Gli algoritmi di tipo bio-ispirati sono metodi di ottimizzazione che si basano su proprietà osservate in natura (quali cooperazione, evoluzione, socialità). La loro efficacia è stata dimostrata in molte attività di ottimizzazione, soprattutto in problemi di difficile soluzione, multi-modali e multi-dimensionali quali, per l'appunto, il riconoscimento di oggetti. Anche se queste euristiche sono generalmente efficaci, esse dipendono da molti parametri che influenzano profondamente le loro prestazioni; pertanto, è spesso richiesto uno sforzo significativo per capire come farle esprimere al massimo delle loro potenzialità. Questa tesi descrive un metodo per (i) individuare automaticamente buoni parametri per tecniche bio-ispirate, sia per un problema specifico che più di uno alla volta, e (ii) acquisire maggior conoscenza sul ruolo di un parametro in questi algoritmi. Inoltre, viene mostrato come le tecniche bio-ispirate possono essere applicate con successo in diversi ambiti nel riconoscimento di oggetti, e come è possibile migliorare ulteriormente le loro prestazioni mediante il tuning automatico dei loro parametri.Object recognition is the task of automatically finding a given object in an image or in a video sequence. This task is very important in many fields such as medical diagnosis, advanced driving assistance, image understanding, surveillance, virtual reality. Nevertheless, this task can be very challenging because of artefacts (related with the acquisition system, the environment or other optical effects like perspective, illumination changes, etc.) which may affect the aspect even of easy-to-identify and well-defined objects. A possible way to achieve object recognition is using model-based approaches: in this scenario a model (also called template) representing the properties of the target object is created; then, hypotheses on the position of the object are generated, and the model is transformed accordingly, until the best match with the actual appearance of the object is found. To generate these hypotheses intelligently, a good optimization algorithm is required. Bio-inspired techniques are optimization methods whose foundations rely on properties observed in nature (such as cooperation, evolution, emergence). Their effectiveness has been proved in many optimization tasks, especially in multi-modal, multi-dimensional hard problems like object recognition. Although these heuristics are generally effective, they depend on many parameters that strongly affect their performances; therefore, a significant effort must be spent to understand how to let them express their full potentialities. This thesis describes a method to (i) automatically find good parameters for bio-inspired techniques, both for a specific problem and for more than one at the same time, and (ii) acquire more knowledge of a parameter's role in such algorithms. Then, it shows how bio-inspired techniques can be successfully applied to different object recognition tasks, and how it is possible to further improve their performances by means of automatic parameter tuning

Adaptive Heterogeneous Multi-Population Cultural Algorithm

Optimization problems is a class of problems where the goal is to make a system as effective as possible. The goal of this research area is to design an algorithm to solve optimization problems effectively and efficiently. Being effective means that the algorithm should be able to find the optimal solution (or near optimal solutions), while efficiency refers to the computational effort required by the algorithm to find an optimal solution. In other words, an optimization algorithm should be able to find the optimal solution in an acceptable time. Therefore, the aim of this dissertation is to come up with a new algorithm which presents an effective as well as efficient performance. There are various kinds of algorithms proposed to deal with optimization problems. Evolutionary Algorithms (EAs) is a subset of population-based methods which are successfully applied to solve optimization problems. In this dissertation the area of evolutionary methods and specially Cultural Algorithms (CAs) are investigated. The results of this investigation reveal that there are some room for improving the existing EAs. Consequently, a number of EAs are proposed to deal with different optimization problems. The proposed EAs offer better performance compared to the state-of-the-art methods. The main contribution of this dissertation is to introduce a new architecture for optimization algorithms which is called Heterogeneous Multi-Population Cultural Algorithm (HMP-CA). The new architecture first incorporates a decomposition technique to divide the given problem into a number of sub-problems, and then it assigns the sub-problems to different local CAs to be optimized separately in parallel. In order to evaluate the proposed architecture, it is applied on numerical optimization problems. The evaluation results reveal that HMP-CA is fully effective such that it can find the optimal solution for every single run. Furthermore, HMP-CA outperforms the state-of-the-art methods by offering a more efficient performance. The proposed HMP-CA is further improved by incorporating an adaptive decomposition technique. The improved version which is called Adaptive HMP-CA (A-HMP-CA) is evaluated over large scale global optimization problems. The results of this evaluation show that HMP-CA significantly outperforms the state-of-the-art methods in terms of both effectiveness and efficiency