Enhancement of Multiobjective Hierarchical Bayesian Optimization Algorithm using Sporadic Model Building

This paper describes and analyzes the efficiency enhancement of Multiobjective hierarchical Bayesian Optimization Algorithm (mohBOA) by using Sporadic Model Building (SMB). Firstly, Multiobjective hierarchical Bayesian Optimization Algorithm is shortly described. Secondly, sporadic model building is presented. Using sporadic model building, the structure of a probabilistic model is updated once every few iterations, whereas in the remaining iterations only model parameters (conditional and marginal probabilities) are updated. Since the time of learning the structure of a model is much longer than the time of updating model parameters, sporadic model building decreases the total time complexity of model building. The results of experiments show that the theoretical predictions about using sporadic model building to the enhancement of mohBOA are true. Finally, short discussion about the results of experiments is added

Enhancement of Multiobjective Hierarchical Bayesian Optimization Algorithm using Sporadic Model Building

This paper describes and analyzes the efficiency enhancement of Multiobjective hierarchical Bayesian Optimization Algorithm (mohBOA) by using Sporadic Model Building (SMB). Firstly, Multiobjective hierarchical Bayesian Optimization Algorithm is shortly described. Secondly, sporadic model building is presented. Using sporadic model building, the structure of a probabilistic model is updated once every few iterations, whereas in the remaining iterations only model parameters (conditional and marginal probabilities) are updated. Since the time of learning the structure of a model is much longer than the time of updating model parameters, sporadic model building decreases the total time complexity of model building. The results of experiments show that the theoretical predictions about using sporadic model building to the enhancement of mohBOA are true. Finally, short discussion about the results of experiments is added

Dependency structure matrix, genetic algorithms, and effective recombination

In many different fields, researchers are often confronted by problems arising from complex systems. Simple heuristics or even enumeration works quite well on small and easy problems; however, to efficiently solve large and difficult problems, proper decomposition is the key. In this paper, investigating and analyzing interactions between components of complex systems shed some light on problem decomposition. By recognizing three bare-bones interactions-modularity, hierarchy, and overlap, facet-wise models arc developed to dissect and inspect problem decomposition in the context of genetic algorithms. The proposed genetic algorithm design utilizes a matrix representation of an interaction graph to analyze and explicitly decompose the problem. The results from this paper should benefit research both technically and scientifically. Technically, this paper develops an automated dependency structure matrix clustering technique and utilizes it to design a model-building genetic algorithm that learns and delivers the problem structure. Scientifically, the explicit interaction model describes the problem structure very well and helps researchers gain important insights through the explicitness of the procedure.This work was sponsored by Taiwan National Science Council under grant NSC97- 2218-E-002-020-MY3, U.S. Air Force Office of Scientific Research, Air Force Material Command, USAF, under grants FA9550-06-1-0370 and FA9550-06-1-0096, U.S. National Science Foundation under CAREER grant ECS-0547013, ITR grant DMR-03-25939 at Materials Computation Center, grant ISS-02-09199 at US National Center for Supercomputing Applications, UIUC, and the Portuguese Foundation for Science and Technology under grants SFRH/BD/16980/2004 and PTDC/EIA/67776/2006

Combining model-based and genetics-based offspring generation for multi-objective optimization using a convergence criterion

Abstract — In our previous work [1], it has been shown that the performance of evolutionary multi-objective algorithms can be greatly enhanced if the regularity in the distribution of Pareto-optimal solutions is taken advantage using a probabilistic model. This paper suggests a new hybrid multi-objective evolutionary algorithm by introducing a convergence based criterion to determine when the model-based method and when the genetics-based method should be used to generate offspring in each generation. The basic idea is that the genetics-based method, i.e., crossover and mutation, should be used when the population is far away from the Pareto front and no obvious regularity in population distribution can be observed. When the population moves towards the Pareto front, the distribution of the individuals will show increasing regularity and in this case, the model-based method should be used to generate offspring. The proposed hybrid method is verified on widely used test problems and our simulation results show that the method is effective in achieving Pareto-optimal solutions compared to two state-of-the-art evolutionary multi-objective algorithms: NSGA-II and SPEA2, and our pervious method in [1]. I

Multi-objective gene-pool optimal mixing evolutionary algorithms

The recently introduced Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA), with a lean, but sufficient, linkage model and an efficient variation operator, has been shown to be a robust and efficient methodology for solving single objective (SO) optimization problems with superior performance compared to classic genetic algorithms (GAs) and estimation-of-distribution algorithms (EDAs). In this paper, we bring the strengths of GOMEAs to the multi-objective (MO) optimization realm. To this end, we modify the linkage learning procedure and the variation operator of GOMEAs to better suit the need of finding the whole Pareto-optimal front rather than a single best solution. Based on state-of-the-art studies on MOEAs, we further pinpoint and incorporate two other essential components for a scalable MO optimizer. First, the use of an elitist archive is beneficial for keeping track of non-dominated solutions when the main population size is limited. Second, clustering can be crucial if different parts of the Pareto-optimal front need to be handled differently. By combining these elements, we construct a multi-objective GOMEA (MO-GOMEA). Experimental results on various MO optimization problems confirm the capability and scalability of our MO-GOMEA that compare favorably with those of the well-known GA NSGA-II and the more recently introduced EDA mohBOA

MONEDA: scalable multi-objective optimization with a neural network-based estimation of distribution algorithm

The Extension Of Estimation Of Distribution Algorithms (Edas) To The Multiobjective Domain Has Led To Multi-Objective Optimization Edas (Moedas). Most Moedas Have Limited Themselves To Porting Single-Objective Edas To The Multi-Objective Domain. Although Moedas Have Proved To Be A Valid Approach, The Last Point Is An Obstacle To The Achievement Of A Significant Improvement Regarding "Standard" Multi-Objective Optimization Evolutionary Algorithms. Adapting The Model-Building Algorithm Is One Way To Achieve A Substantial Advance. Most Model-Building Schemes Used So Far By Edas Employ Off-The-Shelf Machine Learning Methods. However, The Model-Building Problem Has Particular Requirements That Those Methods Do Not Meet And Even Evade. The Focus Of This Paper Is On The Model- Building Issue And How It Has Not Been Properly Understood And Addressed By Most Moedas. We Delve Down Into The Roots Of This Matter And Hypothesize About Its Causes. To Gain A Deeper Understanding Of The Subject We Propose A Novel Algorithm Intended To Overcome The Draw-Backs Of Current Moedas. This New Algorithm Is The Multi-Objective Neural Estimation Of Distribution Algorithm (Moneda). Moneda Uses A Modified Growing Neural Gas Network For Model-Building (Mb-Gng). Mb-Gng Is A Custom-Made Clustering Algorithm That Meets The Above Demands. Thanks To Its Custom-Made Model-Building Algorithm, The Preservation Of Elite Individuals And Its Individual Replacement Scheme, Moneda Is Capable Of Scalably Solving Continuous Multi-Objective Optimization Problems. It Performs Better Than Similar Algorithms In Terms Of A Set Of Quality Indicators And Computational Resource Requirements.This work has been funded in part by projects CNPq BJT 407851/2012-7, FAPERJ APQ1 211.451/2015, MINECO TEC2014-57022-C2-2-R and TEC2012-37832-C02-01

Using Prior Knowledge and Learning from Experience in Estimation of Distribution Algorithms

Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. One of the primary advantages of EDAs over many other stochastic optimization techniques is that after each run they leave behind a sequence of probabilistic models describing useful decompositions of the problem. This sequence of models can be seen as a roadmap of how the EDA solves the problem. While this roadmap holds a great deal of information about the problem, until recently this information has largely been ignored. My thesis is that it is possible to exploit this information to speed up problem solving in EDAs in a principled way. The main contribution of this dissertation will be to show that there are multiple ways to exploit this problem-specific knowledge. Most importantly, it can be done in a principled way such that these methods lead to substantial speedups without requiring parameter tuning or hand-inspection of models