Discovering the Elite Hypervolume by Leveraging Interspecies Correlation

Evolution has produced an astonishing diversity of species, each filling a different niche. Algorithms like MAP-Elites mimic this divergent evolutionary process to find a set of behaviorally diverse but high-performing solutions, called the elites. Our key insight is that species in nature often share a surprisingly large part of their genome, in spite of occupying very different niches; similarly, the elites are likely to be concentrated in a specific "elite hypervolume" whose shape is defined by their common features. In this paper, we first introduce the elite hypervolume concept and propose two metrics to characterize it: the genotypic spread and the genotypic similarity. We then introduce a new variation operator, called "directional variation", that exploits interspecies (or inter-elites) correlations to accelerate the MAP-Elites algorithm. We demonstrate the effectiveness of this operator in three problems (a toy function, a redundant robotic arm, and a hexapod robot).Comment: In GECCO 201

Development and Integration of Geometric and Optimization Algorithms for Packing and Layout Design

The research work presented in this dissertation focuses on the development and application of optimization and geometric algorithms to packing and layout optimization problems. As part of this research work, a compact packing algorithm, a physically-based shape morphing algorithm, and a general purpose constrained multi-objective optimization algorithm are proposed. The compact packing algorithm is designed to pack three-dimensional free-form objects with full rotational freedom inside an arbitrary enclosure such that the packing efficiency is maximized. The proposed compact packing algorithm can handle objects with holes or cavities and its performance does not degrade significantly with the increase in the complexity of the enclosure or the objects. It outputs the location and orientation of all the objects, the packing sequence, and the packed configuration at the end of the packing operation. An improved layout algorithm that works with arbitrary enclosure geometry is also proposed. Different layout algorithms for the SAE and ISO luggage are proposed that exploit the unique characteristics of the problem under consideration. Several heuristics to improve the performance of the packing algorithm are also proposed. The proposed compact packing algorithm is benchmarked on a wide variety of synthetic and hypothetical problems and is shown to outperform other similar approaches. The physically-based shape morphing algorithm proposed in this dissertation is specifically designed for packing and layout applications, and thus it augments the compact packing algorithm. The proposed shape morphing algorithm is based on a modified mass-spring system which is used to model the morphable object. The shape morphing algorithm mimics a quasi-physical process similar to the inflation/deflation of a balloon filled with air. The morphing algorithm starts with an initial manifold geometry and morphs it to obtain a desired volume such that the obtained geometry does not interfere with the objects surrounding it. Several modifications to the original mass-spring system and to the underlying physics that governs it are proposed to significantly speed-up the shape morphing process. Since the geometry of a morphable object continuously changes during the morphing process, most collision detection algorithms that assume the colliding objects to be rigid cannot be used efficiently. And therefore, a general-purpose surface collision detection algorithm is also proposed that works with deformable objects and does not require any preprocessing. Many industrial design problems such as packing and layout optimization are computationally expensive, and a faster optimization algorithm can reduce the number of iterations (function evaluations) required to find the satisfycing solutions. A new multi-objective optimization algorithm namely Archive-based Micro Genetic Algorithm (AMGA2) is presented in this dissertation. Improved formulation for various operators used by the AMGA2 such as diversity preservation techniques, genetic variation operators, and the selection mechanism are also proposed. The AMGA2 also borrows several concepts from mathematical sciences to improve its performance and benefits from the existing literature in evolutionary optimization. A comprehensive benchmarking and comparison of AMGA2 with other state-of-the-art optimization algorithms on a wide variety of mathematical problems gleaned from literature demonstrates the superior performance of AMGA2. Thus, the research work presented in this dissertation makes contributions to the development and application of optimization and geometric algorithms

Optimal Scalarizations for Sublinear Hypervolume Regret

Scalarization is a general technique that can be deployed in any multiobjective setting to reduce multiple objectives into one, such as recently in RLHF for training reward models that align human preferences. Yet some have dismissed this classical approach because linear scalarizations are known to miss concave regions of the Pareto frontier. To that end, we aim to find simple non-linear scalarizations that can explore a diverse set of $k$ objectives on the Pareto frontier, as measured by the dominated hypervolume. We show that hypervolume scalarizations with uniformly random weights are surprisingly optimal for provably minimizing the hypervolume regret, achieving an optimal sublinear regret bound of $O(T^{-1/k})$, with matching lower bounds that preclude any algorithm from doing better asymptotically. As a theoretical case study, we consider the multiobjective stochastic linear bandits problem and demonstrate that by exploiting the sublinear regret bounds of the hypervolume scalarizations, we can derive a novel non-Euclidean analysis that produces improved hypervolume regret bounds of $\tilde{O}( d T^{-1/2} + T^{-1/k})$. We support our theory with strong empirical performance of using simple hypervolume scalarizations that consistently outperforms both the linear and Chebyshev scalarizations, as well as standard multiobjective algorithms in bayesian optimization, such as EHVI.Comment: ICML 2023 Worksho

A hybrid multi-objective evolutionary approach for optimal path planning of a hexapod robot

Hexapod robots are six-legged robotic systems, which have been widely investigated in the literature for various applications including exploration, rescue, and surveillance. Designing hexapod robots requires to carefully considering a number of different aspects. One of the aspects that require careful design attention is the planning of leg trajectories. In particular, given the high demand for fast motion and high-energy autonomy it is important to identify proper leg operation paths that can minimize energy consumption while maximizing the velocity of the movements. In this frame, this paper presents a preliminary study on the application of a hybrid multi-objective optimization approach for the computer-aided optimal design of a legged robot. To assess the methodology, a kinematic and dynamic model of a leg of a hexapod robot is proposed as referring to the main design parameters of a leg. Optimal criteria have been identified for minimizing the energy consumption and efficiency as well as maximizing the walking speed and the size of obstacles that a leg can overtake. We evaluate the performance of the hybrid multi-objective evolutionary approach to explore the design space and provide a designer with an optimal setting of the parameters. Our simulations demonstrate the effectiveness of the hybrid approach by obtaining improved Pareto sets of trade-off solutions as compared with a standard evolutionary algorithm. Computational costs show an acceptable increase for an off-line path planner. © Springer International Publishing Switzerland 2016

Automated discovery of trade-off between utility, privacy and fairness in machine learning models

Machine learning models are deployed as a central component in decision making and policy operations with direct impact on individuals' lives. In order to act ethically and comply with government regulations, these models need to make fair decisions and protect the users' privacy. However, such requirements can come with decrease in models' performance compared to their potentially biased, privacy-leaking counterparts. Thus the trade-off between fairness, privacy and performance of ML models emerges, and practitioners need a way of quantifying this trade-off to enable deployment decisions. In this work we interpret this trade-off as a multi-objective optimization problem, and propose PFairDP, a pipeline that uses Bayesian optimization for discovery of Pareto-optimal points between fairness, privacy and utility of ML models. We show how PFairDP can be used to replicate known results that were achieved through manual constraint setting process. We further demonstrate effectiveness of PFairDP with experiments on multiple models and datasets.Comment: 3rd Workshop on Bias and Fairness in AI (BIAS), ECML 202

Multi-objective optimization with a Gaussian PSO algorithm

Particle Swarm Optimization es una heurística popular usada para resolver adecuada y efectivamente problemas mono-objetivo. En este artículo, presentamos una primera adaptación de esta heurística para tratar problemas multi-objetivo sin restricciones. La propuesta (llamada G-MOPSO) incorpora una actualización Gaussiana, dominancia Pareto, una política elitista, un archivo externo y un shake-mecanismo para mantener la diversidad. Para validar nuestro algoritmo, usamos cuatro funciones de prueba bien conocidas, con diferentes características. Los resultados preliminares son comparados con los valores obtenidos por un algoritmo evolutivo multi-objetivo representativo del estado del arte en el área: NSGA-II. También comparamos los resultados con los obtenidos por OMOPSO, un algoritmo multi-objetivo basado en la heurística PSO. La performance de nuestra propuesta es comparable con la de NSGA-II y supera a la de OMOPSOParticle Swarm Optimization is a popular heuristic used to solve suitably and effectively mono-objective problems. In this paper, we present an adaptation of this heuristic to treat unconstrained multi-objective problems. The proposed approach (called G-MOPSO) incorporates a Gaussian update of individuals, Pareto dominance, an elitist policy, and a shake-mechanism to maintain diversity. In order to validate our algorithm, we use four well-known test functions with different characteristics. Preliminary results are compared with respect to those obtained by a multi-objective evolutionary algorithm representative of the state-of-the-art: NSGA-II. We also compare the results with those obtained by OMOPSO, a multi-objective PSO based algorithm. The performance of our approach is comparable with the NSGA-II and outperforms the OMOPSO.Workshop de Agentes y Sistemas Inteligentes (WASI)Red de Universidades con Carreras en Informática (RedUNCI

A Bayesian approach to constrained single- and multi-objective optimization

This article addresses the problem of derivative-free (single- or multi-objective) optimization subject to multiple inequality constraints. Both the objective and constraint functions are assumed to be smooth, non-linear and expensive to evaluate. As a consequence, the number of evaluations that can be used to carry out the optimization is very limited, as in complex industrial design optimization problems. The method we propose to overcome this difficulty has its roots in both the Bayesian and the multi-objective optimization literatures. More specifically, an extended domination rule is used to handle objectives and constraints in a unified way, and a corresponding expected hyper-volume improvement sampling criterion is proposed. This new criterion is naturally adapted to the search of a feasible point when none is available, and reduces to existing Bayesian sampling criteria---the classical Expected Improvement (EI) criterion and some of its constrained/multi-objective extensions---as soon as at least one feasible point is available. The calculation and optimization of the criterion are performed using Sequential Monte Carlo techniques. In particular, an algorithm similar to the subset simulation method, which is well known in the field of structural reliability, is used to estimate the criterion. The method, which we call BMOO (for Bayesian Multi-Objective Optimization), is compared to state-of-the-art algorithms for single- and multi-objective constrained optimization