2,404 research outputs found
Self-Adaptive Surrogate-Assisted Covariance Matrix Adaptation Evolution Strategy
This paper presents a novel mechanism to adapt surrogate-assisted
population-based algorithms. This mechanism is applied to ACM-ES, a recently
proposed surrogate-assisted variant of CMA-ES. The resulting algorithm,
saACM-ES, adjusts online the lifelength of the current surrogate model (the
number of CMA-ES generations before learning a new surrogate) and the surrogate
hyper-parameters. Both heuristics significantly improve the quality of the
surrogate model, yielding a significant speed-up of saACM-ES compared to the
ACM-ES and CMA-ES baselines. The empirical validation of saACM-ES on the
BBOB-2012 noiseless testbed demonstrates the efficiency and the scalability
w.r.t the problem dimension and the population size of the proposed approach,
that reaches new best results on some of the benchmark problems.Comment: Genetic and Evolutionary Computation Conference (GECCO 2012) (2012
Bayesian Quality-Diversity approaches for constrained optimization problems with mixed continuous, discrete and categorical variables
Complex engineering design problems, such as those involved in aerospace,
civil, or energy engineering, require the use of numerically costly simulation
codes in order to predict the behavior and performance of the system to be
designed. To perform the design of the systems, these codes are often embedded
into an optimization process to provide the best design while satisfying the
design constraints. Recently, new approaches, called Quality-Diversity, have
been proposed in order to enhance the exploration of the design space and to
provide a set of optimal diversified solutions with respect to some feature
functions. These functions are interesting to assess trade-offs. Furthermore,
complex engineering design problems often involve mixed continuous, discrete,
and categorical design variables allowing to take into account technological
choices in the optimization problem. In this paper, a new Quality-Diversity
methodology based on mixed continuous, discrete and categorical Bayesian
optimization strategy is proposed. This approach allows to reduce the
computational cost with respect to classical Quality - Diversity approaches
while dealing with discrete choices and constraints. The performance of the
proposed method is assessed on a benchmark of analytical problems as well as on
an industrial design optimization problem dealing with aerospace systems
Landscape Analysis for Surrogate Models in the Evolutionary Black-Box Context
Surrogate modeling has become a valuable technique for black-box optimization
tasks with expensive evaluation of the objective function. In this paper, we
investigate the relationship between the predictive accuracy of surrogate
models and features of the black-box function landscape. We also study
properties of features for landscape analysis in the context of different
transformations and ways of selecting the input data. We perform the landscape
analysis of a large set of data generated using runs of a surrogate-assisted
version of the Covariance Matrix Adaptation Evolution Strategy on the noiseless
part of the Comparing Continuous Optimisers benchmark function testbed.Comment: 25 pages main article, 28 pages supplementary material, 3 figures,
currently under review at Evolutionary Computation journa
A new Taxonomy of Continuous Global Optimization Algorithms
Surrogate-based optimization, nature-inspired metaheuristics, and hybrid
combinations have become state of the art in algorithm design for solving
real-world optimization problems. Still, it is difficult for practitioners to
get an overview that explains their advantages in comparison to a large number
of available methods in the scope of optimization. Available taxonomies lack
the embedding of current approaches in the larger context of this broad field.
This article presents a taxonomy of the field, which explores and matches
algorithm strategies by extracting similarities and differences in their search
strategies. A particular focus lies on algorithms using surrogates,
nature-inspired designs, and those created by design optimization. The
extracted features of components or operators allow us to create a set of
classification indicators to distinguish between a small number of classes. The
features allow a deeper understanding of components of the search strategies
and further indicate the close connections between the different algorithm
designs. We present intuitive analogies to explain the basic principles of the
search algorithms, particularly useful for novices in this research field.
Furthermore, this taxonomy allows recommendations for the applicability of the
corresponding algorithms.Comment: 35 pages total, 28 written pages, 4 figures, 2019 Reworked Versio
Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution
A master face is a face image that passes face-based identity authentication
for a high percentage of the population. These faces can be used to
impersonate, with a high probability of success, any user, without having
access to any user information. We optimize these faces for 2D and 3D face
verification models, by using an evolutionary algorithm in the latent embedding
space of the StyleGAN face generator. For 2D face verification, multiple
evolutionary strategies are compared, and we propose a novel approach that
employs a neural network to direct the search toward promising samples, without
adding fitness evaluations. The results we present demonstrate that it is
possible to obtain a considerable coverage of the identities in the LFW or RFW
datasets with less than 10 master faces, for six leading deep face recognition
systems. In 3D, we generate faces using the 2D StyleGAN2 generator and predict
a 3D structure using a deep 3D face reconstruction network. When employing two
different 3D face recognition systems, we are able to obtain a coverage of
40%-50%. Additionally, we present the generation of paired 2D RGB and 3D master
faces, which simultaneously match 2D and 3D models with high impersonation
rates.Comment: accepted for publication in IEEE Transactions on Biometrics,
Behavior, and Identity Science (TBIOM). This paper extends arXiv:2108.01077
that was accepted to IEEE FG 202
Cooperative Coevolution for Non-Separable Large-Scale Black-Box Optimization: Convergence Analyses and Distributed Accelerations
Given the ubiquity of non-separable optimization problems in real worlds, in
this paper we analyze and extend the large-scale version of the well-known
cooperative coevolution (CC), a divide-and-conquer optimization framework, on
non-separable functions. First, we reveal empirical reasons of why
decomposition-based methods are preferred or not in practice on some
non-separable large-scale problems, which have not been clearly pointed out in
many previous CC papers. Then, we formalize CC to a continuous game model via
simplification, but without losing its essential property. Different from
previous evolutionary game theory for CC, our new model provides a much simpler
but useful viewpoint to analyze its convergence, since only the pure Nash
equilibrium concept is needed and more general fitness landscapes can be
explicitly considered. Based on convergence analyses, we propose a hierarchical
decomposition strategy for better generalization, as for any decomposition
there is a risk of getting trapped into a suboptimal Nash equilibrium. Finally,
we use powerful distributed computing to accelerate it under the multi-level
learning framework, which combines the fine-tuning ability from decomposition
with the invariance property of CMA-ES. Experiments on a set of
high-dimensional functions validate both its search performance and scalability
(w.r.t. CPU cores) on a clustering computing platform with 400 CPU cores
Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of the acquisition function, a task complicated by the fact that acquisition functions tend to be non-convex and thus nontrivial to optimise. In this paper, we undertake a comprehensive empirical study of approaches to maximise the acquisition function. Additionally, by deriving novel, yet mathematically equivalent, compositional forms for popular acquisition functions, we recast the maximisation task as a compositional optimisation problem, allowing us to benefit from the extensive literature in this field. We highlight the empirical advantages of the compositional approach to acquisition function maximisation across 3958 individual experiments comprising synthetic optimisation tasks as well as tasks from Bayesmark. Given the generality of the acquisition function maximisation subroutine, we posit that the adoption of compositional optimisers has the potential to yield performance improvements across all domains in which Bayesian optimisation is currently being applied. An open-source implementation is made available at https://github.com/huawei-noah/noah-research/tree/CompBO/BO/HEBO/CompBO
- …