Shared active subspace for multivariate vector-valued functions

This paper proposes several approaches as baselines to compute a shared active subspace for multivariate vector-valued functions. The goal is to minimize the deviation between the function evaluations on the original space and those on the reconstructed one. This is done either by manipulating the gradients or the symmetric positive (semi-)definite (SPD) matrices computed from the gradients of each component function so as to get a single structure common to all component functions. These approaches can be applied to any data irrespective of the underlying distribution unlike the existing vector-valued approach that is constrained to a normal distribution. We test the effectiveness of these methods on five optimization problems. The experiments show that, in general, the SPD-level methods are superior to the gradient-level ones, and are close to the vector-valued approach in the case of a normal distribution. Interestingly, in most cases it suffices to take the sum of the SPD matrices to identify the best shared active subspace

Heteroskedastic Gaussian processes for simulation experiments

International audienceAn increasing number of time-consuming simulators exhibit a complex noise structure that depends on the inputs. To conduct studies with limited budgets of evaluations, new surrogate methods are required to model simultaneously the mean and variance fields. To this end, we present recent advances in Gaussian process modeling with input-dependent noise. First, we describe a simple, yet efficient, joint modeling framework that rely on replication for both speed and accuracy. Then we tackle the issue of leveraging replication and exploration in a sequential manner for various goals, such as obtaining a globally accurate model, for optimization, contour finding, and active subspace estimation. We illustrate these on applications coming from epidemiology and inventory management

Replication or exploration? Sequential design for stochastic simulation experiments

We investigate the merits of replication, and provide methods for optimal design (including replicates), with the goal of obtaining globally accurate emulation of noisy computer simulation experiments. We first show that replication can be beneficial from both design and computational perspectives, in the context of Gaussian process surrogate modeling. We then develop a lookahead based sequential design scheme that can determine if a new run should be at an existing input location (i.e., replicate) or at a new one (explore). When paired with a newly developed heteroskedastic Gaussian process model, our dynamic design scheme facilitates learning of signal and noise relationships which can vary throughout the input space. We show that it does so efficiently, on both computational and statistical grounds. In addition to illustrative synthetic examples, we demonstrate performance on two challenging real-data simulation experiments, from inventory management and epidemiology.Comment: 34 pages, 9 figure

A portfolio approach to massively parallel Bayesian optimization

One way to reduce the time of conducting optimization studies is to evaluate designs in parallel rather than just one-at-a-time. For expensive-to-evaluate black-boxes, batch versions of Bayesian optimization have been proposed. They work by building a surrogate model of the black-box that can be used to select the designs to evaluate efficiently via an infill criterion. Still, with higher levels of parallelization becoming available, the strategies that work for a few tens of parallel evaluations become limiting, in particular due to the complexity of selecting more evaluations. It is even more crucial when the black-box is noisy, necessitating more evaluations as well as repeating experiments. Here we propose a scalable strategy that can keep up with massive batching natively, focused on the exploration/exploitation trade-off and a portfolio allocation. We compare the approach with related methods on deterministic and noisy functions, for mono and multiobjective optimization tasks. These experiments show similar or better performance than existing methods, while being orders of magnitude faster

A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization

International audienceBayesian Optimization, the application of Bayesian function approximation to finding optima of expensive functions, has exploded in popularity in recent years. In particular, much attention has been paid to improving its efficiency on problems with many parameters to optimize. This attention has trickled down to the workhorse of high dimensional BO, high dimensional Gaussian process regression, which is also of independent interest. The great flexibility that the Gaussian process prior implies is a boon when modeling complicated, low dimensional surfaces but simply says too little when dimension grows too large. A variety of structural model assumptions have been tested to tame high dimensions, from variable selection and additive decomposition to low dimensional embeddings and beyond. Most of these approaches in turn require modifications of the acquisition function optimization strategy as well. Here we review the defining structural model assumptions and discuss the benefits and drawbacks of these approaches in practice

A game theoretic perspective on Bayesian multi-objective optimization

This chapter addresses the question of how to efficiently solve many-objective optimization problems in a computationally demanding black-box simulation context. We shall motivate the question by applications in machine learning and engineering, and discuss specific harsh challenges in using classical Pareto approaches when the number of objectives is four or more. Then, we review solutions combining approaches from Bayesian optimization, e.g., with Gaussian processes, and concepts from game theory like Nash equilibria, Kalai-Smorodinsky solutions and detail extensions like Nash-Kalai-Smorodinsky solutions. We finally introduce the corresponding algorithms and provide some illustrating results

CAD-Consistent Aerodynamic Design via the Isogeometric Paradigm

International audienceThis work aims at proposing an aerodynamic design optimization methodology entirely based on Computer-Aided Design (CAD) representations, yielding a fully integrated geometry-simulation-optimization framework. Specifically, the geometry to optimize is defined thanks to CAD standards, like Non-Uniform Rational B-Splines (NURBS); the resolution of compressible Euler / Navier-Stokes equation is achieved using a Discontinuous Galerkin (DG) method based on rational Bézier elements with adaptive mesh refinement, which is geometrically exact with respect to CAD boundaries; The global optimization of CAD control points is carried out using a Bayesian Optimization method. The whole design chain is therefore consistent with respect to the CAD geometry. The proposed approach is demonstrated for the optimization of an airfoil in transonic regime

Geometrically Consistent Aerodynamic Optimization using an Isogeometric Discontinuous Galerkin Method

International audienceThe objective of the current work is to define a design optimization methodology in aerodynamics, in which all numerical components are based on a unique geometrical representation, consistent with Computer-Aided Design (CAD) standards. In particular, the design is parameterized by Non-Uniform Rational B-Splines (NURBS), the computational domain is automatically constructed using rational Bézier elements extracted from NURBS boundaries without any approximation and the resolution of the flow equations relies on an adaptive Discontinuous Galerkin (DG) method based on rational representations. A Bayesian framework is used to optimize NURBS control points, in a single-or multi-objective, constrained, global optimization framework. The resulting methodology is therefore fully CAD-consistent, high-order in space and time, includes local adaption and shock capturing capabilities, and exhibits high parallelization performance. The proposed methods are described in details and their properties are established. Finally, two design optimization problems are provided as illustrations: the shape optimization of an airfoil in transonic regime, for drag reduction with lift constraint, and the multi-objective optimization of the control law of a morphing airfoil in subsonic regime, regarding the time-averaged lift, the minimum instantaneous lift and the energy consumption