Differentiating the multipoint Expected Improvement for optimal batch design

This work deals with parallel optimization of expensive objective functions which are modeled as sample realizations of Gaussian processes. The study is formalized as a Bayesian optimization problem, or continuous multi-armed bandit problem, where a batch of q > 0 arms is pulled in parallel at each iteration. Several algorithms have been developed for choosing batches by trading off exploitation and exploration. As of today, the maximum Expected Improvement (EI) and Upper Confidence Bound (UCB) selection rules appear as the most prominent approaches for batch selection. Here, we build upon recent work on the multipoint Expected Improvement criterion, for which an analytic expansion relying on Tallis' formula was recently established. The computational burden of this selection rule being still an issue in application, we derive a closed-form expression for the gradient of the multipoint Expected Improvement, which aims at facilitating its maximization using gradient-based ascent algorithms. Substantial computational savings are shown in application. In addition, our algorithms are tested numerically and compared to state-of-the-art UCB-based batch-sequential algorithms. Combining starting designs relying on UCB with gradient-based EI local optimization finally appears as a sound option for batch design in distributed Gaussian Process optimization

Bayesian Quantile and Expectile Optimisation

Bayesian optimisation is widely used to optimise stochastic black box functions. While most strategies are focused on optimising conditional expectations, a large variety of applications require risk-averse decisions and alternative criteria accounting for the distribution tails need to be considered. In this paper, we propose new variational models for Bayesian quantile and expectile regression that are well-suited for heteroscedastic settings. Our models consist of two latent Gaussian processes accounting respectively for the conditional quantile (or expectile) and variance that are chained through asymmetric likelihood functions. Furthermore, we propose two Bayesian optimisation strategies, either derived from a GP-UCB or Thompson sampling, that are tailored to such models and that can accommodate large batches of points. As illustrated in the experimental section, the proposed approach clearly outperforms the state of the art

Future benefits and applications of intelligent on-board processing to VSAT services

The trends and roles of VSAT services in the year 2010 time frame are examined based on an overall network and service model for that period. An estimate of the VSAT traffic is then made and the service and general network requirements are identified. In order to accommodate these traffic needs, four satellite VSAT architectures based on the use of fixed or scanning multibeam antennas in conjunction with IF switching or onboard regeneration and baseband processing are suggested. The performance of each of these architectures is assessed and the key enabling technologies are identified

New Paradigms for Exploiting Parallel Experiments in Bayesian Optimization

Bayesian optimization (BO) is one of the most effective methods for closed-loop experimental design and black-box optimization. However, a key limitation of BO is that it is an inherently sequential algorithm (one experiment is proposed per round) and thus cannot directly exploit high-throughput (parallel) experiments. Diverse modifications to the BO framework have been proposed in the literature to enable exploitation of parallel experiments but such approaches are limited in the degree of parallelization that they can achieve and can lead to redundant experiments (thus wasting resources and potentially compromising performance). In this work, we present new parallel BO paradigms that exploit the structure of the system to partition the design space. Specifically, we propose an approach that partitions the design space by following the level sets of the performance function and an approach that exploits partially-separable structures of the performance function found. We conduct extensive numerical experiments using a reactor case study to benchmark the effectiveness of these approaches against a variety of state-of-the-art parallel algorithms reported in the literature. Our computational results show that our approaches significantly reduce the required search time and increase the probability of finding a global (rather than local) solution.Comment: 32 pages, 16 figures, 7 algorithm

Enabling Automated, Reliable and Efficient Aerodynamic Shape Optimization With Output-Based Adapted Meshes

Simulation-based aerodynamic shape optimization has been greatly pushed forward during the past several decades, largely due to the developments of computational fluid dynamics (CFD), geometry parameterization methods, mesh deformation techniques, sensitivity computation, and numerical optimization algorithms. Effective integration of these components has made aerodynamic shape optimization a highly automated process, requiring less and less human interference. Mesh generation, on the other hand, has become the main overhead of setting up the optimization problem. Obtaining a good computational mesh is essential in CFD simulations for accurate output predictions, which as a result significantly affects the reliability of optimization results. However, this is in general a nontrivial task, heavily relying on the user’s experience, and it can be worse with the emerging high-fidelity requirements or in the design of novel configurations. On the other hand, mesh quality and the associated numerical errors are typically only studied before and after the optimization, leaving the design search path unveiled to numerical errors. This work tackles these issues by integrating an additional component, output-based mesh adaptation, within traditional aerodynamic shape optimizations. First, we develop a more suitable error estimator for optimization problems by taking into account errors in both the objective and constraint outputs. The localized output errors are then used to drive mesh adaptation to achieve the desired accuracy on both the objective and constraint outputs. With the variable fidelity offered by the adaptive meshes, multi-fidelity optimization frameworks are developed to tightly couple mesh adaptation and shape optimization. The objective functional and its sensitivity are first evaluated on an initial coarse mesh, which is then subsequently adapted as the shape optimization proceeds. The effort to set up the optimization is minimal since the initial mesh can be fairly coarse and easy to generate. Meanwhile, the proposed framework saves computational costs by reducing the mesh size at the early stages of the optimization, when the design is far from optimal, and avoiding exhaustive search on low-fidelity meshes when the outputs are inaccurate. To further improve the computational efficiency, we also introduce new methods to accelerate the error estimation and mesh adaptation using machine learning techniques. Surrogate models are developed to predict the localized output error and optimal mesh anisotropy to guide the adaptation. The proposed machine learning approaches demonstrate good performance in two-dimensional test problems, encouraging more study and developments to incorporate them within aerodynamic optimization techniques. Although CFD has been extensively used in aircraft design and optimization, the design automation, reliability, and efficiency are largely limited by the mesh generation process and the fixed-mesh optimization paradigm. With the emerging high-fidelity requirements and the further developments of unconventional configurations, CFD-based optimization has to be made more accurate and more efficient to achieve higher design reliability and lower computational cost. Furthermore, future aerodynamic optimization needs to avoid unnecessary overhead in mesh generation and optimization setup to further automate the design process. The author expects the methods developed in this work to be the keys to enable more automated, reliable, and efficient aerodynamic shape optimization, making CFD-based optimization a more powerful tool in aircraft design.PHDAerospace EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163034/1/cgderic_1.pd