Some considerations regarding the use of multi-fidelity Kriging in the construction of surrogate models

Surrogate models or metamodels are commonly used to exploit expensive computational simulations within a design optimization framework. The application of multi-fidelity surrogate modeling approaches has recently been gaining ground due to the potential for further reductions in simulation effort over single fidelity approaches. However, given a black box problem when exactly should a designer select a multi-fidelity approach over a single fidelity approach and vice versa? Using a series of analytical test functions and engineering design examples from the literature, the following paper illustrates the potential pitfalls of choosing one technique over the other without a careful consideration of the optimization problem at hand. These examples are then used to define and validate a set of guidelines for the creation of a multi-fidelity Kriging model. The resulting guidelines state that the different fidelity functions should be well correlated, that the amount of low fidelity data in the model should be greater than the amount of high fidelity data and that more than 10\% and less than 80\% of the total simulation budget should be spent on low fidelity simulations in order for the resulting multi-fidelity model to perform better than the equivalent costing high fidelity model

Multi-objective optimization of transonic airfoils using variable-fidelity models, co-kriging surrogates, and design space reduction

Computationally efficient constrained multi-objective design optimization of transonic airfoils is considered. The proposed methodology focuses on fixed-lift design aimed at finding the best possible trade-offs between the conflicting objectives. The algorithm exploits the surrogate-based optimization principle, variable-fidelity computational fluid dynamics (CFD) models, as well as auxiliary approximation surrogates (here, using kriging). The kriging models constructed within a reduced design space. The optimization process has three major stages: (i) design space reduction which involves the identification of the extreme points of the Pareto front through single-objective optimization, (ii) construction of the kriging model and an initial Pareto front generation using multi-objective evolutionary algorithm, and (iii) Pareto front refinement using co-kriging models. For the sake of computational efficiency, stages (i) and (ii) are realized at the level of low-fidelity CFD models. The proposed algorithm is applied to the multi-objective optimization of a transonic airfoil at a Mach number of 0.734 and a fixed lift coefficient of 0.824. The shape is parameterized with eight B-spline control points. The fluid flow is taken to be inviscid. The high-fidelity model solves the compressible Euler equations. The low-fidelity model is the same as the high-fidelity one, but with a coarser description and is much faster to execute. With the proposed approach, the entire Pareto front of the drag coefficient and the pitching moment coefficient is obtained using 100 low-fidelity samples and 3 high-fidelity model samples. This cost is not only considerably lower (up to two orders of magnitude) than the cost of direct high-fidelity mode optimization using metaheuristics without design space reduction, but, more importantly, renders multi-objective optimization of transonic airfoil shapes computationally tractable, even at the level of accurate CFD models

Multi-Fidelity Local Surrogate Model for Computationally Efficient Microwave Component Design Optimization

Publisher's version (útgefin grein)In order to minimize the number of evaluations of high-fidelity (fine) model in the optimization process, to increase the optimization speed, and to improve optimal solution accuracy, a robust and computational-efficient multi-fidelity local surrogate-model optimization method is proposed. Based on the principle of response surface approximation, the proposed method exploits the multi-fidelity coarse models and polynomial interpolation to construct a series of local surrogate models. In the optimization process, local region modeling and optimization are performed iteratively. A judgment factor is introduced to provide information for local region size update. The last local surrogate model is refined by space mapping techniques to obtain the optimal design with high accuracy. The operation and efficiency of the approach are demonstrated through design of a bandpass filter and a compact ultra-wide-band (UWB) multiple-in multiple-out (MIMO) antenna. The response of the optimized design of the fine model meet the design specification. The proposed method not only has better convergence compared to an existing local surrogate method, but also reduces the computational cost substantially.The National Natural Science Foundation of China Grant 61471258 and by Science & Technology Innovation Committee of Shenzhen Municipality Grant KQJSCX20170328153625183."Peer Reviewed

Multi-Fidelity Methods for Optimization: A Survey

Real-world black-box optimization often involves time-consuming or costly experiments and simulations. Multi-fidelity optimization (MFO) stands out as a cost-effective strategy that balances high-fidelity accuracy with computational efficiency through a hierarchical fidelity approach. This survey presents a systematic exploration of MFO, underpinned by a novel text mining framework based on a pre-trained language model. We delve deep into the foundational principles and methodologies of MFO, focusing on three core components -- multi-fidelity surrogate models, fidelity management strategies, and optimization techniques. Additionally, this survey highlights the diverse applications of MFO across several key domains, including machine learning, engineering design optimization, and scientific discovery, showcasing the adaptability and effectiveness of MFO in tackling complex computational challenges. Furthermore, we also envision several emerging challenges and prospects in the MFO landscape, spanning scalability, the composition of lower fidelities, and the integration of human-in-the-loop approaches at the algorithmic level. We also address critical issues related to benchmarking and the advancement of open science within the MFO community. Overall, this survey aims to catalyze further research and foster collaborations in MFO, setting the stage for future innovations and breakthroughs in the field.Comment: 47 pages, 9 figure

Pareto Ranking Bisection Algorithm for EM-Driven Multi-Objective Design of Antennas in Highly-Dimensional Parameter Spaces

A deterministic technique for fast surrogate-assisted multi-objective design optimization of antennas in highly-dimensional parameters spaces has been discussed. In this two-stage approach, the initial approximation of the Pareto set representing the best compromise between conflicting objectives is obtained using a bisection algorithm which finds new Pareto-optimal designs by dividing the line segments interconnecting previously found optimal points, and executing poll-type search that involves Pareto ranking. The initial Pareto front is generated at the level of the coarsely-discretized EM model of the antenna. In the second stage of the algorithm, the high-fidelity Pareto designs are obtained through optimization of corrected local-approximation models. The considered optimization method is verified using a 17-variable uniplanar antenna operating in ultra-wideband frequency range. The method is compared to three state-of-the-art surrogate-assisted multi-objective optimization algorithms

Using High-fidelity Time-Domain Simulation Data to Construct Multi-fidelity State Derivative Function Surrogate Models for use in Control and Optimization

Models that balance accuracy against computational costs are advantageous when designing dynamic systems with optimization studies, as several hundred predictive function evaluations might be necessary to identify the optimal solution. The efficacy and use of derivative function surrogate models (DFSMs), or approximate models of the state derivative function, have been well-established in the literature. However, previous studies have assumed an a priori state dynamic model is available that can be directly evaluated to construct the DFSM. In this article, we propose an approach to extract the state derivative information from system simulations using piecewise polynomial approximations. Once the required information is available, we propose a multi-fidelity DFSM approach as a predictive model for the system's dynamic response. This multi-fidelity model consists of summation between a linear-fit lower-fidelity model and an additional nonlinear error corrective function that compensates for the error between the high-fidelity simulations and low-fidelity models. We validate the model by comparing the simulation results from the DFSM to the high-fidelity tools. The DFSM model is, on average, five times faster than the high-fidelity tools while capturing the key time domain and power spectral density~(PSD) trends. Then, an optimal control study using the DFSM is conducted with outcomes showing that the DFSM approach can be used for complex systems like floating offshore wind turbines~(FOWTs) and help identify control trends and trade-offs.Comment: 14 pages,45 figure

MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements

Hyperparameter optimization (HPO) is a fundamental problem in automatic machine learning (AutoML). However, due to the expensive evaluation cost of models (e.g., training deep learning models or training models on large datasets), vanilla Bayesian optimization (BO) is typically computationally infeasible. To alleviate this issue, Hyperband (HB) utilizes the early stopping mechanism to speed up configuration evaluations by terminating those badly-performing configurations in advance. This leads to two kinds of quality measurements: (1) many low-fidelity measurements for configurations that get early-stopped, and (2) few high-fidelity measurements for configurations that are evaluated without being early stopped. The state-of-the-art HB-style method, BOHB, aims to combine the benefits of both BO and HB. Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only. However, the scarcity of high-fidelity measurements greatly hampers the efficiency of BO to guide the configuration search. In this paper, we present MFES-HB, an efficient Hyperband method that is capable of utilizing both the high-fidelity and low-fidelity measurements to accelerate the convergence of HPO tasks. Designing MFES-HB is not trivial as the low-fidelity measurements can be biased yet informative to guide the configuration search. Thus we propose to build a Multi- Fidelity Ensemble Surrogate (MFES) based on the generalized Product of Experts framework, which can integrate useful information from multi-fidelity measurements effectively. The empirical studies on the real-world AutoML tasks demonstrate that MFES-HB can achieve 3.3-8.9x speedups over the state-of-the-art approach - BOHB

Investigation of robust optimization and evidence theory with stochastic expansions for aerospace applications under mixed uncertainty

One of the primary objectives of this research is to develop a method to model and propagate mixed (aleatory and epistemic) uncertainty in aerospace simulations using DSTE. In order to avoid excessive computational cost associated with large scale applications and the evaluation of Dempster Shafer structures, stochastic expansions are implemented for efficient UQ. The mixed UQ with DSTE approach was demonstrated on an analytical example and high fidelity computational fluid dynamics (CFD) study of transonic flow over a RAE 2822 airfoil. Another objective is to devise a DSTE based performance assessment framework through the use of quantification of margins and uncertainties. Efficient uncertainty propagation in system design performance metrics and performance boundaries is achieved through the use of stochastic expansions. The technique is demonstrated on: (1) a model problem with non-linear analytical functions representing the outputs and performance boundaries of two coupled systems and (2) a multi-disciplinary analysis of a supersonic civil transport. Finally, the stochastic expansions are applied to aerodynamic shape optimization under uncertainty. A robust optimization algorithm is presented for computationally efficient airfoil design under mixed uncertainty using a multi-fidelity approach. This algorithm exploits stochastic expansions to create surrogate models utilized in the optimization process. To reduce the computational cost, output space mapping technique is implemented to replace the high-fidelity CFD model by a suitably corrected low-fidelity one. The proposed algorithm is demonstrated on the robust optimization of NACA 4-digit airfoils under mixed uncertainties in transonic flow. --Abstract, page iii

Multi-fidelity Gaussian process regression for prediction of random fields

We propose a new multi-fidelity Gaussian process regression (GPR) approach for prediction of random fields based on observations of surrogate models or hierarchies of surrogate models. Our method builds upon recent work on recursive Bayesian techniques, in particular recursive co-kriging, and extends it to vector-valued fields and various types of covariances, including separable and non-separable ones. The framework we propose is general and can be used to perform uncertainty propagation and quantification in model-based simulations, multi-fidelity data fusion, and surrogate-based optimization. We demonstrate the effectiveness of the proposed recursive GPR techniques through various examples. Specifically, we study the stochastic Burgers equation and the stochastic Oberbeck\u2013Boussinesq equations describing natural convection within a square enclosure. In both cases we find that the standard deviation of the Gaussian predictors as well as the absolute errors relative to benchmark stochastic solutions are very small, suggesting that the proposed multi-fidelity GPR approaches can yield highly accurate results