28 research outputs found
Multi-fidelity optimization via surrogate modelling
This paper demonstrates the application of correlated Gaussian process based
approximations to optimization where multiple levels of analysis are available, using an
extension to the geostatistical method of co-kriging. An exchange algorithm is used to
choose which points of the search space to sample within each level of analysis. The
derivation of the co-kriging equations is presented in an intuitive manner, along with a new
variance estimator to account for varying degrees of computational ‘noise’ in the multiple
levels of analysis. A multi-fidelity wing optimization is used to demonstrate the
methodology
Robust Gaussian process-based global optimization using a fully Bayesian expected improvement criterion
International audienceWe consider the problem of optimizing a real-valued continuous function f, which is supposed to be expensive to evaluate and, consequently, can only be evaluated a limited number of times. This article focuses on the Bayesian approach to this problem, which consists in combining evaluation results and prior information about f in order to efficiently select new evaluation points, as long as the budget for evaluations is not exhausted. The algorithm called efficient global optimization (EGO), proposed by Jones, Schonlau and Welch (J. Global Optim., 13(4):455-492, 1998), is one of the most popular Bayesian optimization algorithms. It is based on a sampling criterion called the expected improvement (EI), which assumes a Gaussian process prior about f. In the EGO algorithm, the parameters of the covariance of the Gaussian process are estimated from the evaluation results by maximum likelihood, and these parameters are then plugged in the EI sampling criterion. However, it is well-known that this plug-in strategy can lead to very disappointing results when the evaluation results do not carry enough information about f to estimate the parameters in a satisfactory manner. We advocate a fully Bayesian approach to this problem, and derive an analytical expression for the EI criterion in the case of Student predictive distributions. Numerical experiments show that the fully Bayesian approach makes EI-based optimization more robust while maintaining an average loss similar to that of the EGO algorithm
Compositional Policy Priors
This paper describes a probabilistic framework for incorporating structured inductive biases into reinforcement learning. These inductive biases arise from policy priors, probability distributions over optimal policies. Borrowing recent ideas from computational linguistics and Bayesian nonparametrics, we define several families of policy priors that express compositional, abstract structure in a domain. Compositionality is expressed using probabilistic context-free grammars, enabling a compact representation of hierarchically organized sub-tasks. Useful sequences of sub-tasks can be cached and reused by extending the grammars nonparametrically using Fragment Grammars. We present Monte Carlo methods for performing inference, and show how structured policy priors lead to substantially faster learning in complex domains compared to methods without inductive biases.This work was supported by AFOSR FA9550-07-1-0075 and ONR
N00014-07-1-0937. SJG was supported by a Graduate Research Fellowship from the NSF