231 research outputs found

    Efficient structure search with multi-task Bayesian optimization

    Get PDF
    Computational materials science aims to discover new functional materials and optimize their properties, which often includes resource-intensive calculations. To address structure search tasks with the least number of expensive calculations, the Bayesian Optimization Structure Search (BOSS) algorithm has been implemented. BOSS applies active learning in combination with Gaussian process regression to sample-efficiently optimize a target function, which in this case represents the total energy of the material. Materials can be simulated with approximated methods which are fast but less accurate or with costly and accurate electronic structure methods. This work investigates how BOSS can become even more resource-efficient by incorporating calculations from different levels of accuracy. Multi-fidelity BOSS uses the Intrinsic Model of Coregionalization (ICM) to integrate data from different atomistic simulators, all focusing on the same objective, the total energy of the material. This work focuses on multi-fidelity learning acquisition functions, which are one of the key components of the multi-fidelity algorithm. In particular, I developed and implemented several multi-fidelity acquisition functions. To test the functions, I applied multi-fidelity BOSS on the alanine structure search task, where I used simulations of the alanine system based on force fields (AMBER18), density-functional theory (FHI-aims with PBE-exchange correlation functional) and quantum chemistry accuracy (Gaussian16 with CCSD(T)). I found that multi-fidelity BOSS reduced the CPU cost by up to 90% CPU when used with the ELCB or MES acquisition functions. Both acquisition functions enable large savings when used in combination with different separable or inseparable sampling strategies. I also found, that the possible savings depends significantly on the sampling costs of the atomistic simulators, the correlation between the different fidelities and the dimension of the search space

    MUMBO:MUlti-task Max-value Bayesian Optimization

    Get PDF
    We propose MUMBO, the first high-performing yet computationally efficient acquisition function for multi-task Bayesian optimization. Here, the challenge is to perform efficient optimization by evaluating low-cost functions somehow related to our true target function. This is a broad class of problems including the popular task of multi-fidelity optimization. However, while information-theoretic acquisition functions are known to provide state-of-the-art Bayesian optimization, existing implementations for multi-task scenarios have prohibitive computational requirements. Previous acquisition functions have therefore been suitable only for problems with both low-dimensional parameter spaces and function query costs sufficiently large to overshadow very significant optimization overheads. In this work, we derive a novel multi-task version of entropy search, delivering robust performance with low computational overheads across classic optimization challenges and multi-task hyper-parameter tuning. MUMBO is scalable and efficient, allowing multi-task Bayesian optimization to be deployed in problems with rich parameter and fidelity spaces

    Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

    Full text link
    Bayesian optimization has become a successful tool for hyperparameter optimization of machine learning algorithms, such as support vector machines or deep neural networks. Despite its success, for large datasets, training and validating a single configuration often takes hours, days, or even weeks, which limits the achievable performance. To accelerate hyperparameter optimization, we propose a generative model for the validation error as a function of training set size, which is learned during the optimization process and allows exploration of preliminary configurations on small subsets, by extrapolating to the full dataset. We construct a Bayesian optimization procedure, dubbed Fabolas, which models loss and training time as a function of dataset size and automatically trades off high information gain about the global optimum against computational cost. Experiments optimizing support vector machines and deep neural networks show that Fabolas often finds high-quality solutions 10 to 100 times faster than other state-of-the-art Bayesian optimization methods or the recently proposed bandit strategy Hyperband
    • …
    corecore