Efficient computation of the Knowledge Gradient for Bayesian Optimization

Bayesian optimization is a powerful collection of methods for optimizing stochastic expensive black box functions. One key component of a Bayesian optimization algorithm is the acquisition function that determines which solution should be evaluated in every iteration. A popular and very effective choice is the Knowledge Gradient acquisition function, however there is no analytical way to compute it. Several different implementations make different approximations. In this paper, we review and compare the spectrum of Knowledge Gradient implementations and propose One-shot Hybrid KG, a new approach that combines several of the previously proposed ideas and is cheap to compute as well as powerful and efficient. We prove the new method preserves theoretical properties of previous methods and empirically show the drastically reduced computational overhead with equal or improved performance. All experiments are implemented in BOTorch and code is available on github

Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks

The graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a competitive construction heuristic for graph colouring. Our proposed approach, ReLCol, uses deep Q-learning together with a graph neural network for feature extraction, and employs a novel way of parameterising the graph that results in improved performance. Using standard benchmark graphs with varied topologies, we empirically evaluate the benefits and limitations of the heuristic learned by ReLCol relative to existing construction algorithms, and demonstrate that reinforcement learning is a promising direction for further research on the graph colouring problem.Comment: 15 pages, 6 figures, to be published in LION17 conference proceeding

Pharmaceutical R & D pipeline management under trial duration uncertainty

We consider a pharmaceutical Research & Development (R & D) pipeline management problem under two significant uncertainties: the outcomes of clinical trials and their durations. We present an Approximate Dynamic Programming (ADP) approach to solve the problem efficiently. Given an initial list of potential drug candidates, ADP derives a policy that suggests the trials to be performed at each decision point and state. For the classical R&D pipeline planning problem with deterministic trial durations, we compare our ADP approach with other methods from the literature, and find that it can find better solutions more quickly in particular for larger problem instances. For the case with stochastic trial durations, we compare the ADP algorithm with a myopic approach and show that the expected net profit obtained by the derived ADP policy is higher (almost 20% for a 10-drug portfolio)

When to elicit preferences in multi-objective Bayesian optimization

We consider the case of interactive multi-objective Bayesian optimization where decision maker (DM) preferences can be elicited by asking the DM to select the more preferred among pairs of observations. Assuming that there is a cost to evaluating a solution as well as to eliciting preferences, and given a total budget, we propose an acquisition function that, in each iteration, decides whether to evaluate another solution or to query the DM. Thus, the approach automatically chooses how often and when to interact with the DM. It furthermore decides which pair of observations is likely to be most informative when shown to the DM. We show empirically that the proposed criterion is not only able to pick suitable pairs of observations, but also automatically results in a sensible balance between optimization and querying the DM

Bayesian optimisation vs. input uncertainty reduction

Simulators often require calibration inputs estimated from real world data and the estimate can significantly affect simulation output. Particularly when performing simulation optimisation to find an optimal solution, the uncertainty in the inputs significantly affects the quality of the found solution. One remedy is to search for the solution that has the best performance on average over the uncertain range of inputs yielding an optimal compromise solution. We consider the more general setting where a user may choose between either running simulations or instead querying an external data source, improving the input estimate and enabling the search for a more targeted, less compromised solution. We explicitly examine the trade-off between simulation and real data collection in order to find the optimal solution of the simulator with the true inputs. Using a value of information procedure, we propose a novel unified simulation optimisation procedure called Bayesian Information Collection and Optimisation (BICO) that, in each iteration, automatically determines which of the two actions (running simulations or data collection) is more beneficial. We theoretically prove convergence in the infinite budget limit and perform numerical experiments demonstrating that the proposed algorithm is able to automatically determine an appropriate balance between optimisation and data collection

Reproducibility in evolutionary computation

Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies have increased in recent times, reflecting similar concerns in other scientific fields. In this article, we discuss, within the context of EC, the different types of reproducibility and suggest a classification that refines the badge system of the Association of Computing Machinery (ACM) adopted by ACM Transactions on Evolutionary Learning and Optimization (TELO). We identify cultural and technical obstacles to reproducibility in the EC field. Finally, we provide guidelines and suggest tools that may help to overcome some of these reproducibility obstacles

Bayesian optimisation for quality diversity search with coupled descriptor functions

Quality Diversity (QD) algorithms such as the Multi- Dimensional Archive of Phenotypic Elites (MAP-Elites) are a class of optimisation techniques that attempt to find many high performing points that all behave differently according to a userdefined behavioural metric. In this paper we propose the Bayesian Optimisation of Elites (BOP-Elites) algorithm. Designed for problems with expensive fitness functions and coupled behaviour descriptors, it is able to return a QD solution-set with excellent performance already after a relatively small number of samples. BOP-Elites models both fitness and behavioural descriptors with Gaussian Process surrogate models and uses Bayesian Optimisation strategies for choosing points to evaluate in order to solve the quality-diversity problem. In addition, BOP-Elites produces high quality surrogate models which can be used after convergence to predict solutions with any behaviour in a continuous range. An empirical comparison shows that BOP-Elites significantly outperforms other state-of-the-art algorithms without the need for problem-specific parameter tuning

Tackling neural architecture search with quality diversity optimization

Neural architecture search (NAS) has been studied extensively and has grown to become a research field with substantial impact. While classical single-objective NAS searches for the architecture with the best performance, multi-objective NAS considers multiple objectives that should be optimized simultaneously, e.g., minimizing resource usage along the validation error. Although considerable progress has been made in the field of multiobjective NAS, we argue that there is some discrepancy between the actual optimization problem of practical interest and the optimization problem that multi-objective NAS tries to solve. We resolve this discrepancy by formulating the multi-objective NAS problem as a quality diversity optimization (QDO) problem and introduce three quality diversity NAS optimizers (two of them belonging to the group of multifidelity optimizers), which search for high-performing yet diverse architectures that are optimal for application-specific niches, e.g., hardware constraints. By comparing these optimizers to their multi-objective counterparts, we demonstrate that quality diversity NAS in general outperforms multiobjective NAS with respect to quality of solutions and efficiency. We further show how applications and future NAS research can thrive on QDO