Multi-objective global optimization for hydrologic models

The development of automated (computer-based) calibration methods has focused mainly on the selection of a single-objective measure of the distance between the model-simulated output and the data and the selection of an automatic optimization algorithm to search for the parameter values which minimize that distance. However, practical experience with model calibration suggests that no single-objective function is adequate to measure the ways in which the model fails to match the important characteristics of the observed data. Given that some of the latest hydrologic models simulate several of the watershed output fluxes (e.g. water, energy, chemical constituents, etc.), there is a need for effective and efficient multi-objective calibration procedures capable of exploiting all of the useful information about the physical system contained in the measurement data time series. The MOCOM-UA algorithm, an effective and efficient methodology for solving the multiple-objective global optimization problem, is presented in this paper. The method is an extension of the successful SCE-UA single-objective global optimization algorithm. The features and capabilities of MOCOM-UA are illustrated by means of a simple hydrologic model calibration study

Methods for many-objective optimization: an analysis

Decomposition-based methods are often cited as the solution to problems related with many-objective optimization. Decomposition-based methods employ a scalarizing function to reduce a many-objective problem into a set of single objective problems, which upon solution yields a good approximation of the set of optimal solutions. This set is commonly referred to as Pareto front. In this work we explore the implications of using decomposition-based methods over Pareto-based methods from a probabilistic point of view. Namely, we investigate whether there is an advantage of using a decomposition-based method, for example using the Chebyshev scalarizing function, over Paretobased methods

Learning from Scarce Experience

Searching the space of policies directly for the optimal policy has been one popular method for solving partially observable reinforcement learning problems. Typically, with each change of the target policy, its value is estimated from the results of following that very policy. This requires a large number of interactions with the environment as different polices are considered. We present a family of algorithms based on likelihood ratio estimation that use data gathered when executing one policy (or collection of policies) to estimate the value of a different policy. The algorithms combine estimation and optimization stages. The former utilizes experience to build a non-parametric representation of an optimized function. The latter performs optimization on this estimate. We show positive empirical results and provide the sample complexity bound.Comment: 8 pages 4 figure

A simulation comparison of methods for new product location

Includes bibliographical references (p. 29-31)

Better Optimism By Bayes: Adaptive Planning with Rich Models

The computational costs of inference and planning have confined Bayesian model-based reinforcement learning to one of two dismal fates: powerful Bayes-adaptive planning but only for simplistic models, or powerful, Bayesian non-parametric models but using simple, myopic planning strategies such as Thompson sampling. We ask whether it is feasible and truly beneficial to combine rich probabilistic models with a closer approximation to fully Bayesian planning. First, we use a collection of counterexamples to show formal problems with the over-optimism inherent in Thompson sampling. Then we leverage state-of-the-art techniques in efficient Bayes-adaptive planning and non-parametric Bayesian methods to perform qualitatively better than both existing conventional algorithms and Thompson sampling on two contextual bandit-like problems.Comment: 11 pages, 11 figure