15 research outputs found
Safe Exploration for Optimization with Gaussian Processes
We consider sequential decision problems under uncertainty, where we seek to optimize an unknown function from noisy samples. This requires balancing exploration (learning about the objective) and exploitation (localizing the maximum), a problem well-studied in the multi-armed bandit literature. In many applications, however, we require that the sampled function values exceed some prespecified "safety" threshold, a requirement that existing algorithms fail to meet. Examples include medical applications where patient comfort must be guaranteed, recommender systems aiming to avoid user dissatisfaction, and robotic control, where one seeks to avoid controls causing physical harm to the platform. We tackle this novel, yet rich, set of problems under the assumption that the unknown function satisfies regularity conditions expressed via a Gaussian process prior. We develop an efficient algorithm called SafeOpt, and theoretically guarantee its convergence to a natural notion of optimum reachable under safety constraints. We evaluate SafeOpt on synthetic data, as well as two real applications: movie recommendation, and therapeutic spinal cord stimulation
Active model learning and diverse action sampling for task and motion planning
The objective of this work is to augment the basic abilities of a robot by
learning to use new sensorimotor primitives to enable the solution of complex
long-horizon problems. Solving long-horizon problems in complex domains
requires flexible generative planning that can combine primitive abilities in
novel combinations to solve problems as they arise in the world. In order to
plan to combine primitive actions, we must have models of the preconditions and
effects of those actions: under what circumstances will executing this
primitive achieve some particular effect in the world?
We use, and develop novel improvements on, state-of-the-art methods for
active learning and sampling. We use Gaussian process methods for learning the
conditions of operator effectiveness from small numbers of expensive training
examples collected by experimentation on a robot. We develop adaptive sampling
methods for generating diverse elements of continuous sets (such as robot
configurations and object poses) during planning for solving a new task, so
that planning is as efficient as possible. We demonstrate these methods in an
integrated system, combining newly learned models with an efficient
continuous-space robot task and motion planner to learn to solve long horizon
problems more efficiently than was previously possible.Comment: Proceedings of the 2018 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS), Madrid, Spain.
https://www.youtube.com/playlist?list=PLoWhBFPMfSzDbc8CYelsbHZa1d3uz-W_
Data-driven computation of invariant sets of discrete time-invariant black-box systems
We consider the problem of computing the maximal invariant set of
discrete-time black-box nonlinear systems without analytic dynamical models.
Under the assumption that the system is asymptotically stable, the maximal
invariant set coincides with the domain of attraction. A data-driven framework
relying on the observation of trajectories is proposed to compute
almost-invariant sets, which are invariant almost everywhere except a small
subset. Based on these observations, scenario optimization problems are
formulated and solved. We show that probabilistic invariance guarantees on the
almost-invariant sets can be established. To get explicit expressions of such
sets, a set identification procedure is designed with a verification step that
provides inner and outer approximations in a probabilistic sense. The proposed
data-driven framework is illustrated by several numerical examples.Comment: A shorter version with the title "Scenario-based set invariance
verification for black-box nonlinear systems" is published in the IEEE
Control Systems Letters (L-CSS
Bayesian sequential design of computer experiments to estimate reliable sets
We consider an unknown multivariate function representing a system-such as a
complex numerical simulator-taking both deterministic and uncertain inputs. Our
objective is to estimate the set of deterministic inputs leading to outputs
whose probability (with respect to the distribution of the uncertain inputs) to
belong to a given set is controlled by a given threshold. To solve this
problem, we propose a Bayesian strategy based on the Stepwise Uncertainty
Reduction (SUR) principle to sequentially choose the points at which the
function should be evaluated to approximate the set of interest. We illustrate
its performance and interest in several numerical experiments
Batch Bayesian active learning for feasible region identification by local penalization
Identifying all designs satisfying a set of constraints is an important part of the engineering design process. With physics-based simulation codes, evaluating the constraints becomes considerable expensive. Active learning can provide an elegant approach to efficiently characterize the feasible region, i.e., the set of feasible designs. Although active learning strategies have been proposed for this task, most of them are dealing with adding just one sample per iteration as opposed to selecting multiple samples per iteration, also known as batch active learning. While this is efficient with respect to the amount of information gained per iteration, it neglects available computation resources. We propose a batch Bayesian active learning technique for feasible region identification by assuming that the constraint function is Lipschitz continuous. In addition, we extend current state-of-the-art batch methods to also handle feasible region identification. Experiments show better performance of the proposed method than the extended batch methods
Safe Exploration for Optimization with Gaussian Processes
We consider sequential decision problems under uncertainty, where we seek to optimize an unknown function from noisy samples. This requires balancing exploration (learning about the objective) and exploitation (localizing the maximum), a problem well-studied in the multi-armed bandit literature. In many applications, however, we require that the sampled function values exceed some prespecified "safety" threshold, a requirement that existing algorithms fail to meet. Examples include medical applications where patient comfort must be guaranteed, recommender systems aiming to avoid user dissatisfaction, and robotic control, where one seeks to avoid controls causing physical harm to the platform. We tackle this novel, yet rich, set of problems under the assumption that the unknown function satisfies regularity conditions expressed via a Gaussian process prior. We develop an efficient algorithm called SafeOpt, and theoretically guarantee its convergence to a natural notion of optimum reachable under safety constraints. We evaluate SafeOpt on synthetic data, as well as two real applications: movie recommendation, and therapeutic spinal cord stimulation
Generalizing Bayesian Optimization with Decision-theoretic Entropies
Bayesian optimization (BO) is a popular method for efficiently inferring
optima of an expensive black-box function via a sequence of queries. Existing
information-theoretic BO procedures aim to make queries that most reduce the
uncertainty about optima, where the uncertainty is captured by Shannon entropy.
However, an optimal measure of uncertainty would, ideally, factor in how we
intend to use the inferred quantity in some downstream procedure. In this
paper, we instead consider a generalization of Shannon entropy from work in
statistical decision theory (DeGroot 1962, Rao 1984), which contains a broad
class of uncertainty measures parameterized by a problem-specific loss function
corresponding to a downstream task. We first show that special cases of this
entropy lead to popular acquisition functions used in BO procedures such as
knowledge gradient, expected improvement, and entropy search. We then show how
alternative choices for the loss yield a flexible family of acquisition
functions that can be customized for use in novel optimization settings.
Additionally, we develop gradient-based methods to efficiently optimize our
proposed family of acquisition functions, and demonstrate strong empirical
performance on a diverse set of sequential decision making tasks, including
variants of top- optimization, multi-level set estimation, and sequence
search.Comment: Appears in Proceedings of the 36th Conference on Neural Information
Processing Systems (NeurIPS 2022