1,639 research outputs found
Better safe than sorry: Risky function exploitation through safe optimization
Exploration-exploitation of functions, that is learning and optimizing a
mapping between inputs and expected outputs, is ubiquitous to many real world
situations. These situations sometimes require us to avoid certain outcomes at
all cost, for example because they are poisonous, harmful, or otherwise
dangerous. We test participants' behavior in scenarios in which they have to
find the optimum of a function while at the same time avoid outputs below a
certain threshold. In two experiments, we find that Safe-Optimization, a
Gaussian Process-based exploration-exploitation algorithm, describes
participants' behavior well and that participants seem to care firstly whether
a point is safe and then try to pick the optimal point from all such safe
points. This means that their trade-off between exploration and exploitation
can be seen as an intelligent, approximate, and homeostasis-driven strategy.Comment: 6 pages, submitted to Cognitive Science Conferenc
Time-Varying Gaussian Process Bandit Optimization
We consider the sequential Bayesian optimization problem with bandit
feedback, adopting a formulation that allows for the reward function to vary
with time. We model the reward function using a Gaussian process whose
evolution obeys a simple Markov model. We introduce two natural extensions of
the classical Gaussian process upper confidence bound (GP-UCB) algorithm. The
first, R-GP-UCB, resets GP-UCB at regular intervals. The second, TV-GP-UCB,
instead forgets about old data in a smooth fashion. Our main contribution
comprises of novel regret bounds for these algorithms, providing an explicit
characterization of the trade-off between the time horizon and the rate at
which the function varies. We illustrate the performance of the algorithms on
both synthetic and real data, and we find the gradual forgetting of TV-GP-UCB
to perform favorably compared to the sharp resetting of R-GP-UCB. Moreover,
both algorithms significantly outperform classical GP-UCB, since it treats
stale and fresh data equally.Comment: To appear in AISTATS 201
- …