123 research outputs found
Multi-scale exploration of convex functions and bandit convex optimization
We construct a new map from a convex function to a distribution on its
domain, with the property that this distribution is a multi-scale exploration
of the function. We use this map to solve a decade-old open problem in
adversarial bandit convex optimization by showing that the minimax regret for
this problem is , where is the
dimension and the number of rounds. This bound is obtained by studying the
dual Bayesian maximin regret via the information ratio analysis of Russo and
Van Roy, and then using the multi-scale exploration to solve the Bayesian
problem.Comment: Preliminary version; 22 page
Pure Exploration for Multi-Armed Bandit Problems
We consider the framework of stochastic multi-armed bandit problems and study
the possibilities and limitations of forecasters that perform an on-line
exploration of the arms. These forecasters are assessed in terms of their
simple regret, a regret notion that captures the fact that exploration is only
constrained by the number of available rounds (not necessarily known in
advance), in contrast to the case when the cumulative regret is considered and
when exploitation needs to be performed at the same time. We believe that this
performance criterion is suited to situations when the cost of pulling an arm
is expressed in terms of resources rather than rewards. We discuss the links
between the simple and the cumulative regret. One of the main results in the
case of a finite number of arms is a general lower bound on the simple regret
of a forecaster in terms of its cumulative regret: the smaller the latter, the
larger the former. Keeping this result in mind, we then exhibit upper bounds on
the simple regret of some forecasters. The paper ends with a study devoted to
continuous-armed bandit problems; we show that the simple regret can be
minimized with respect to a family of probability distributions if and only if
the cumulative regret can be minimized for it. Based on this equivalence, we
are able to prove that the separable metric spaces are exactly the metric
spaces on which these regrets can be minimized with respect to the family of
all probability distributions with continuous mean-payoff functions
- …