1,272 research outputs found
Adaptive Stratified Sampling for Monte-Carlo integration of Differentiable functions
We consider the problem of adaptive stratified sampling for Monte Carlo
integration of a differentiable function given a finite number of evaluations
to the function. We construct a sampling scheme that samples more often in
regions where the function oscillates more, while allocating the samples such
that they are well spread on the domain (this notion shares similitude with low
discrepancy). We prove that the estimate returned by the algorithm is almost
similarly accurate as the estimate that an optimal oracle strategy (that would
know the variations of the function everywhere) would return, and provide a
finite-sample analysis.Comment: 23 pages, 3 figures, to appear in NIPS 2012 conference proceeding
Bandit Theory meets Compressed Sensing for high dimensional Stochastic Linear Bandit
We consider a linear stochastic bandit problem where the dimension of the
unknown parameter is larger than the sampling budget . In such
cases, it is in general impossible to derive sub-linear regret bounds since
usual linear bandit algorithms have a regret in . In this paper
we assume that is sparse, i.e. has at most non-zero
components, and that the space of arms is the unit ball for the norm.
We combine ideas from Compressed Sensing and Bandit Theory and derive
algorithms with regret bounds in
Bandit Algorithms for Tree Search
Bandit based methods for tree search have recently gained popularity when
applied to huge trees, e.g. in the game of go (Gelly et al., 2006). The UCT
algorithm (Kocsis and Szepesvari, 2006), a tree search method based on Upper
Confidence Bounds (UCB) (Auer et al., 2002), is believed to adapt locally to
the effective smoothness of the tree. However, we show that UCT is too
``optimistic'' in some cases, leading to a regret O(exp(exp(D))) where D is the
depth of the tree. We propose alternative bandit algorithms for tree search.
First, a modification of UCT using a confidence sequence that scales
exponentially with the horizon depth is proven to have a regret O(2^D
\sqrt{n}), but does not adapt to possible smoothness in the tree. We then
analyze Flat-UCB performed on the leaves and provide a finite regret bound with
high probability. Then, we introduce a UCB-based Bandit Algorithm for Smooth
Trees which takes into account actual smoothness of the rewards for performing
efficient ``cuts'' of sub-optimal branches with high confidence. Finally, we
present an incremental tree search version which applies when the full tree is
too big (possibly infinite) to be entirely represented and show that with high
probability, essentially only the optimal branches is indefinitely developed.
We illustrate these methods on a global optimization problem of a Lipschitz
function, given noisy data
Pure Exploration for Multi-Armed Bandit Problems
We consider the framework of stochastic multi-armed bandit problems and study
the possibilities and limitations of forecasters that perform an on-line
exploration of the arms. These forecasters are assessed in terms of their
simple regret, a regret notion that captures the fact that exploration is only
constrained by the number of available rounds (not necessarily known in
advance), in contrast to the case when the cumulative regret is considered and
when exploitation needs to be performed at the same time. We believe that this
performance criterion is suited to situations when the cost of pulling an arm
is expressed in terms of resources rather than rewards. We discuss the links
between the simple and the cumulative regret. One of the main results in the
case of a finite number of arms is a general lower bound on the simple regret
of a forecaster in terms of its cumulative regret: the smaller the latter, the
larger the former. Keeping this result in mind, we then exhibit upper bounds on
the simple regret of some forecasters. The paper ends with a study devoted to
continuous-armed bandit problems; we show that the simple regret can be
minimized with respect to a family of probability distributions if and only if
the cumulative regret can be minimized for it. Based on this equivalence, we
are able to prove that the separable metric spaces are exactly the metric
spaces on which these regrets can be minimized with respect to the family of
all probability distributions with continuous mean-payoff functions
Best-Arm Identification in Linear Bandits
We study the best-arm identification problem in linear bandit, where the
rewards of the arms depend linearly on an unknown parameter and the
objective is to return the arm with the largest reward. We characterize the
complexity of the problem and introduce sample allocation strategies that pull
arms to identify the best arm with a fixed confidence, while minimizing the
sample budget. In particular, we show the importance of exploiting the global
linear structure to improve the estimate of the reward of near-optimal arms. We
analyze the proposed strategies and compare their empirical performance.
Finally, as a by-product of our analysis, we point out the connection to the
-optimality criterion used in optimal experimental design.Comment: In Advances in Neural Information Processing Systems 27 (NIPS), 201
- …