4 research outputs found
MaxGap Bandit: Adaptive Algorithms for Approximate Ranking
This paper studies the problem of adaptively sampling from K distributions
(arms) in order to identify the largest gap between any two adjacent means. We
call this the MaxGap-bandit problem. This problem arises naturally in
approximate ranking, noisy sorting, outlier detection, and top-arm
identification in bandits. The key novelty of the MaxGap-bandit problem is that
it aims to adaptively determine the natural partitioning of the distributions
into a subset with larger means and a subset with smaller means, where the
split is determined by the largest gap rather than a pre-specified rank or
threshold. Estimating an arm's gap requires sampling its neighboring arms in
addition to itself, and this dependence results in a novel hardness parameter
that characterizes the sample complexity of the problem. We propose elimination
and UCB-style algorithms and show that they are minimax optimal. Our
experiments show that the UCB-style algorithms require 6-8x fewer samples than
non-adaptive sampling to achieve the same error
Combinatorial Pure Exploration with Continuous and Separable Reward Functions and Its Applications
We study the Combinatorial Pure Exploration problem with Continuous and Separable reward functions (CPE-CS) in the stochastic multi-armed bandit setting. In a CPE-CS instance, we are given several stochastic arms with unknown distributions, as well as a collection of possible decisions. Each decision has a reward according to the distributions of arms. The goal is to identify the decision with the maximum reward, using as few arm samples as possible. The problem generalizes the combinatorial pure exploration problem with linear rewards, which has attracted significant attention in recent years. In this paper, we propose an adaptive learning algorithm for the CPE-CS problem, and analyze its sample complexity. In particular, we introduce a new hardness measure called the consistent optimality hardness, and give both the upper and lower bounds of sample complexity. Moreover, we give examples to demonstrate that our solution has the capacity to deal with non-linear reward functions.1