15 research outputs found
MaxGap Bandit: Adaptive Algorithms for Approximate Ranking
This paper studies the problem of adaptively sampling from K distributions
(arms) in order to identify the largest gap between any two adjacent means. We
call this the MaxGap-bandit problem. This problem arises naturally in
approximate ranking, noisy sorting, outlier detection, and top-arm
identification in bandits. The key novelty of the MaxGap-bandit problem is that
it aims to adaptively determine the natural partitioning of the distributions
into a subset with larger means and a subset with smaller means, where the
split is determined by the largest gap rather than a pre-specified rank or
threshold. Estimating an arm's gap requires sampling its neighboring arms in
addition to itself, and this dependence results in a novel hardness parameter
that characterizes the sample complexity of the problem. We propose elimination
and UCB-style algorithms and show that they are minimax optimal. Our
experiments show that the UCB-style algorithms require 6-8x fewer samples than
non-adaptive sampling to achieve the same error
Top Arm Identification in Multi-Armed Bandits with Batch Arm Pulls
Abstract We introduce a new multi-armed bandit (MAB) problem in which arms must be sampled in batches, rather than one at a time. This is motivated by applications in social media monitoring and biological experimentation where such batch constraints naturally arise. This paper develops and analyzes algorithms for batch MABs and top arm identification, for both fixed confidence and fixed budget settings. Our main theoretical results show that the batch constraint does not significantly a↵ect the sample complexity of top arm identification compared to unconstrained MAB algorithms. Alternatively, if one views a batch as the fundamental sampling unit, then the results can be interpreted as showing that the sample complexity of batch MABs can be significantly less than traditional MABs. We demonstrate the new batch MAB algorithms with simulations and in two interesting real-world applications: (i) microwell array experiments for identifying genes that are important in virus replication and (ii) finding the most active users in Twitter on a specific topic