992 research outputs found
Scalable discrete sampling as a multi-armed bandit problem
Drawing a sample from a discrete distribution is one of the building
components for Monte Carlo methods. Like other sampling algorithms, discrete
sampling suffers from the high computational burden in large-scale inference
problems. We study the problem of sampling a discrete random variable with a
high degree of dependency that is typical in large-scale Bayesian inference and
graphical models, and propose an efficient approximate solution with a
subsampling approach. We make a novel connection between the discrete sampling
and Multi-Armed Bandits problems with a finite reward population and provide
three algorithms with theoretical guarantees. Empirical evaluations show the
robustness and efficiency of the approximate algorithms in both synthetic and
real-world large-scale problems.We acknowledge
funding from the Alan Turing Institute, Google,
Microsoft Research and EPSRC Grant EP/I036575/1.This is the accepted manuscript. It is currently embargoed pending publication
Influence Maximization with Bandits
We consider the problem of \emph{influence maximization}, the problem of
maximizing the number of people that become aware of a product by finding the
`best' set of `seed' users to expose the product to. Most prior work on this
topic assumes that we know the probability of each user influencing each other
user, or we have data that lets us estimate these influences. However, this
information is typically not initially available or is difficult to obtain. To
avoid this assumption, we adopt a combinatorial multi-armed bandit paradigm
that estimates the influence probabilities as we sequentially try different
seed sets. We establish bounds on the performance of this procedure under the
existing edge-level feedback as well as a novel and more realistic node-level
feedback. Beyond our theoretical results, we describe a practical
implementation and experimentally demonstrate its efficiency and effectiveness
on four real datasets.Comment: 12 page
Scalable Multiagent Coordination with Distributed Online Open Loop Planning
We propose distributed online open loop planning (DOOLP), a general framework
for online multiagent coordination and decision making under uncertainty. DOOLP
is based on online heuristic search in the space defined by a generative model
of the domain dynamics, which is exploited by agents to simulate and evaluate
the consequences of their potential choices.
We also propose distributed online Thompson sampling (DOTS) as an effective
instantiation of the DOOLP framework. DOTS models sequences of agent choices by
concatenating a number of multiarmed bandits for each agent and uses Thompson
sampling for dealing with action value uncertainty. The Bayesian approach
underlying Thompson sampling allows to effectively model and estimate
uncertainty about (a) own action values and (b) other agents' behavior. This
approach yields a principled and statistically sound solution to the
exploration-exploitation dilemma when exploring large search spaces with
limited resources.
We implemented DOTS in a smart factory case study with positive empirical
results. We observed effective, robust and scalable planning and coordination
capabilities even when only searching a fraction of the potential search space
- …