73 research outputs found
Influence Maximization with Bandits
We consider the problem of \emph{influence maximization}, the problem of
maximizing the number of people that become aware of a product by finding the
`best' set of `seed' users to expose the product to. Most prior work on this
topic assumes that we know the probability of each user influencing each other
user, or we have data that lets us estimate these influences. However, this
information is typically not initially available or is difficult to obtain. To
avoid this assumption, we adopt a combinatorial multi-armed bandit paradigm
that estimates the influence probabilities as we sequentially try different
seed sets. We establish bounds on the performance of this procedure under the
existing edge-level feedback as well as a novel and more realistic node-level
feedback. Beyond our theoretical results, we describe a practical
implementation and experimentally demonstrate its efficiency and effectiveness
on four real datasets.Comment: 12 page
Linear Combinatorial Semi-Bandit with Causally Related Rewards
In a sequential decision-making problem, having a structural dependency
amongst the reward distributions associated with the arms makes it challenging
to identify a subset of alternatives that guarantees the optimal collective
outcome. Thus, besides individual actions' reward, learning the causal
relations is essential to improve the decision-making strategy. To solve the
two-fold learning problem described above, we develop the 'combinatorial
semi-bandit framework with causally related rewards', where we model the causal
relations by a directed graph in a stationary structural equation model. The
nodal observation in the graph signal comprises the corresponding base arm's
instantaneous reward and an additional term resulting from the causal
influences of other base arms' rewards. The objective is to maximize the
long-term average payoff, which is a linear function of the base arms' rewards
and depends strongly on the network topology. To achieve this objective, we
propose a policy that determines the causal relations by learning the network's
topology and simultaneously exploits this knowledge to optimize the
decision-making process. We establish a sublinear regret bound for the proposed
algorithm. Numerical experiments using synthetic and real-world datasets
demonstrate the superior performance of our proposed method compared to several
benchmarks
Finding the bandit in a graph: Sequential search-and-stop
We consider the problem where an agent wants to find a hidden object that is
randomly located in some vertex of a directed acyclic graph (DAG) according to
a fixed but possibly unknown distribution. The agent can only examine vertices
whose in-neighbors have already been examined. In this paper, we address a
learning setting where we allow the agent to stop before having found the
object and restart searching on a new independent instance of the same problem.
Our goal is to maximize the total number of hidden objects found given a time
budget. The agent can thus skip an instance after realizing that it would spend
too much time on it. Our contributions are both to the search theory and
multi-armed bandits. If the distribution is known, we provide a quasi-optimal
and efficient stationary strategy. If the distribution is unknown, we
additionally show how to sequentially approximate it and, at the same time, act
near-optimally in order to collect as many hidden objects as possible.Comment: in International Conference on Artificial Intelligence and Statistics
(AISTATS 2019), April 2019, Naha, Okinawa, Japa
- …