164 research outputs found
Matroid Bandits: Fast Combinatorial Optimization with Learning
A matroid is a notion of independence in combinatorial optimization which is
closely related to computational efficiency. In particular, it is well known
that the maximum of a constrained modular function can be found greedily if and
only if the constraints are associated with a matroid. In this paper, we bring
together the ideas of bandits and matroids, and propose a new class of
combinatorial bandits, matroid bandits. The objective in these problems is to
learn how to maximize a modular function on a matroid. This function is
stochastic and initially unknown. We propose a practical algorithm for solving
our problem, Optimistic Matroid Maximization (OMM); and prove two upper bounds,
gap-dependent and gap-free, on its regret. Both bounds are sublinear in time
and at most linear in all other quantities of interest. The gap-dependent upper
bound is tight and we prove a matching lower bound on a partition matroid
bandit. Finally, we evaluate our method on three real-world problems and show
that it is practical
Trend Detection based Regret Minimization for Bandit Problems
We study a variation of the classical multi-armed bandits problem. In this
problem, the learner has to make a sequence of decisions, picking from a fixed
set of choices. In each round, she receives as feedback only the loss incurred
from the chosen action. Conventionally, this problem has been studied when
losses of the actions are drawn from an unknown distribution or when they are
adversarial. In this paper, we study this problem when the losses of the
actions also satisfy certain structural properties, and especially, do show a
trend structure. When this is true, we show that using \textit{trend
detection}, we can achieve regret of order with
respect to a switching strategy for the version of the problem where a single
action is chosen in each round and when actions
are chosen each round. This guarantee is a significant improvement over the
conventional benchmark. Our approach can, as a framework, be applied in
combination with various well-known bandit algorithms, like Exp3. For both
versions of the problem, we give regret guarantees also for the
\textit{anytime} setting, i.e. when the length of the choice-sequence is not
known in advance. Finally, we pinpoint the advantages of our method by
comparing it to some well-known other strategies
The Price of Information in Combinatorial Optimization
Consider a network design application where we wish to lay down a
minimum-cost spanning tree in a given graph; however, we only have stochastic
information about the edge costs. To learn the precise cost of any edge, we
have to conduct a study that incurs a price. Our goal is to find a spanning
tree while minimizing the disutility, which is the sum of the tree cost and the
total price that we spend on the studies. In a different application, each edge
gives a stochastic reward value. Our goal is to find a spanning tree while
maximizing the utility, which is the tree reward minus the prices that we pay.
Situations such as the above two often arise in practice where we wish to
find a good solution to an optimization problem, but we start with only some
partial knowledge about the parameters of the problem. The missing information
can be found only after paying a probing price, which we call the price of
information. What strategy should we adopt to optimize our expected
utility/disutility?
A classical example of the above setting is Weitzman's "Pandora's box"
problem where we are given probability distributions on values of
independent random variables. The goal is to choose a single variable with a
large value, but we can find the actual outcomes only after paying a price. Our
work is a generalization of this model to other combinatorial optimization
problems such as matching, set cover, facility location, and prize-collecting
Steiner tree. We give a technique that reduces such problems to their non-price
counterparts, and use it to design exact/approximation algorithms to optimize
our utility/disutility. Our techniques extend to situations where there are
additional constraints on what parameters can be probed or when we can
simultaneously probe a subset of the parameters.Comment: SODA 201
- …