13 research outputs found
Incentivizing Exploration with Heterogeneous Value of Money
Recently, Frazier et al. proposed a natural model for crowdsourced
exploration of different a priori unknown options: a principal is interested in
the long-term welfare of a population of agents who arrive one by one in a
multi-armed bandit setting. However, each agent is myopic, so in order to
incentivize him to explore options with better long-term prospects, the
principal must offer the agent money. Frazier et al. showed that a simple class
of policies called time-expanded are optimal in the worst case, and
characterized their budget-reward tradeoff.
The previous work assumed that all agents are equally and uniformly
susceptible to financial incentives. In reality, agents may have different
utility for money. We therefore extend the model of Frazier et al. to allow
agents that have heterogeneous and non-linear utilities for money. The
principal is informed of the agent's tradeoff via a signal that could be more
or less informative.
Our main result is to show that a convex program can be used to derive a
signal-dependent time-expanded policy which achieves the best possible
Lagrangian reward in the worst case. The worst-case guarantee is matched by
so-called "Diamonds in the Rough" instances; the proof that the guarantees
match is based on showing that two different convex programs have the same
optimal solution for these specific instances. These results also extend to the
budgeted case as in Frazier et al. We also show that the optimal policy is
monotone with respect to information, i.e., the approximation ratio of the
optimal policy improves as the signals become more informative.Comment: WINE 201
Approximation Algorithms for Correlated Knapsacks and Non-Martingale Bandits
In the stochastic knapsack problem, we are given a knapsack of size B, and a
set of jobs whose sizes and rewards are drawn from a known probability
distribution. However, we know the actual size and reward only when the job
completes. How should we schedule jobs to maximize the expected total reward?
We know O(1)-approximations when we assume that (i) rewards and sizes are
independent random variables, and (ii) we cannot prematurely cancel jobs. What
can we say when either or both of these assumptions are changed?
The stochastic knapsack problem is of interest in its own right, but
techniques developed for it are applicable to other stochastic packing
problems. Indeed, ideas for this problem have been useful for budgeted learning
problems, where one is given several arms which evolve in a specified
stochastic fashion with each pull, and the goal is to pull the arms a total of
B times to maximize the reward obtained. Much recent work on this problem focus
on the case when the evolution of the arms follows a martingale, i.e., when the
expected reward from the future is the same as the reward at the current state.
What can we say when the rewards do not form a martingale?
In this paper, we give constant-factor approximation algorithms for the
stochastic knapsack problem with correlations and/or cancellations, and also
for budgeted learning problems where the martingale condition is not satisfied.
Indeed, we can show that previously proposed LP relaxations have large
integrality gaps. We propose new time-indexed LP relaxations, and convert the
fractional solutions into distributions over strategies, and then use the LP
values and the time ordering information from these strategies to devise a
randomized adaptive scheduling algorithm. We hope our LP formulation and
decomposition methods may provide a new way to address other correlated bandit
problems with more general contexts
Matroid prophet inequalities and Bayesian mechanism design
Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 42-44).Consider a gambler who observes a sequence of independent, non-negative random numbers and is allowed to stop the sequence at any time, claiming a reward equal to the most recent observation. The famous prophet inequality of Krengel, Sucheston, and Garling asserts that a gambler who knows the distribution of each random variable can achieve at least half as much reward, in expectation, as a "prophet" who knows the sampled values of each random variable and can choose the largest one. We generalize this result to the setting in which the gambler and the prophet are allowed to make more than one selection, subject to a matroid constraint. We show that the gambler can still achieve at least half as much reward as the prophet; this result is the best possible, since it is known that the ratio cannot be improved even in the original prophet inequality, which corresponds to the special case of rank-one matroids. Generalizing the result still further, we show that under an intersection of p matroid constraints, the prophet's reward exceeds the gambler's by a factor of at most 0(p), and this factor is also tight. Beyond their interest as theorems about pure online algoritms or optimal stopping rules, these results also have applications to mechanism design. Our results imply improved bounds on the ability of sequential posted-price mechanisms to approximate optimal mechanisms in both single-parameter and multi-parameter Bayesian settings. In particular, our results imply the first efficiently computable constant-factor approximations to the Bayesian optimal revenue in certain multi-parameter settings. This work was done in collaboration with Robert Kleinberg.by S. Matthew Weinberg.S.M