2,138 research outputs found
A Relative Exponential Weighing Algorithm for Adversarial Utility-based Dueling Bandits
We study the K-armed dueling bandit problem which is a variation of the
classical Multi-Armed Bandit (MAB) problem in which the learner receives only
relative feedback about the selected pairs of arms. We propose a new algorithm
called Relative Exponential-weight algorithm for Exploration and Exploitation
(REX3) to handle the adversarial utility-based formulation of this problem.
This algorithm is a non-trivial extension of the Exponential-weight algorithm
for Exploration and Exploitation (EXP3) algorithm. We prove a finite time
expected regret upper bound of order O(sqrt(K ln(K)T)) for this algorithm and a
general lower bound of order omega(sqrt(KT)). At the end, we provide
experimental results using real data from information retrieval applications
Relative Upper Confidence Bound for the K-Armed Dueling Bandit Problem
This paper proposes a new method for the K-armed dueling bandit problem, a
variation on the regular K-armed bandit problem that offers only relative
feedback about pairs of arms. Our approach extends the Upper Confidence Bound
algorithm to the relative setting by using estimates of the pairwise
probabilities to select a promising arm and applying Upper Confidence Bound
with the winner as a benchmark. We prove a finite-time regret bound of order
O(log t). In addition, our empirical results using real data from an
information retrieval application show that it greatly outperforms the state of
the art.Comment: 13 pages, 6 figure
Regret Bounds and Regimes of Optimality for User-User and Item-Item Collaborative Filtering
We consider an online model for recommendation systems, with each user being
recommended an item at each time-step and providing 'like' or 'dislike'
feedback. Each user may be recommended a given item at most once. A latent
variable model specifies the user preferences: both users and items are
clustered into types. All users of a given type have identical preferences for
the items, and similarly, items of a given type are either all liked or all
disliked by a given user. We assume that the matrix encoding the preferences of
each user type for each item type is randomly generated; in this way, the model
captures structure in both the item and user spaces, the amount of structure
depending on the number of each of the types. The measure of performance of the
recommendation system is the expected number of disliked recommendations per
user, defined as expected regret. We propose two algorithms inspired by
user-user and item-item collaborative filtering (CF), modified to explicitly
make exploratory recommendations, and prove performance guarantees in terms of
their expected regret. For two regimes of model parameters, with structure only
in item space or only in user space, we prove information-theoretic lower
bounds on regret that match our upper bounds up to logarithmic factors. Our
analysis elucidates system operating regimes in which existing CF algorithms
are nearly optimal.Comment: 51 page
A quadratic lower bound for Rocchio’s similarity-based relevance feedback algorithm with a fixed query updating factor
Rocchio’s similarity-based relevance feedback algorithm, one of the most important query reformation methods in information retrieval, is essentially an adaptive supervised learning algorithm from examples. In practice, Rocchio’s algorithm often uses a fixed query updating factor. When this is the case, we strengthen the linear Ω(n) lower bound obtained by Chen and Zhu (Inf. Retr. 5:61–86, 2002) and prove that Rocchio’s algorithm makes Ω(k(n−k)) mistakes in searching for a collection of documents represented by a monotone disjunction of k relevant features over the n-dimensional binary vector space {0,1}n, when the inner product similarity measure is used. A quadratic lower bound is obtained when k is linearly proportional to n. We also prove an O(k(n−k)3) upper bound for Rocchio’s algorithm with the inner product similarity measure in searching for such a collection of documents with a constant query updating factor and a zero classification threshold
- …