6 research outputs found
BPR: Bayesian Personalized Ranking from Implicit Feedback
Item recommendation is the task of predicting a personalized ranking on a set
of items (e.g. websites, movies, products). In this paper, we investigate the
most common scenario with implicit feedback (e.g. clicks, purchases). There are
many methods for item recommendation from implicit feedback like matrix
factorization (MF) or adaptive knearest-neighbor (kNN). Even though these
methods are designed for the item prediction task of personalized ranking, none
of them is directly optimized for ranking. In this paper we present a generic
optimization criterion BPR-Opt for personalized ranking that is the maximum
posterior estimator derived from a Bayesian analysis of the problem. We also
provide a generic learning algorithm for optimizing models with respect to
BPR-Opt. The learning method is based on stochastic gradient descent with
bootstrap sampling. We show how to apply our method to two state-of-the-art
recommender models: matrix factorization and adaptive kNN. Our experiments
indicate that for the task of personalized ranking our optimization method
outperforms the standard learning techniques for MF and kNN. The results show
the importance of optimizing models for the right criterion.Comment: Appears in Proceedings of the Twenty-Fifth Conference on Uncertainty
in Artificial Intelligence (UAI2009
Fastfood: Approximate Kernel Expansions in Loglinear Time
Despite their successes, what makes kernel methods difficult to use in many
large scale problems is the fact that storing and computing the decision
function is typically expensive, especially at prediction time. In this paper,
we overcome this difficulty by proposing Fastfood, an approximation that
accelerates such computation significantly. Key to Fastfood is the observation
that Hadamard matrices, when combined with diagonal Gaussian matrices, exhibit
properties similar to dense Gaussian random matrices. Yet unlike the latter,
Hadamard and diagonal matrices are inexpensive to multiply and store. These two
matrices can be used in lieu of Gaussian matrices in Random Kitchen Sinks
proposed by Rahimi and Recht (2009) and thereby speeding up the computation for
a large range of kernel functions. Specifically, Fastfood requires O(n log d)
time and O(n) storage to compute n non-linear basis functions in d dimensions,
a significant improvement from O(nd) computation and storage, without
sacrificing accuracy.
Our method applies to any translation invariant and any dot-product kernel,
such as the popular RBF kernels and polynomial kernels. We prove that the
approximation is unbiased and has low variance. Experiments show that we
achieve similar accuracy to full kernel expansions and Random Kitchen Sinks
while being 100x faster and using 1000x less memory. These improvements,
especially in terms of memory usage, make kernel methods more practical for
applications that have large training sets and/or require real-time prediction
Uncovering the Riffled Independence Structure of Rankings
Representing distributions over permutations can be a daunting task due to
the fact that the number of permutations of objects scales factorially in
. One recent way that has been used to reduce storage complexity has been to
exploit probabilistic independence, but as we argue, full independence
assumptions impose strong sparsity constraints on distributions and are
unsuitable for modeling rankings. We identify a novel class of independence
structures, called \emph{riffled independence}, encompassing a more expressive
family of distributions while retaining many of the properties necessary for
performing efficient inference and reducing sample complexity. In riffled
independence, one draws two permutations independently, then performs the
\emph{riffle shuffle}, common in card games, to combine the two permutations to
form a single permutation. Within the context of ranking, riffled independence
corresponds to ranking disjoint sets of objects independently, then
interleaving those rankings. In this paper, we provide a formal introduction to
riffled independence and present algorithms for using riffled independence
within Fourier-theoretic frameworks which have been explored by a number of
recent papers. Additionally, we propose an automated method for discovering
sets of items which are riffle independent from a training set of rankings. We
show that our clustering-like algorithms can be used to discover meaningful
latent coalitions from real preference ranking datasets and to learn the
structure of hierarchically decomposable models based on riffled independence.Comment: 65 page
A Multiresolution Analysis Framework for the Statistical Analysis of Incomplete Rankings
Though the statistical analysis of ranking data has been a subject of
interest over the past centuries, especially in economics, psychology or social
choice theory, it has been revitalized in the past 15 years by recent
applications such as recommender or search engines and is receiving now
increasing interest in the machine learning literature. Numerous modern systems
indeed generate ranking data, representing for instance ordered results to a
query or user preferences. Each such ranking usually involves a small but
varying subset of the whole catalog of items only. The study of the variability
of these data, i.e. the statistical analysis of incomplete rank-ings, is
however a great statistical and computational challenge, because of their
heterogeneity and the related combinatorial complexity of the problem. Whereas
many statistical methods for analyzing full rankings (orderings of all the
items in the catalog) are documented in the dedicated literature, partial
rankings (full rankings with ties) or pairwise comparisons, only a few
approaches are available today to deal with incomplete ranking, relying each on
a strong specific assumption. It is the purpose of this article to introduce a
novel general framework for the statistical analysis of incomplete rankings. It
is based on a representation tailored to these specific data, whose
construction is also explained here, which fits with the natural multi-scale
structure of incomplete rankings and provides a new decomposition of rank
information with a multiresolu-tion analysis interpretation (MRA). We show that
the MRA representation naturally allows to overcome both the statistical and
computational challenges without any structural assumption on the data. It
therefore provides a general and flexible framework to solve a wide variety of
statistical problems, where data are of the form of incomplete rankings
State-Space Abstractions for Probabilistic Inference: A Systematic Review
Tasks such as social network analysis, human behavior recognition, or
modeling biochemical reactions, can be solved elegantly by using the
probabilistic inference framework. However, standard probabilistic inference
algorithms work at a propositional level, and thus cannot capture the
symmetries and redundancies that are present in these tasks. Algorithms that
exploit those symmetries have been devised in different research fields, for
example by the lifted inference-, multiple object tracking-, and modeling and
simulation-communities. The common idea, that we call state space abstraction,
is to perform inference over compact representations of sets of symmetric
states. Although they are concerned with a similar topic, the relationship
between these approaches has not been investigated systematically. This survey
provides the following contributions. We perform a systematic literature review
to outline the state of the art in probabilistic inference methods exploiting
symmetries. From an initial set of more than 4,000 papers, we identify 116
relevant papers. Furthermore, we provide new high-level categories that
classify the approaches, based on common properties of the approaches. The
research areas underlying each of the categories are introduced concisely.
Researchers from different fields that are confronted with a state space
explosion problem in a probabilistic system can use this classification to
identify possible solutions. Finally, based on this conceptualization, we
identify potentials for future research, as some relevant application domains
are not addressed by current approaches
Signal Processing on the Permutahedron: Tight Spectral Frames for Ranked Data Analysis
Ranked data sets, where m judges/voters specify a preference ranking of n
objects/candidates, are increasingly prevalent in contexts such as political
elections, computer vision, recommender systems, and bioinformatics. The vote
counts for each ranking can be viewed as an n! data vector lying on the
permutahedron, which is a Cayley graph of the symmetric group with vertices
labeled by permutations and an edge when two permutations differ by an adjacent
transposition. Leveraging combinatorial representation theory and recent
progress in signal processing on graphs, we investigate a novel, scalable
transform method to interpret and exploit structure in ranked data. We
represent data on the permutahedron using an overcomplete dictionary of atoms,
each of which captures both smoothness information about the data (typically
the focus of spectral graph decomposition methods in graph signal processing)
and structural information about the data (typically the focus of symmetry
decomposition methods from representation theory). These atoms have a more
naturally interpretable structure than any known basis for signals on the
permutahedron, and they form a Parseval frame, ensuring beneficial numerical
properties such as energy preservation. We develop specialized algorithms and
open software that take advantage of the symmetry and structure of the
permutahedron to improve the scalability of the proposed method, making it more
applicable to the high-dimensional ranked data found in applications