10,122 research outputs found
Distributed Bayesian Probabilistic Matrix Factorization
Matrix factorization is a common machine learning technique for recommender
systems. Despite its high prediction accuracy, the Bayesian Probabilistic
Matrix Factorization algorithm (BPMF) has not been widely used on large scale
data because of its high computational cost. In this paper we propose a
distributed high-performance parallel implementation of BPMF on shared memory
and distributed architectures. We show by using efficient load balancing using
work stealing on a single node, and by using asynchronous communication in the
distributed version we beat state of the art implementations
A Bayesian Perspective for Determinant Minimization Based Robust Structured Matrix Factorizatio
We introduce a Bayesian perspective for the structured matrix factorization
problem. The proposed framework provides a probabilistic interpretation for
existing geometric methods based on determinant minimization. We model input
data vectors as linear transformations of latent vectors drawn from a
distribution uniform over a particular domain reflecting structural
assumptions, such as the probability simplex in Nonnegative Matrix
Factorization and polytopes in Polytopic Matrix Factorization. We represent the
rows of the linear transformation matrix as vectors generated independently
from a normal distribution whose covariance matrix is inverse Wishart
distributed. We show that the corresponding maximum a posteriori estimation
problem boils down to the robust determinant minimization approach for
structured matrix factorization, providing insights about parameter selections
and potential algorithmic extensions
Bayesian Probabilistic Matrix Factorization: A User Frequency Analysis
Matrix factorization (MF) has become a common approach to collaborative
filtering, due to ease of implementation and scalability to large data sets.
Two existing drawbacks of the basic model is that it does not incorporate side
information on either users or items, and assumes a common variance for all
users. We extend the work of constrained probabilistic matrix factorization by
deriving the Gibbs updates for the side feature vectors for items
(Salakhutdinov and Minh, 2008). We show that this Bayesian treatment to the
constrained PMF model outperforms simple MAP estimation. We also consider
extensions to heteroskedastic precision introduced in the literature
(Lakshminarayanan, Bouchard, and Archambeau, 2011). We show that this tends
result in overfitting for deterministic approximation algorithms (ex:
Variational inference) when the observed entries in the user / item matrix are
distributed in an non-uniform manner. In light of this, we propose a truncated
precision model. Our experimental results suggest that this model tends to
delay overfitting
Distributed Bayesian Matrix Factorization with Limited Communication
Bayesian matrix factorization (BMF) is a powerful tool for producing low-rank
representations of matrices and for predicting missing values and providing
confidence intervals. Scaling up the posterior inference for massive-scale
matrices is challenging and requires distributing both data and computation
over many workers, making communication the main computational bottleneck.
Embarrassingly parallel inference would remove the communication needed, by
using completely independent computations on different data subsets, but it
suffers from the inherent unidentifiability of BMF solutions. We introduce a
hierarchical decomposition of the joint posterior distribution, which couples
the subset inferences, allowing for embarrassingly parallel computations in a
sequence of at most three stages. Using an efficient approximate
implementation, we show improvements empirically on both real and simulated
data. Our distributed approach is able to achieve a speed-up of almost an order
of magnitude over the full posterior, with a negligible effect on predictive
accuracy. Our method outperforms state-of-the-art embarrassingly parallel MCMC
methods in accuracy, and achieves results competitive to other available
distributed and parallel implementations of BMF.Comment: 28 pages, 8 figures. The paper is published in Machine Learning
journal. An implementation of the method is is available in SMURFF software
on github (bmfpp branch): https://github.com/ExaScience/smurf
Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Processes
Probabilistic matrix factorization (PMF) is a powerful method for modeling
data associated with pairwise relationships, finding use in collaborative
filtering, computational biology, and document analysis, among other areas. In
many domains, there is additional information that can assist in prediction.
For example, when modeling movie ratings, we might know when the rating
occurred, where the user lives, or what actors appear in the movie. It is
difficult, however, to incorporate this side information into the PMF model. We
propose a framework for incorporating side information by coupling together
multiple PMF problems via Gaussian process priors. We replace scalar latent
features with functions that vary over the space of side information. The GP
priors on these functions require them to vary smoothly and share information.
We successfully use this new method to predict the scores of professional
basketball games, where side information about the venue and date of the game
are relevant for the outcome.Comment: 18 pages, 4 figures, Submitted to UAI 201
Dynamic Poisson Factorization
Models for recommender systems use latent factors to explain the preferences
and behaviors of users with respect to a set of items (e.g., movies, books,
academic papers). Typically, the latent factors are assumed to be static and,
given these factors, the observed preferences and behaviors of users are
assumed to be generated without order. These assumptions limit the explorative
and predictive capabilities of such models, since users' interests and item
popularity may evolve over time. To address this, we propose dPF, a dynamic
matrix factorization model based on the recent Poisson factorization model for
recommendations. dPF models the time evolving latent factors with a Kalman
filter and the actions with Poisson distributions. We derive a scalable
variational inference algorithm to infer the latent factors. Finally, we
demonstrate dPF on 10 years of user click data from arXiv.org, one of the
largest repository of scientific papers and a formidable source of information
about the behavior of scientists. Empirically we show performance improvement
over both static and, more recently proposed, dynamic recommendation models. We
also provide a thorough exploration of the inferred posteriors over the latent
variables.Comment: RecSys 201
- …