22,461 research outputs found
Hierarchical Compound Poisson Factorization
Non-negative matrix factorization models based on a hierarchical
Gamma-Poisson structure capture user and item behavior effectively in extremely
sparse data sets, making them the ideal choice for collaborative filtering
applications. Hierarchical Poisson factorization (HPF) in particular has proved
successful for scalable recommendation systems with extreme sparsity. HPF,
however, suffers from a tight coupling of sparsity model (absence of a rating)
and response model (the value of the rating), which limits the expressiveness
of the latter. Here, we introduce hierarchical compound Poisson factorization
(HCPF) that has the favorable Gamma-Poisson structure and scalability of HPF to
high-dimensional extremely sparse matrices. More importantly, HCPF decouples
the sparsity model from the response model, allowing us to choose the most
suitable distribution for the response. HCPF can capture binary, non-negative
discrete, non-negative continuous, and zero-inflated continuous responses. We
compare HCPF with HPF on nine discrete and three continuous data sets and
conclude that HCPF captures the relationship between sparsity and response
better than HPF.Comment: Will appear on Proceedings of the 33 rd International Conference on
Machine Learning, New York, NY, USA, 2016. JMLR: W&CP volume 4
Discrete Factorization Machines for Fast Feature-based Recommendation
User and item features of side information are crucial for accurate
recommendation. However, the large number of feature dimensions, e.g., usually
larger than 10^7, results in expensive storage and computational cost. This
prohibits fast recommendation especially on mobile applications where the
computational resource is very limited. In this paper, we develop a generic
feature-based recommendation model, called Discrete Factorization Machine
(DFM), for fast and accurate recommendation. DFM binarizes the real-valued
model parameters (e.g., float32) of every feature embedding into binary codes
(e.g., boolean), and thus supports efficient storage and fast user-item score
computation. To avoid the severe quantization loss of the binarization, we
propose a convergent updating rule that resolves the challenging discrete
optimization of DFM. Through extensive experiments on two real-world datasets,
we show that 1) DFM consistently outperforms state-of-the-art binarized
recommendation models, and 2) DFM shows very competitive performance compared
to its real-valued version (FM), demonstrating the minimized quantization loss.
This work is accepted by IJCAI 2018.Comment: Appeared in IJCAI 201
Dynamic Poisson Factorization
Models for recommender systems use latent factors to explain the preferences
and behaviors of users with respect to a set of items (e.g., movies, books,
academic papers). Typically, the latent factors are assumed to be static and,
given these factors, the observed preferences and behaviors of users are
assumed to be generated without order. These assumptions limit the explorative
and predictive capabilities of such models, since users' interests and item
popularity may evolve over time. To address this, we propose dPF, a dynamic
matrix factorization model based on the recent Poisson factorization model for
recommendations. dPF models the time evolving latent factors with a Kalman
filter and the actions with Poisson distributions. We derive a scalable
variational inference algorithm to infer the latent factors. Finally, we
demonstrate dPF on 10 years of user click data from arXiv.org, one of the
largest repository of scientific papers and a formidable source of information
about the behavior of scientists. Empirically we show performance improvement
over both static and, more recently proposed, dynamic recommendation models. We
also provide a thorough exploration of the inferred posteriors over the latent
variables.Comment: RecSys 201
Automatic Variational Inference in Stan
Variational inference is a scalable technique for approximate Bayesian
inference. Deriving variational inference algorithms requires tedious
model-specific calculations; this makes it difficult to automate. We propose an
automatic variational inference algorithm, automatic differentiation
variational inference (ADVI). The user only provides a Bayesian model and a
dataset; nothing else. We make no conjugacy assumptions and support a broad
class of models. The algorithm automatically determines an appropriate
variational family and optimizes the variational objective. We implement ADVI
in Stan (code available now), a probabilistic programming framework. We compare
ADVI to MCMC sampling across hierarchical generalized linear models,
nonconjugate matrix factorization, and a mixture model. We train the mixture
model on a quarter million images. With ADVI we can use variational inference
on any model we write in Stan
Learning computationally efficient dictionaries and their implementation as fast transforms
Dictionary learning is a branch of signal processing and machine learning
that aims at finding a frame (called dictionary) in which some training data
admits a sparse representation. The sparser the representation, the better the
dictionary. The resulting dictionary is in general a dense matrix, and its
manipulation can be computationally costly both at the learning stage and later
in the usage of this dictionary, for tasks such as sparse coding. Dictionary
learning is thus limited to relatively small-scale problems. In this paper,
inspired by usual fast transforms, we consider a general dictionary structure
that allows cheaper manipulation, and propose an algorithm to learn such
dictionaries --and their fast implementation-- over training data. The approach
is demonstrated experimentally with the factorization of the Hadamard matrix
and with synthetic dictionary learning experiments
Automatic Differentiation Variational Inference
Probabilistic modeling is iterative. A scientist posits a simple model, fits
it to her data, refines it according to her analysis, and repeats. However,
fitting complex models to large data is a bottleneck in this process. Deriving
algorithms for new models can be both mathematically and computationally
challenging, which makes it difficult to efficiently cycle through the steps.
To this end, we develop automatic differentiation variational inference (ADVI).
Using our method, the scientist only provides a probabilistic model and a
dataset, nothing else. ADVI automatically derives an efficient variational
inference algorithm, freeing the scientist to refine and explore many models.
ADVI supports a broad class of models-no conjugacy assumptions are required. We
study ADVI across ten different models and apply it to a dataset with millions
of observations. ADVI is integrated into Stan, a probabilistic programming
system; it is available for immediate use
- …