2,604 research outputs found
Boolean Matrix Factorization and Noisy Completion via Message Passing
Boolean matrix factorization and Boolean matrix completion from noisy
observations are desirable unsupervised data-analysis methods due to their
interpretability, but hard to perform due to their NP-hardness. We treat these
problems as maximum a posteriori inference problems in a graphical model and
present a message passing approach that scales linearly with the number of
observations and factors. Our empirical study demonstrates that message passing
is able to recover low-rank Boolean matrices, in the boundaries of
theoretically possible recovery and compares favorably with state-of-the-art in
real-world applications, such collaborative filtering with large-scale Boolean
data
Fast Low-Rank Bayesian Matrix Completion with Hierarchical Gaussian Prior Models
The problem of low rank matrix completion is considered in this paper. To
exploit the underlying low-rank structure of the data matrix, we propose a
hierarchical Gaussian prior model, where columns of the low-rank matrix are
assumed to follow a Gaussian distribution with zero mean and a common precision
matrix, and a Wishart distribution is specified as a hyperprior over the
precision matrix. We show that such a hierarchical Gaussian prior has the
potential to encourage a low-rank solution. Based on the proposed hierarchical
prior model, a variational Bayesian method is developed for matrix completion,
where the generalized approximate massage passing (GAMP) technique is embedded
into the variational Bayesian inference in order to circumvent cumbersome
matrix inverse operations. Simulation results show that our proposed method
demonstrates superiority over existing state-of-the-art matrix completion
methods
Collaborative filtering via sparse Markov random fields
Recommender systems play a central role in providing individualized access to
information and services. This paper focuses on collaborative filtering, an
approach that exploits the shared structure among mind-liked users and similar
items. In particular, we focus on a formal probabilistic framework known as
Markov random fields (MRF). We address the open problem of structure learning
and introduce a sparsity-inducing algorithm to automatically estimate the
interaction structures between users and between items. Item-item and user-user
correlation networks are obtained as a by-product. Large-scale experiments on
movie recommendation and date matching datasets demonstrate the power of the
proposed method
Statistical Estimation: From Denoising to Sparse Regression and Hidden Cliques
These notes review six lectures given by Prof. Andrea Montanari on the topic
of statistical estimation for linear models. The first two lectures cover the
principles of signal recovery from linear measurements in terms of minimax
risk. Subsequent lectures demonstrate the application of these principles to
several practical problems in science and engineering. Specifically, these
topics include denoising of error-laden signals, recovery of compressively
sensed signals, reconstruction of low-rank matrices, and also the discovery of
hidden cliques within large networks.Comment: Chapter of "Statistical Physics, Optimization, Inference, and
Message-Passing Algorithms", Eds.: F. Krzakala, F. Ricci-Tersenghi, L.
Zdeborova, R. Zecchina, E. W. Tramel, L. F. Cugliandolo (Oxford University
Press, to appear
Learning Actor Relation Graphs for Group Activity Recognition
Modeling relation between actors is important for recognizing group activity
in a multi-person scene. This paper aims at learning discriminative relation
between actors efficiently using deep models. To this end, we propose to build
a flexible and efficient Actor Relation Graph (ARG) to simultaneously capture
the appearance and position relation between actors. Thanks to the Graph
Convolutional Network, the connections in ARG could be automatically learned
from group activity videos in an end-to-end manner, and the inference on ARG
could be efficiently performed with standard matrix operations. Furthermore, in
practice, we come up with two variants to sparsify ARG for more effective
modeling in videos: spatially localized ARG and temporal randomized ARG. We
perform extensive experiments on two standard group activity recognition
datasets: the Volleyball dataset and the Collective Activity dataset, where
state-of-the-art performance is achieved on both datasets. We also visualize
the learned actor graphs and relation features, which demonstrate that the
proposed ARG is able to capture the discriminative relation information for
group activity recognition.Comment: Accepted by CVPR 201
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
There has been a surge of recent interest in learning representations for
graph-structured data. Graph representation learning methods have generally
fallen into three main categories, based on the availability of labeled data.
The first, network embedding (such as shallow graph embedding or graph
auto-encoders), focuses on learning unsupervised representations of relational
structure. The second, graph regularized neural networks, leverages graphs to
augment neural network losses with a regularization objective for
semi-supervised learning. The third, graph neural networks, aims to learn
differentiable functions over discrete topologies with arbitrary structure.
However, despite the popularity of these areas there has been surprisingly
little work on unifying the three paradigms. Here, we aim to bridge the gap
between graph neural networks, network embedding and graph regularization
models. We propose a comprehensive taxonomy of representation learning methods
for graph-structured data, aiming to unify several disparate bodies of work.
Specifically, we propose a Graph Encoder Decoder Model (GRAPHEDM), which
generalizes popular algorithms for semi-supervised learning on graphs (e.g.
GraphSage, Graph Convolutional Networks, Graph Attention Networks), and
unsupervised learning of graph representations (e.g. DeepWalk, node2vec, etc)
into a single consistent approach. To illustrate the generality of this
approach, we fit over thirty existing methods into this framework. We believe
that this unifying view both provides a solid foundation for understanding the
intuition behind these methods, and enables future research in the area
Hinge-Loss Markov Random Fields and Probabilistic Soft Logic
A fundamental challenge in developing high-impact machine learning
technologies is balancing the need to model rich, structured domains with the
ability to scale to big data. Many important problem areas are both richly
structured and large scale, from social and biological networks, to knowledge
graphs and the Web, to images, video, and natural language. In this paper, we
introduce two new formalisms for modeling structured data, and show that they
can both capture rich structure and scale to big data. The first, hinge-loss
Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model
that generalizes different approaches to convex inference. We unite three
approaches from the randomized algorithms, probabilistic graphical models, and
fuzzy logic communities, showing that all three lead to the same inference
objective. We then define HL-MRFs by generalizing this unified objective. The
second new formalism, probabilistic soft logic (PSL), is a probabilistic
programming language that makes HL-MRFs easy to define using a syntax based on
first-order logic. We introduce an algorithm for inferring most-probable
variable assignments (MAP inference) that is much more scalable than
general-purpose convex optimization methods, because it uses message passing to
take advantage of sparse dependency structures. We then show how to learn the
parameters of HL-MRFs. The learned HL-MRFs are as accurate as analogous
discrete models, but much more scalable. Together, these algorithms enable
HL-MRFs and PSL to model rich, structured data at scales not previously
possible
Crowd Labeling: a survey
Recently, there has been a burst in the number of research projects on human
computation via crowdsourcing. Multiple choice (or labeling) questions could be
referred to as a common type of problem which is solved by this approach. As an
application, crowd labeling is applied to find true labels for large machine
learning datasets. Since crowds are not necessarily experts, the labels they
provide are rather noisy and erroneous. This challenge is usually resolved by
collecting multiple labels for each sample, and then aggregating them to
estimate the true label. Although the mechanism leads to high-quality labels,
it is not actually cost-effective. As a result, efforts are currently made to
maximize the accuracy in estimating true labels, while fixing the number of
acquired labels.
This paper surveys methods to aggregate redundant crowd labels in order to
estimate unknown true labels. It presents a unified statistical latent model
where the differences among popular methods in the field correspond to
different choices for the parameters of the model. Afterwards, algorithms to
make inference on these models will be surveyed. Moreover, adaptive methods
which iteratively collect labels based on the previously collected labels and
estimated models will be discussed. In addition, this paper compares the
distinguished methods, and provides guidelines for future work required to
address the current open issues.Comment: Under consideration for publication in Knowledge and Information
System
Gaussian Belief Propagation: Theory and Aplication
The canonical problem of solving a system of linear equations arises in
numerous contexts in information theory, communication theory, and related
fields. In this contribution, we develop a solution based upon Gaussian belief
propagation (GaBP) that does not involve direct matrix inversion. The iterative
nature of our approach allows for a distributed message-passing implementation
of the solution algorithm. In the first part of this thesis, we address the
properties of the GaBP solver. We characterize the rate of convergence, enhance
its message-passing efficiency by introducing a broadcast version, discuss its
relation to classical solution methods including numerical examples. We present
a new method for forcing the GaBP algorithm to converge to the correct solution
for arbitrary column dependent matrices.
In the second part we give five applications to illustrate the applicability
of the GaBP algorithm to very large computer networks: Peer-to-Peer rating,
linear detection, distributed computation of support vector regression,
efficient computation of Kalman filter and distributed linear programming.
Using extensive simulations on up to 1,024 CPUs in parallel using IBM Bluegene
supercomputer we demonstrate the attractiveness and applicability of the GaBP
algorithm, using real network topologies with up to millions of nodes and
hundreds of millions of communication links. We further relate to several other
algorithms and explore their connection to the GaBP algorithm.Comment: Ph.D. Thesis, Submitted to the Senate of the Hebrew University of
Jerusalem, October 2008. 2nd Revision: July 200
Personalized Bundle Recommendation in Online Games
In business domains, \textit{bundling} is one of the most important marketing
strategies to conduct product promotions, which is commonly used in online
e-commerce and offline retailers. Existing recommender systems mostly focus on
recommending individual items that users may be interested in. In this paper,
we target at a practical but less explored recommendation problem named bundle
recommendation, which aims to offer a combination of items to users. To tackle
this specific recommendation problem in the context of the \emph{virtual mall}
in online games, we formalize it as a link prediction problem on a
user-item-bundle tripartite graph constructed from the historical interactions,
and solve it with a neural network model that can learn directly on the
graph-structure data. Extensive experiments on three public datasets and one
industrial game dataset demonstrate the effectiveness of the proposed method.
Further, the bundle recommendation model has been deployed in production for
more than one year in a popular online game developed by Netease Games, and the
launch of the model yields more than 60\% improvement on conversion rate of
bundles, and a relative improvement of more than 15\% on gross merchandise
volume (GMV).Comment: 8 pages, 10 figures, accepted paper on CIKM 202
- …