1,123 research outputs found
Modeling heterogeneity in random graphs through latent space models: a selective review
We present a selective review on probabilistic modeling of heterogeneity in
random graphs. We focus on latent space models and more particularly on
stochastic block models and their extensions that have undergone major
developments in the last five years
Multidimensional Membership Mixture Models
We present the multidimensional membership mixture (M3) models where every
dimension of the membership represents an independent mixture model and each
data point is generated from the selected mixture components jointly. This is
helpful when the data has a certain shared structure. For example, three unique
means and three unique variances can effectively form a Gaussian mixture model
with nine components, while requiring only six parameters to fully describe it.
In this paper, we present three instantiations of M3 models (together with the
learning and inference algorithms): infinite, finite, and hybrid, depending on
whether the number of mixtures is fixed or not. They are built upon Dirichlet
process mixture models, latent Dirichlet allocation, and a combination
respectively. We then consider two applications: topic modeling and learning 3D
object arrangements. Our experiments show that our M3 models achieve better
performance using fewer topics than many classic topic models. We also observe
that topics from the different dimensions of M3 models are meaningful and
orthogonal to each other.Comment: 9 pages, 7 figure
Hierarchically Clustered Representation Learning
The joint optimization of representation learning and clustering in the
embedding space has experienced a breakthrough in recent years. In spite of the
advance, clustering with representation learning has been limited to flat-level
categories, which often involves cohesive clustering with a focus on instance
relations. To overcome the limitations of flat clustering, we introduce
hierarchically-clustered representation learning (HCRL), which simultaneously
optimizes representation learning and hierarchical clustering in the embedding
space. Compared with a few prior works, HCRL firstly attempts to consider a
generation of deep embeddings from every component of the hierarchy, not just
leaf components. In addition to obtaining hierarchically clustered embeddings,
we can reconstruct data by the various abstraction levels, infer the intrinsic
hierarchical structure, and learn the level-proportion features. We conducted
evaluations with image and text domains, and our quantitative analyses showed
competent likelihoods and the best accuracies compared with the baselines.Comment: 10 pages, 7 figures, Under review as a conference pape
Variational Learning for Finite Inverted Dirichlet Mixture Models and Its Applications
Clustering is an important step in data mining, machine learning, computer vision and image processing. It is the process of assigning similar objects to the same subset. Among available clustering techniques, finite mixture models have been remarkably used, since they have the ability to consider prior knowledge about the data. Employing mixture models requires, choosing a standard distribution, determining the number of mixture components and estimating the model parameters. Currently, the combination of Gaussian distribution, as the standard distribution, and Expectation Maximization (EM), as the parameter estimator, has been widely used with mixture models. However, each of these choices has its own limitations. In this thesis, these limitations are discussed and addressed via defining a variational inference framework for finite inverted Dirichlet mixture model, which is able to provide a better capability in modeling multivariate positive data, that appear frequently in many real world applications. Finite inverted Dirichlet mixtures enable us to model high-dimensional, both symmetric and asymmetric data. Compared to the conventional expectation maximization (EM) algorithm, the variational approach has the following advantages: it is computationally more efficient, it converges fast, and is able to estimate the parameters and the number of the mixture model components, automatically and simultaneously. The experimental results validate the presented approach on different synthetic datasets and shows its performance for two interesting and challenging real world applications, namely natural scene categorization and human activity classification
Probabilistic task modelling for meta-learning
We propose probabilistic task modelling -- a generative probabilistic model
for collections of tasks used in meta-learning. The proposed model combines
variational auto-encoding and latent Dirichlet allocation to model each task as
a mixture of Gaussian distribution in an embedding space. Such modelling
provides an explicit representation of a task through its task-theme mixture.
We present an efficient approximation inference technique based on variational
inference method for empirical Bayes parameter estimation. We perform empirical
evaluations to validate the task uncertainty and task distance produced by the
proposed method through correlation diagrams of the prediction accuracy on
testing tasks. We also carry out experiments of task selection in meta-learning
to demonstrate how the task relatedness inferred from the proposed model help
to facilitate meta-learning algorithms.Comment: Accepted at UAI 202
Mixed membership stochastic blockmodels
Observations consisting of measurements on relationships for pairs of objects
arise in many settings, such as protein interaction and gene regulatory
networks, collections of author-recipient email, and social networks. Analyzing
such data with probabilisic models can be delicate because the simple
exchangeability assumptions underlying many boilerplate models no longer hold.
In this paper, we describe a latent variable model of such data called the
mixed membership stochastic blockmodel. This model extends blockmodels for
relational data to ones which capture mixed membership latent relational
structure, thus providing an object-specific low-dimensional representation. We
develop a general variational inference algorithm for fast approximate
posterior inference. We explore applications to social and protein interaction
networks.Comment: 46 pages, 14 figures, 3 table
Bayesian Semi-supervised Learning with Graph Gaussian Processes
We propose a data-efficient Gaussian process-based Bayesian approach to the
semi-supervised learning problem on graphs. The proposed model shows extremely
competitive performance when compared to the state-of-the-art graph neural
networks on semi-supervised learning benchmark experiments, and outperforms the
neural networks in active learning experiments where labels are scarce.
Furthermore, the model does not require a validation data set for early
stopping to control over-fitting. Our model can be viewed as an instance of
empirical distribution regression weighted locally by network connectivity. We
further motivate the intuitive construction of the model with a Bayesian linear
model interpretation where the node features are filtered by an operator
related to the graph Laplacian. The method can be easily implemented by
adapting off-the-shelf scalable variational inference algorithms for Gaussian
processes.Comment: To appear in NIPS 2018 Fixed an error in Figure 2. The previous arxiv
version contains two identical sub-figure
- …