83 research outputs found
Summary Statistics for Partitionings and Feature Allocations
Infinite mixture models are commonly used for clustering. One can sample from
the posterior of mixture assignments by Monte Carlo methods or find its maximum
a posteriori solution by optimization. However, in some problems the posterior
is diffuse and it is hard to interpret the sampled partitionings. In this
paper, we introduce novel statistics based on block sizes for representing
sample sets of partitionings and feature allocations. We develop an
element-based definition of entropy to quantify segmentation among their
elements. Then we propose a simple algorithm called entropy agglomeration (EA)
to summarize and visualize this information. Experiments on various infinite
mixture posteriors as well as a feature allocation dataset demonstrate that the
proposed statistics are useful in practice.Comment: Accepted to NIPS 2013:
https://nips.cc/Conferences/2013/Program/event.php?ID=376
Bayesian Inference for Nonnegative Matrix Factorisation Models
We describe nonnegative matrix factorisation (NMF) with a Kullback-Leibler (KL) error measure in a statistical
framework, with a hierarchical generative model consisting of an observation and a prior component. Omitting the prior
leads to the standard KL-NMF algorithms as special cases, where maximum likelihood parameter estimation is carried
out via the Expectation-Maximisation (EM) algorithm. Starting from this view, we develop full Bayesian inference
via variational Bayes or Monte Carlo. Our construction retains conjugacy and enables us to develop more powerful
models while retaining attractive features of standard NMF such as monotonic convergence and easy implementation.
We illustrate our approach on model order selection and image reconstruction
Bayesian Model Comparison With the g-Prior
Model comparison and selection is an important problem in many model-based signal processing applications. Often, very simple information criteria such as the Akaike information criterion or the Bayesian information criterion are used despite their shortcomings. Compared to these methods, Djuric’s asymptotic MAP rule was an improvement, and in this paper we extend the work by Djuric in several ways. Specifically, we consider the elicitation of proper prior distributions, treat the case of real- and complex-valued data simultaneously in a Bayesian framework similar to that considered by Djuric, and develop new model selection rules for a regression model containing both linear and non-linear parameters. Moreover, we use this framework to give a new interpretation of the popular information criteria and relate their performance to the signal-to-noise ratio of the data. By use of simulations, we also demonstrate that our proposed model comparison and selection rules outperform the traditional information criteria both in terms of detecting the true model and in terms of predicting unobserved data. The simulation code is available online
- …