614 research outputs found
Multi-Modal Mean-Fields via Cardinality-Based Clamping
Mean Field inference is central to statistical physics. It has attracted much
interest in the Computer Vision community to efficiently solve problems
expressible in terms of large Conditional Random Fields. However, since it
models the posterior probability distribution as a product of marginal
probabilities, it may fail to properly account for important dependencies
between variables. We therefore replace the fully factorized distribution of
Mean Field by a weighted mixture of such distributions, that similarly
minimizes the KL-Divergence to the true posterior. By introducing two new
ideas, namely, conditioning on groups of variables instead of single ones and
using a parameter of the conditional random field potentials, that we identify
to the temperature in the sense of statistical physics to select such groups,
we can perform this minimization efficiently. Our extension of the clamping
method proposed in previous works allows us to both produce a more descriptive
approximation of the true posterior and, inspired by the diverse MAP paradigms,
fit a mixture of Mean Field approximations. We demonstrate that this positively
impacts real-world algorithms that initially relied on mean fields.Comment: Submitted for review to CVPR 201
Relabelling Algorithms for Large Dataset Mixture Models
Mixture models are flexible tools in density estimation and classification
problems. Bayesian estimation of such models typically relies on sampling from
the posterior distribution using Markov chain Monte Carlo. Label switching
arises because the posterior is invariant to permutations of the component
parameters. Methods for dealing with label switching have been studied fairly
extensively in the literature, with the most popular approaches being those
based on loss functions. However, many of these algorithms turn out to be too
slow in practice, and can be infeasible as the size and dimension of the data
grow. In this article, we review earlier solutions which can scale up well for
large data sets, and compare their performances on simulated and real datasets.
In addition, we propose a new, and computationally efficient algorithm based on
a loss function interpretation, and show that it can scale up well in larger
problems. We conclude with some discussions and recommendations of all the
methods studied
Bayesian Symbol Detection in Wireless Relay Networks via Likelihood-Free Inference
This paper presents a general stochastic model developed for a class of
cooperative wireless relay networks, in which imperfect knowledge of the
channel state information at the destination node is assumed. The framework
incorporates multiple relay nodes operating under general known non-linear
processing functions. When a non-linear relay function is considered, the
likelihood function is generally intractable resulting in the maximum
likelihood and the maximum a posteriori detectors not admitting closed form
solutions. We illustrate our methodology to overcome this intractability under
the example of a popular optimal non-linear relay function choice and
demonstrate how our algorithms are capable of solving the previously
intractable detection problem. Overcoming this intractability involves
development of specialised Bayesian models. We develop three novel algorithms
to perform detection for this Bayesian model, these include a Markov chain
Monte Carlo Approximate Bayesian Computation (MCMC-ABC) approach; an Auxiliary
Variable MCMC (MCMC-AV) approach; and a Suboptimal Exhaustive Search Zero
Forcing (SES-ZF) approach. Finally, numerical examples comparing the symbol
error rate (SER) performance versus signal to noise ratio (SNR) of the three
detection algorithms are studied in simulated examples
Sophisticated Inference
Active inference offers a first principle account of sentient behaviour, from
which special and important cases can be derived, e.g., reinforcement learning,
active learning, Bayes optimal inference, Bayes optimal design, etc. Active
inference resolves the exploitation-exploration dilemma in relation to prior
preferences, by placing information gain on the same footing as reward or
value. In brief, active inference replaces value functions with functionals of
(Bayesian) beliefs, in the form of an expected (variational) free energy. In
this paper, we consider a sophisticated kind of active inference, using a
recursive form of expected free energy. Sophistication describes the degree to
which an agent has beliefs about beliefs. We consider agents with beliefs about
the counterfactual consequences of action for states of affairs and beliefs
about those latent states. In other words, we move from simply considering
beliefs about 'what would happen if I did that' to 'what would I believe about
what would happen if I did that'. The recursive form of the free energy
functional effectively implements a deep tree search over actions and outcomes
in the future. Crucially, this search is over sequences of belief states, as
opposed to states per se. We illustrate the competence of this scheme, using
numerical simulations of deep decision problems
Variational approximation for mixtures of linear mixed models
Mixtures of linear mixed models (MLMMs) are useful for clustering grouped
data and can be estimated by likelihood maximization through the EM algorithm.
The conventional approach to determining a suitable number of components is to
compare different mixture models using penalized log-likelihood criteria such
as BIC.We propose fitting MLMMs with variational methods which can perform
parameter estimation and model selection simultaneously. A variational
approximation is described where the variational lower bound and parameter
updates are in closed form, allowing fast evaluation. A new variational greedy
algorithm is developed for model selection and learning of the mixture
components. This approach allows an automatic initialization of the algorithm
and returns a plausible number of mixture components automatically. In cases of
weak identifiability of certain model parameters, we use hierarchical centering
to reparametrize the model and show empirically that there is a gain in
efficiency by variational algorithms similar to that in MCMC algorithms.
Related to this, we prove that the approximate rate of convergence of
variational algorithms by Gaussian approximation is equal to that of the
corresponding Gibbs sampler which suggests that reparametrizations can lead to
improved convergence in variational algorithms as well.Comment: 36 pages, 5 figures, 2 tables, submitted to JCG
- …