16,754 research outputs found
Linear and Parallel Learning of Markov Random Fields
We introduce a new embarrassingly parallel parameter learning algorithm for
Markov random fields with untied parameters which is efficient for a large
class of practical models. Our algorithm parallelizes naturally over cliques
and, for graphs of bounded degree, its complexity is linear in the number of
cliques. Unlike its competitors, our algorithm is fully parallel and for
log-linear models it is also data efficient, requiring only the local
sufficient statistics of the data to estimate parameters
Construction of Bayesian Deformable Models via Stochastic Approximation Algorithm: A Convergence Study
The problem of the definition and the estimation of generative models based
on deformable templates from raw data is of particular importance for modelling
non aligned data affected by various types of geometrical variability. This is
especially true in shape modelling in the computer vision community or in
probabilistic atlas building for Computational Anatomy (CA). A first coherent
statistical framework modelling the geometrical variability as hidden variables
has been given by Allassonni\`ere, Amit and Trouv\'e (JRSS 2006). Setting the
problem in a Bayesian context they proved the consistency of the MAP estimator
and provided a simple iterative deterministic algorithm with an EM flavour
leading to some reasonable approximations of the MAP estimator under low noise
conditions. In this paper we present a stochastic algorithm for approximating
the MAP estimator in the spirit of the SAEM algorithm. We prove its convergence
to a critical point of the observed likelihood with an illustration on images
of handwritten digits
Spatial models generated by nested stochastic partial differential equations, with an application to global ozone mapping
A new class of stochastic field models is constructed using nested stochastic
partial differential equations (SPDEs). The model class is computationally
efficient, applicable to data on general smooth manifolds, and includes both
the Gaussian Mat\'{e}rn fields and a wide family of fields with oscillating
covariance functions. Nonstationary covariance models are obtained by spatially
varying the parameters in the SPDEs, and the model parameters are estimated
using direct numerical optimization, which is more efficient than standard
Markov Chain Monte Carlo procedures. The model class is used to estimate daily
ozone maps using a large data set of spatially irregular global total column
ozone data.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS383 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Inference via low-dimensional couplings
We investigate the low-dimensional structure of deterministic transformations
between random variables, i.e., transport maps between probability measures. In
the context of statistics and machine learning, these transformations can be
used to couple a tractable "reference" measure (e.g., a standard Gaussian) with
a target measure of interest. Direct simulation from the desired measure can
then be achieved by pushing forward reference samples through the map. Yet
characterizing such a map---e.g., representing and evaluating it---grows
challenging in high dimensions. The central contribution of this paper is to
establish a link between the Markov properties of the target measure and the
existence of low-dimensional couplings, induced by transport maps that are
sparse and/or decomposable. Our analysis not only facilitates the construction
of transformations in high-dimensional settings, but also suggests new
inference methodologies for continuous non-Gaussian graphical models. For
instance, in the context of nonlinear state-space models, we describe new
variational algorithms for filtering, smoothing, and sequential parameter
inference. These algorithms can be understood as the natural
generalization---to the non-Gaussian case---of the square-root
Rauch-Tung-Striebel Gaussian smoother.Comment: 78 pages, 25 figure
INLA or MCMC? A Tutorial and Comparative Evaluation for Spatial Prediction in log-Gaussian Cox Processes
We investigate two options for performing Bayesian inference on spatial
log-Gaussian Cox processes assuming a spatially continuous latent field: Markov
chain Monte Carlo (MCMC) and the integrated nested Laplace approximation
(INLA). We first describe the device of approximating a spatially continuous
Gaussian field by a Gaussian Markov random field on a discrete lattice, and
present a simulation study showing that, with careful choice of parameter
values, small neighbourhood sizes can give excellent approximations. We then
introduce the spatial log-Gaussian Cox process and describe MCMC and INLA
methods for spatial prediction within this model class. We report the results
of a simulation study in which we compare MALA and the technique of
approximating the continuous latent field by a discrete one, followed by
approximate Bayesian inference via INLA over a selection of 18 simulated
scenarios. The results question the notion that the latter technique is both
significantly faster and more robust than MCMC in this setting; 100,000
iterations of the MALA algorithm running in 20 minutes on a desktop PC
delivered greater predictive accuracy than the default \verb=INLA= strategy,
which ran in 4 minutes and gave comparative performance to the full Laplace
approximation which ran in 39 minutes.Comment: This replaces the previous version of the report. The new version
includes results from an additional simulation study, and corrects an error
in the implementation of the INLA-based method
Conditional Random Fields as Recurrent Neural Networks
Pixel-level labelling tasks, such as semantic segmentation, play a central
role in image understanding. Recent approaches have attempted to harness the
capabilities of deep learning techniques for image recognition to tackle
pixel-level labelling tasks. One central issue in this methodology is the
limited capacity of deep learning techniques to delineate visual objects. To
solve this problem, we introduce a new form of convolutional neural network
that combines the strengths of Convolutional Neural Networks (CNNs) and
Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To
this end, we formulate mean-field approximate inference for the Conditional
Random Fields with Gaussian pairwise potentials as Recurrent Neural Networks.
This network, called CRF-RNN, is then plugged in as a part of a CNN to obtain a
deep network that has desirable properties of both CNNs and CRFs. Importantly,
our system fully integrates CRF modelling with CNNs, making it possible to
train the whole deep network end-to-end with the usual back-propagation
algorithm, avoiding offline post-processing methods for object delineation. We
apply the proposed method to the problem of semantic image segmentation,
obtaining top results on the challenging Pascal VOC 2012 segmentation
benchmark.Comment: This paper is published in IEEE ICCV 201
- …