26,924 research outputs found
A Geometric Variational Approach to Bayesian Inference
We propose a novel Riemannian geometric framework for variational inference
in Bayesian models based on the nonparametric Fisher-Rao metric on the manifold
of probability density functions. Under the square-root density representation,
the manifold can be identified with the positive orthant of the unit
hypersphere in L2, and the Fisher-Rao metric reduces to the standard L2 metric.
Exploiting such a Riemannian structure, we formulate the task of approximating
the posterior distribution as a variational problem on the hypersphere based on
the alpha-divergence. This provides a tighter lower bound on the marginal
distribution when compared to, and a corresponding upper bound unavailable
with, approaches based on the Kullback-Leibler divergence. We propose a novel
gradient-based algorithm for the variational problem based on Frechet
derivative operators motivated by the geometry of the Hilbert sphere, and
examine its properties. Through simulations and real-data applications, we
demonstrate the utility of the proposed geometric framework and algorithm on
several Bayesian models
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
While nonlinear stochastic partial differential equations arise naturally in
spatiotemporal modeling, inference for such systems often faces two major
challenges: sparse noisy data and ill-posedness of the inverse problem of
parameter estimation. To overcome the challenges, we introduce a strongly
regularized posterior by normalizing the likelihood and by imposing physical
constraints through priors of the parameters and states. We investigate joint
parameter-state estimation by the regularized posterior in a physically
motivated nonlinear stochastic energy balance model (SEBM) for paleoclimate
reconstruction. The high-dimensional posterior is sampled by a particle Gibbs
sampler that combines MCMC with an optimal particle filter exploiting the
structure of the SEBM. In tests using either Gaussian or uniform priors based
on the physical range of parameters, the regularized posteriors overcome the
ill-posedness and lead to samples within physical ranges, quantifying the
uncertainty in estimation. Due to the ill-posedness and the regularization, the
posterior of parameters presents a relatively large uncertainty, and
consequently, the maximum of the posterior, which is the minimizer in a
variational approach, can have a large variation. In contrast, the posterior of
states generally concentrates near the truth, substantially filtering out
observation noise and reducing uncertainty in the unconstrained SEBM
A finite state projection algorithm for the stationary solution of the chemical master equation
The chemical master equation (CME) is frequently used in systems biology to
quantify the effects of stochastic fluctuations that arise due to biomolecular
species with low copy numbers. The CME is a system of ordinary differential
equations that describes the evolution of probability density for each
population vector in the state-space of the stochastic reaction dynamics. For
many examples of interest, this state-space is infinite, making it difficult to
obtain exact solutions of the CME. To deal with this problem, the Finite State
Projection (FSP) algorithm was developed by Munsky and Khammash (Jour. Chem.
Phys. 2006), to provide approximate solutions to the CME by truncating the
state-space. The FSP works well for finite time-periods but it cannot be used
for estimating the stationary solutions of CMEs, which are often of interest in
systems biology. The aim of this paper is to develop a version of FSP which we
refer to as the stationary FSP (sFSP) that allows one to obtain accurate
approximations of the stationary solutions of a CME by solving a finite
linear-algebraic system that yields the stationary distribution of a
continuous-time Markov chain over the truncated state-space. We derive bounds
for the approximation error incurred by sFSP and we establish that under
certain stability conditions, these errors can be made arbitrarily small by
appropriately expanding the truncated state-space. We provide several examples
to illustrate our sFSP method and demonstrate its efficiency in estimating the
stationary distributions. In particular, we show that using a quantised tensor
train (QTT) implementation of our sFSP method, problems admitting more than 100
million states can be efficiently solved.Comment: 8 figure
A Bayesian information criterion for singular models
We consider approximate Bayesian model choice for model selection problems
that involve models whose Fisher-information matrices may fail to be invertible
along other competing submodels. Such singular models do not obey the
regularity conditions underlying the derivation of Schwarz's Bayesian
information criterion (BIC) and the penalty structure in BIC generally does not
reflect the frequentist large-sample behavior of their marginal likelihood.
While large-sample theory for the marginal likelihood of singular models has
been developed recently, the resulting approximations depend on the true
parameter value and lead to a paradox of circular reasoning. Guided by examples
such as determining the number of components of mixture models, the number of
factors in latent factor models or the rank in reduced-rank regression, we
propose a resolution to this paradox and give a practical extension of BIC for
singular model selection problems
- …