18,088 research outputs found
Ensemble Kalman methods for high-dimensional hierarchical dynamic space-time models
We propose a new class of filtering and smoothing methods for inference in
high-dimensional, nonlinear, non-Gaussian, spatio-temporal state-space models.
The main idea is to combine the ensemble Kalman filter and smoother, developed
in the geophysics literature, with state-space algorithms from the statistics
literature. Our algorithms address a variety of estimation scenarios, including
on-line and off-line state and parameter estimation. We take a Bayesian
perspective, for which the goal is to generate samples from the joint posterior
distribution of states and parameters. The key benefit of our approach is the
use of ensemble Kalman methods for dimension reduction, which allows inference
for high-dimensional state vectors. We compare our methods to existing ones,
including ensemble Kalman filters, particle filters, and particle MCMC. Using a
real data example of cloud motion and data simulated under a number of
nonlinear and non-Gaussian scenarios, we show that our approaches outperform
these existing methods
Thermodynamic Limit in Statistical Physics
The thermodynamic limit in statistical thermodynamics of many-particle
systems is an important but often overlooked issue in the various applied
studies of condensed matter physics. To settle this issue, we review tersely
the past and present disposition of thermodynamic limiting procedure in the
structure of the contemporary statistical mechanics and our current
understanding of this problem. We pick out the ingenious approach by N. N.
Bogoliubov, who developed a general formalism for establishing of the limiting
distribution functions in the form of formal series in powers of the density.
In that study he outlined the method of justification of the thermodynamic
limit when he derived the generalized Boltzmann equations. To enrich and to
weave our discussion, we take this opportunity to give a brief survey of the
closely related problems, such as the equipartition of energy and the
equivalence and nonequivalence of statistical ensembles. The validity of the
equipartition of energy permits one to decide what are the boundaries of
applicability of statistical mechanics. The major aim of this work is to
provide a better qualitative understanding of the physical significance of the
thermodynamic limit in modern statistical physics of the infinite and "small"
many-particle systems.Comment: 28 pages, Refs.180. arXiv admin note: text overlap with
arXiv:1011.2981, arXiv:0812.0943 by other author
S-AMP: Approximate Message Passing for General Matrix Ensembles
In this work we propose a novel iterative estimation algorithm for linear
observation systems called S-AMP whose fixed points are the stationary points
of the exact Gibbs free energy under a set of (first- and second-) moment
consistency constraints in the large system limit. S-AMP extends the
approximate message-passing (AMP) algorithm to general matrix ensembles. The
generalization is based on the S-transform (in free probability) of the
spectrum of the measurement matrix. Furthermore, we show that the optimality of
S-AMP follows directly from its design rather than from solving a separate
optimization problem as done for AMP.Comment: 5 pages, 1 figur
MCMC with Strings and Branes: The Suburban Algorithm (Extended Version)
Motivated by the physics of strings and branes, we develop a class of Markov
chain Monte Carlo (MCMC) algorithms involving extended objects. Starting from a
collection of parallel Metropolis-Hastings (MH) samplers, we place them on an
auxiliary grid, and couple them together via nearest neighbor interactions.
This leads to a class of "suburban samplers" (i.e., spread out Metropolis).
Coupling the samplers in this way modifies the mixing rate and speed of
convergence for the Markov chain, and can in many cases allow a sampler to more
easily overcome free energy barriers in a target distribution. We test these
general theoretical considerations by performing several numerical experiments.
For suburban samplers with a fluctuating grid topology, performance is strongly
correlated with the average number of neighbors. Increasing the average number
of neighbors above zero initially leads to an increase in performance, though
there is a critical connectivity with effective dimension d_eff ~ 1, above
which "groupthink" takes over, and the performance of the sampler declines.Comment: v2: 55 pages, 13 figures, references and clarifications added.
Published version. This article is an extended version of "MCMC with Strings
and Branes: The Suburban Algorithm
Molecular modeling for physical property prediction
Multiscale modeling is becoming the standard approach for process study in a broader framework that promotes computer aided integrated product and process design. In addition to usual purity requirements, end products must meet new constraints in terms of environmental impact, safety of goods and people, specific properties. This chapter adresses the use of molecular modeling tools for the prediction of physical property usefull for chemical engineering practice
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
A Nonparametric Bayesian Approach to Uncovering Rat Hippocampal Population Codes During Spatial Navigation
Rodent hippocampal population codes represent important spatial information
about the environment during navigation. Several computational methods have
been developed to uncover the neural representation of spatial topology
embedded in rodent hippocampal ensemble spike activity. Here we extend our
previous work and propose a nonparametric Bayesian approach to infer rat
hippocampal population codes during spatial navigation. To tackle the model
selection problem, we leverage a nonparametric Bayesian model. Specifically, to
analyze rat hippocampal ensemble spiking activity, we apply a hierarchical
Dirichlet process-hidden Markov model (HDP-HMM) using two Bayesian inference
methods, one based on Markov chain Monte Carlo (MCMC) and the other based on
variational Bayes (VB). We demonstrate the effectiveness of our Bayesian
approaches on recordings from a freely-behaving rat navigating in an open field
environment. We find that MCMC-based inference with Hamiltonian Monte Carlo
(HMC) hyperparameter sampling is flexible and efficient, and outperforms VB and
MCMC approaches with hyperparameters set by empirical Bayes
- âŠ