24,999 research outputs found
Bayesian Estimation for Continuous-Time Sparse Stochastic Processes
We consider continuous-time sparse stochastic processes from which we have
only a finite number of noisy/noiseless samples. Our goal is to estimate the
noiseless samples (denoising) and the signal in-between (interpolation
problem).
By relying on tools from the theory of splines, we derive the joint a priori
distribution of the samples and show how this probability density function can
be factorized. The factorization enables us to tractably implement the maximum
a posteriori and minimum mean-square error (MMSE) criteria as two statistical
approaches for estimating the unknowns. We compare the derived statistical
methods with well-known techniques for the recovery of sparse signals, such as
the norm and Log (- relaxation) regularization
methods. The simulation results show that, under certain conditions, the
performance of the regularization techniques can be very close to that of the
MMSE estimator.Comment: To appear in IEEE TS
Locally adaptive factor processes for multivariate time series
In modeling multivariate time series, it is important to allow time-varying
smoothness in the mean and covariance process. In particular, there may be
certain time intervals exhibiting rapid changes and others in which changes are
slow. If such time-varying smoothness is not accounted for, one can obtain
misleading inferences and predictions, with over-smoothing across erratic time
intervals and under-smoothing across times exhibiting slow variation. This can
lead to mis-calibration of predictive intervals, which can be substantially too
narrow or wide depending on the time. We propose a locally adaptive factor
process for characterizing multivariate mean-covariance changes in continuous
time, allowing locally varying smoothness in both the mean and covariance
matrix. This process is constructed utilizing latent dictionary functions
evolving in time through nested Gaussian processes and linearly related to the
observed data with a sparse mapping. Using a differential equation
representation, we bypass usual computational bottlenecks in obtaining MCMC and
online algorithms for approximate Bayesian inference. The performance is
assessed in simulations and illustrated in a financial application
Stochastic expansions using continuous dictionaries: L\'{e}vy adaptive regression kernels
This article describes a new class of prior distributions for nonparametric
function estimation. The unknown function is modeled as a limit of weighted
sums of kernels or generator functions indexed by continuous parameters that
control local and global features such as their translation, dilation,
modulation and shape. L\'{e}vy random fields and their stochastic integrals are
employed to induce prior distributions for the unknown functions or,
equivalently, for the number of kernels and for the parameters governing their
features. Scaling, shape, and other features of the generating functions are
location-specific to allow quite different function properties in different
parts of the space, as with wavelet bases and other methods employing
overcomplete dictionaries. We provide conditions under which the stochastic
expansions converge in specified Besov or Sobolev norms. Under a Gaussian error
model, this may be viewed as a sparse regression problem, with regularization
induced via the L\'{e}vy random field prior distribution. Posterior inference
for the unknown functions is based on a reversible jump Markov chain Monte
Carlo algorithm. We compare the L\'{e}vy Adaptive Regression Kernel (LARK)
method to wavelet-based methods using some of the standard test functions, and
illustrate its flexibility and adaptability in nonstationary applications.Comment: Published in at http://dx.doi.org/10.1214/11-AOS889 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Batch Nonlinear Continuous-Time Trajectory Estimation as Exactly Sparse Gaussian Process Regression
In this paper, we revisit batch state estimation through the lens of Gaussian
process (GP) regression. We consider continuous-discrete estimation problems
wherein a trajectory is viewed as a one-dimensional GP, with time as the
independent variable. Our continuous-time prior can be defined by any
nonlinear, time-varying stochastic differential equation driven by white noise;
this allows the possibility of smoothing our trajectory estimates using a
variety of vehicle dynamics models (e.g., `constant-velocity'). We show that
this class of prior results in an inverse kernel matrix (i.e., covariance
matrix between all pairs of measurement times) that is exactly sparse
(block-tridiagonal) and that this can be exploited to carry out GP regression
(and interpolation) very efficiently. When the prior is based on a linear,
time-varying stochastic differential equation and the measurement model is also
linear, this GP approach is equivalent to classical, discrete-time smoothing
(at the measurement times); when a nonlinearity is present, we iterate over the
whole trajectory to maximize accuracy. We test the approach experimentally on a
simultaneous trajectory estimation and mapping problem using a mobile robot
dataset.Comment: Submitted to Autonomous Robots on 20 November 2014, manuscript #
AURO-D-14-00185, 16 pages, 7 figure
Efficient Transition Probability Computation for Continuous-Time Branching Processes via Compressed Sensing
Branching processes are a class of continuous-time Markov chains (CTMCs) with
ubiquitous applications. A general difficulty in statistical inference under
partially observed CTMC models arises in computing transition probabilities
when the discrete state space is large or uncountable. Classical methods such
as matrix exponentiation are infeasible for large or countably infinite state
spaces, and sampling-based alternatives are computationally intensive,
requiring a large integration step to impute over all possible hidden events.
Recent work has successfully applied generating function techniques to
computing transition probabilities for linear multitype branching processes.
While these techniques often require significantly fewer computations than
matrix exponentiation, they also become prohibitive in applications with large
populations. We propose a compressed sensing framework that significantly
accelerates the generating function method, decreasing computational cost up to
a logarithmic factor by only assuming the probability mass of transitions is
sparse. We demonstrate accurate and efficient transition probability
computations in branching process models for hematopoiesis and transposable
element evolution.Comment: 18 pages, 4 figures, 2 table
Bayesian Inference of Log Determinants
The log-determinant of a kernel matrix appears in a variety of machine
learning problems, ranging from determinantal point processes and generalized
Markov random fields, through to the training of Gaussian processes. Exact
calculation of this term is often intractable when the size of the kernel
matrix exceeds a few thousand. In the spirit of probabilistic numerics, we
reinterpret the problem of computing the log-determinant as a Bayesian
inference problem. In particular, we combine prior knowledge in the form of
bounds from matrix theory and evidence derived from stochastic trace estimation
to obtain probabilistic estimates for the log-determinant and its associated
uncertainty within a given computational budget. Beyond its novelty and
theoretic appeal, the performance of our proposal is competitive with
state-of-the-art approaches to approximating the log-determinant, while also
quantifying the uncertainty due to budget-constrained evidence.Comment: 12 pages, 3 figure
- …