6,419 research outputs found
A Method for Compressing Parameters in Bayesian Models with Application to Logistic Sequence Prediction Models
Bayesian classification and regression with high order interactions is
largely infeasible because Markov chain Monte Carlo (MCMC) would need to be
applied with a great many parameters, whose number increases rapidly with the
order. In this paper we show how to make it feasible by effectively reducing
the number of parameters, exploiting the fact that many interactions have the
same values for all training cases. Our method uses a single ``compressed''
parameter to represent the sum of all parameters associated with a set of
patterns that have the same value for all training cases. Using symmetric
stable distributions as the priors of the original parameters, we can easily
find the priors of these compressed parameters. We therefore need to deal only
with a much smaller number of compressed parameters when training the model
with MCMC. The number of compressed parameters may have converged before
considering the highest possible order. After training the model, we can split
these compressed parameters into the original ones as needed to make
predictions for test cases. We show in detail how to compress parameters for
logistic sequence prediction models. Experiments on both simulated and real
data demonstrate that a huge number of parameters can indeed be reduced by our
compression method.Comment: 29 page
Active Classification for POMDPs: a Kalman-like State Estimator
The problem of state tracking with active observation control is considered
for a system modeled by a discrete-time, finite-state Markov chain observed
through conditionally Gaussian measurement vectors. The measurement model
statistics are shaped by the underlying state and an exogenous control input,
which influence the observations' quality. Exploiting an innovations approach,
an approximate minimum mean-squared error (MMSE) filter is derived to estimate
the Markov chain system state. To optimize the control strategy, the associated
mean-squared error is used as an optimization criterion in a partially
observable Markov decision process formulation. A stochastic dynamic
programming algorithm is proposed to solve for the optimal solution. To enhance
the quality of system state estimates, approximate MMSE smoothing estimators
are also derived. Finally, the performance of the proposed framework is
illustrated on the problem of physical activity detection in wireless body
sensing networks. The power of the proposed framework lies within its ability
to accommodate a broad spectrum of active classification applications including
sensor management for object classification and tracking, estimation of sparse
signals and radar scheduling.Comment: 38 pages, 6 figure
Information Theoretic Limits for Standard and One-Bit Compressed Sensing with Graph-Structured Sparsity
In this paper, we analyze the information theoretic lower bound on the
necessary number of samples needed for recovering a sparse signal under
different compressed sensing settings. We focus on the weighted graph model, a
model-based framework proposed by Hegde et al. (2015), for standard compressed
sensing as well as for one-bit compressed sensing. We study both the noisy and
noiseless regimes. Our analysis is general in the sense that it applies to any
algorithm used to recover the signal. We carefully construct restricted
ensembles for different settings and then apply Fano's inequality to establish
the lower bound on the necessary number of samples. Furthermore, we show that
our bound is tight for one-bit compressed sensing, while for standard
compressed sensing, our bound is tight up to a logarithmic factor of the number
of non-zero entries in the signal
Sparsity-Promoting Bayesian Dynamic Linear Models
Sparsity-promoting priors have become increasingly popular over recent years
due to an increased number of regression and classification applications
involving a large number of predictors. In time series applications where
observations are collected over time, it is often unrealistic to assume that
the underlying sparsity pattern is fixed. We propose here an original class of
flexible Bayesian linear models for dynamic sparsity modelling. The proposed
class of models expands upon the existing Bayesian literature on sparse
regression using generalized multivariate hyperbolic distributions. The
properties of the models are explored through both analytic results and
simulation studies. We demonstrate the model on a financial application where
it is shown that it accurately represents the patterns seen in the analysis of
stock and derivative data, and is able to detect major events by filtering an
artificial portfolio of assets
Recovery from Linear Measurements with Complexity-Matching Universal Signal Estimation
We study the compressed sensing (CS) signal estimation problem where an input
signal is measured via a linear matrix multiplication under additive noise.
While this setup usually assumes sparsity or compressibility in the input
signal during recovery, the signal structure that can be leveraged is often not
known a priori. In this paper, we consider universal CS recovery, where the
statistics of a stationary ergodic signal source are estimated simultaneously
with the signal itself. Inspired by Kolmogorov complexity and minimum
description length, we focus on a maximum a posteriori (MAP) estimation
framework that leverages universal priors to match the complexity of the
source. Our framework can also be applied to general linear inverse problems
where more measurements than in CS might be needed. We provide theoretical
results that support the algorithmic feasibility of universal MAP estimation
using a Markov chain Monte Carlo implementation, which is computationally
challenging. We incorporate some techniques to accelerate the algorithm while
providing comparable and in many cases better reconstruction quality than
existing algorithms. Experimental results show the promise of universality in
CS, particularly for low-complexity sources that do not exhibit standard
sparsity or compressibility.Comment: 29 pages, 8 figure
- …