9,023 research outputs found
Robust Gaussian Filtering using a Pseudo Measurement
Many sensors, such as range, sonar, radar, GPS and visual devices, produce
measurements which are contaminated by outliers. This problem can be addressed
by using fat-tailed sensor models, which account for the possibility of
outliers. Unfortunately, all estimation algorithms belonging to the family of
Gaussian filters (such as the widely-used extended Kalman filter and unscented
Kalman filter) are inherently incompatible with such fat-tailed sensor models.
The contribution of this paper is to show that any Gaussian filter can be made
compatible with fat-tailed sensor models by applying one simple change: Instead
of filtering with the physical measurement, we propose to filter with a pseudo
measurement obtained by applying a feature function to the physical
measurement. We derive such a feature function which is optimal under some
conditions. Simulation results show that the proposed method can effectively
handle measurement outliers and allows for robust filtering in both linear and
nonlinear systems
Robust Inference for State-Space Models with Skewed Measurement Noise
Filtering and smoothing algorithms for linear discrete-time state-space
models with skewed and heavy-tailed measurement noise are presented. The
algorithms use a variational Bayes approximation of the posterior distribution
of models that have normal prior and skew-t-distributed measurement noise. The
proposed filter and smoother are compared with conventional low-complexity
alternatives in a simulated pseudorange positioning scenario. In the
simulations the proposed methods achieve better accuracy than the alternative
methods, the computational complexity of the filter being roughly 5 to 10 times
that of the Kalman filter.Comment: 5 pages, 7 figures. Accepted for publication in IEEE Signal
Processing Letter
A variational Bayesian method for inverse problems with impulsive noise
We propose a novel numerical method for solving inverse problems subject to
impulsive noises which possibly contain a large number of outliers. The
approach is of Bayesian type, and it exploits a heavy-tailed t distribution for
data noise to achieve robustness with respect to outliers. A hierarchical model
with all hyper-parameters automatically determined from the given data is
described. An algorithm of variational type by minimizing the Kullback-Leibler
divergence between the true posteriori distribution and a separable
approximation is developed. The numerical method is illustrated on several one-
and two-dimensional linear and nonlinear inverse problems arising from heat
conduction, including estimating boundary temperature, heat flux and heat
transfer coefficient. The results show its robustness to outliers and the fast
and steady convergence of the algorithm.Comment: 20 pages, to appear in J. Comput. Phy
t-Exponential Memory Networks for Question-Answering Machines
Recent advances in deep learning have brought to the fore models that can
make multiple computational steps in the service of completing a task; these
are capable of describ- ing long-term dependencies in sequential data. Novel
recurrent attention models over possibly large external memory modules
constitute the core mechanisms that enable these capabilities. Our work
addresses learning subtler and more complex underlying temporal dynamics in
language modeling tasks that deal with sparse sequential data. To this end, we
improve upon these recent advances, by adopting concepts from the field of
Bayesian statistics, namely variational inference. Our proposed approach
consists in treating the network parameters as latent variables with a prior
distribution imposed over them. Our statistical assumptions go beyond the
standard practice of postulating Gaussian priors. Indeed, to allow for handling
outliers, which are prevalent in long observed sequences of multivariate data,
multivariate t-exponential distributions are imposed. On this basis, we proceed
to infer corresponding posteriors; these can be used for inference and
prediction at test time, in a way that accounts for the uncertainty in the
available sparse training data. Specifically, to allow for our approach to best
exploit the merits of the t-exponential family, our method considers a new
t-divergence measure, which generalizes the concept of the Kullback-Leibler
divergence. We perform an extensive experimental evaluation of our approach,
using challenging language modeling benchmarks, and illustrate its superiority
over existing state-of-the-art techniques
The Extended Parameter Filter
The parameters of temporal models, such as dynamic Bayesian networks, may be
modelled in a Bayesian context as static or atemporal variables that influence
transition probabilities at every time step. Particle filters fail for models
that include such variables, while methods that use Gibbs sampling of parameter
variables may incur a per-sample cost that grows linearly with the length of
the observation sequence. Storvik devised a method for incremental computation
of exact sufficient statistics that, for some cases, reduces the per-sample
cost to a constant. In this paper, we demonstrate a connection between
Storvik's filter and a Kalman filter in parameter space and establish more
general conditions under which Storvik's filter works. Drawing on an analogy to
the extended Kalman filter, we develop and analyze, both theoretically and
experimentally, a Taylor approximation to the parameter posterior that allows
Storvik's method to be applied to a broader class of models. Our experiments on
both synthetic examples and real applications show improvement over existing
methods
Ensemble Kalman methods for high-dimensional hierarchical dynamic space-time models
We propose a new class of filtering and smoothing methods for inference in
high-dimensional, nonlinear, non-Gaussian, spatio-temporal state-space models.
The main idea is to combine the ensemble Kalman filter and smoother, developed
in the geophysics literature, with state-space algorithms from the statistics
literature. Our algorithms address a variety of estimation scenarios, including
on-line and off-line state and parameter estimation. We take a Bayesian
perspective, for which the goal is to generate samples from the joint posterior
distribution of states and parameters. The key benefit of our approach is the
use of ensemble Kalman methods for dimension reduction, which allows inference
for high-dimensional state vectors. We compare our methods to existing ones,
including ensemble Kalman filters, particle filters, and particle MCMC. Using a
real data example of cloud motion and data simulated under a number of
nonlinear and non-Gaussian scenarios, we show that our approaches outperform
these existing methods
Chance, long tails, and inference: a non-Gaussian, Bayesian theory of vocal learning in songbirds
Traditional theories of sensorimotor learning posit that animals use sensory
error signals to find the optimal motor command in the face of Gaussian sensory
and motor noise. However, most such theories cannot explain common behavioral
observations, for example that smaller sensory errors are more readily
corrected than larger errors and that large abrupt (but not gradually
introduced) errors lead to weak learning. Here we propose a new theory of
sensorimotor learning that explains these observations. The theory posits that
the animal learns an entire probability distribution of motor commands rather
than trying to arrive at a single optimal command, and that learning arises via
Bayesian inference when new sensory information becomes available. We test this
theory using data from a songbird, the Bengalese finch, that is adapting the
pitch (fundamental frequency) of its song following perturbations of auditory
feedback using miniature headphones. We observe the distribution of the sung
pitches to have long, non-Gaussian tails, which, within our theory, explains
the observed dynamics of learning. Further, the theory makes surprising
predictions about the dynamics of the shape of the pitch distribution, which we
confirm experimentally
- …