140 research outputs found

    Advanced sequential Monte Carlo methods and their applications to sparse sensor network for detection and estimation

    Get PDF
    The general state space models present a flexible framework for modeling dynamic systems and therefore have vast applications in many disciplines such as engineering, economics, biology, etc. However, optimal estimation problems of non-linear non-Gaussian state space models are analytically intractable in general. Sequential Monte Carlo (SMC) methods become a very popular class of simulation-based methods for the solution of optimal estimation problems. The advantages of SMC methods in comparison with classical filtering methods such as Kalman Filter and Extended Kalman Filter are that they are able to handle non-linear non-Gaussian scenarios without relying on any local linearization techniques. In this thesis, we present an advanced SMC method and the study of its asymptotic behavior. We apply the proposed SMC method in a target tracking problem using different observation models. Specifically, a distributed SMC algorithm is developed for a wireless sensor network (WSN) that incorporates with an informative-sensor detection technique. The novel SMC algorithm is designed to surmount the degeneracy problem by employing a multilevel Markov chain Monte Carlo (MCMC) procedure constructed by engaging drift homotopy and likelihood bridging techniques. The observations are gathered only from the informative sensors, which are sensing useful observations of the nearby moving targets. The detection of those informative sensors, which are typically a small portion of the WSN, is taking place by using a sparsity-aware matrix decomposition technique. Simulation results showcase that our algorithm outperforms current popular tracking algorithms such as bootstrap filter and auxiliary particle filter in many scenarios

    Approximate Bayesian techniques for inference in stochastic dynamical systems

    Get PDF
    This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

    Approximate Bayesian inference methods for stochastic state space models

    Get PDF
    This thesis collects together research results obtained during my doctoral studies related to approximate Bayesian inference in stochastic state-space models. The published research spans a variety of topics including 1) application of Gaussian filtering in satellite orbit prediction, 2) outlier robust linear regression using variational Bayes (VB) approximation, 3) filtering and smoothing in continuous-discrete Gaussian models using VB approximation and 4) parameter estimation using twisted particle filters. The main goal of the introductory part of the thesis is to connect the results to the general framework of estimation of state and model parameters and present them in a unified manner.Bayesian inference for non-linear state space models generally requires use of approximations, since the exact posterior distribution is readily available only for a few special cases. The approximation methods can be roughly classified into to groups: deterministic methods, where the intractable posterior distribution is approximated from a family of more tractable distributions (e.g. Gaussian and VB approximations), and stochastic sampling based methods (e.g. particle filters). Gaussian approximation refers to directly approximating the posterior with a Gaussian distribution, and can be readily applied for models with Gaussian process and measurement noise. Well known examples are the extended Kalman filter and sigma-point based unscented Kalman filter. The VB method is based on minimizing the Kullback-Leibler divergence of the true posterior with respect to the approximate distribution, chosen from a family of more tractable simpler distributions.The first main contribution of the thesis is the development of a VB approximation for linear regression problems with outlier robust measurement distributions. A broad family of outlier robust distributions can be presented as an infinite mixture of Gaussians, called Gaussian scale mixture models, and include e.g. the t-distribution, the Laplace distribution and the contaminated normal distribution. The VB approximation for the regression problem can be readily extended to the estimation of state space models and is presented in the introductory part.VB approximations can be also used for approximate inference in continuous-discrete Gaussian models, where the dynamics are modeled with stochastic differential equations and measurements are obtained at discrete time instants. The second main contribution is the presentation of a VB approximation for these models and the explanation of how the resulting algorithm connects to the Gaussian filtering and smoothing framework.The third contribution of the thesis is the development of parameter estimation using particle Markov Chain Monte Carlo (PMCMC) method and twisted particle filters. Twisted particle filters are obtained from standard particle filters by applying a special weighting to the sampling law of the filter. The weighting is chosen to minimize the variance of the marginal likelihood estimate, and the resulting particle filter is more efficient than conventional PMCMC algorithms. The exact optimal weighting is generally not available, but can be approximated using the Gaussian filtering and smoothing framework

    Inference and parameter estimation for diffusion processes

    Get PDF
    Diffusion processes provide a natural way of modelling a variety of physical and economic phenomena. It is often the case that one is unable to observe a diffusion process directly, and must instead rely on noisy observations that are discretely spaced in time. Given these discrete, noisy observations, one is faced with the task of inferring properties of the underlying diffusion process. For example, one might be interested in inferring the current state of the process given observations up to the present time (this is known as the filtering problem). Alternatively, one might wish to infer parameters governing the time evolution the diffusion process. In general, one cannot apply Bayes’ theorem directly, since the transition density of a general nonlinear diffusion is not computationally tractable. In this thesis, we investigate a novel method of simplifying the problem. The stochastic differential equation that describes the diffusion process is replaced with a simpler ordinary differential equation, which has a random driving noise that approximates Brownian motion. We show how one can exploit this approximation to improve on standard methods for inferring properties of nonlinear diffusion processes

    Unsupervised methods for large-scale, cell-resolution neural data analysis

    Get PDF
    In order to keep up with the volume of data, as well as the complexity of experiments and models in modern neuroscience, we need scalable and principled analytic programmes that take into account the scientific goals and the challenges of biological experiments. This work focuses on algorithms that tackle problems throughout the whole data analysis process. I first investigate how to best transform two-photon calcium imaging microscopy recordings – sets of contiguous images – into an easier-to-analyse matrix containing time courses of individual neurons. For this I first estimate how the true fluorescence signal gets transformed by tissue artefacts and the microscope setup, by learning the parameters of a realistic physical model from recorded data. Next, I describe how individual neural cell bodies may be segmented from the images, based on a cost function tailored to neural characteristics. Finally, I describe an interpretable non-linear dynamical model of neural population activity, which provides immediate scientific insight into complex system behaviour, and may spawn a new way of investigating stochastic non-linear dynamical systems. I hope the algorithms described here will not only be integrated into analytic pipelines of neural recordings, but also point out that algorithmic design should be informed by communication with the broader community, understanding and tackling the challenges inherent in experimental biological science

    Efficient Reinforcement Learning using Gaussian Processes

    Get PDF
    This book examines Gaussian processes (GPs) in model-based reinforcement learning (RL) and inference in nonlinear dynamic systems. First, we introduce PILCO, a fully Bayesian approach for efficient RL in continuous-valued state and action spaces when no expert knowledge is available. PILCO learns fast since it takes model uncertainties consistently into account during long-term planning and decision making. Thus, it reduces model bias, a common problem in model-based RL. Due to its generality and efficiency, PILCO is a conceptual and practical approach to jointly learning models and controllers fully automatically. Across all tasks, we report an unprecedented degree of automation and an unprecedented speed of learning. Second, we propose principled algorithms for robust filtering and smoothing in GP dynamic systems. Our methods are based on analytic moment matching and clearly advance state-of-the-art methods

    Reactive Probabilistic Programming for Scalable Bayesian Inference

    Get PDF
    corecore