Identification of multi-object dynamical systems: consistency and Fisher information

Learning the model parameters of a multi-object dynamical system from partial and perturbed observations is a challenging task. Despite recent numerical advancements in learning these parameters, theoretical guarantees are extremely scarce. In this article, we study the identifiability of these parameters and the consistency of the corresponding maximum likelihood estimate (MLE) under assumptions on the different components of the underlying multi-object system. In order to understand the impact of the various sources of observation noise on the ability to learn the model parameters, we study the asymptotic variance of the MLE through the associated Fisher information matrix. For example, we show that specific aspects of the multi-target tracking (MTT) problem such as detection failures and unknown data association lead to a loss of information which is quantified in special cases of interest. To the best of the authors' knowledge, these are new theoretically-backed insights on the subtleties of MTT parameter learning.Funding: All authors were supported by Singapore Ministry of Education tier 1 grant R-155-000-182-114

Distributed Maximum Likelihood for Simultaneous Self-localization and Tracking in Sensor Networks

We show that the sensor self-localization problem can be cast as a static parameter estimation problem for Hidden Markov Models and we implement fully decentralized versions of the Recursive Maximum Likelihood and on-line Expectation-Maximization algorithms to localize the sensor network simultaneously with target tracking. For linear Gaussian models, our algorithms can be implemented exactly using a distributed version of the Kalman filter and a novel message passing algorithm. The latter allows each node to compute the local derivatives of the likelihood or the sufficient statistics needed for Expectation-Maximization. In the non-linear case, a solution based on local linearization in the spirit of the Extended Kalman Filter is proposed. In numerical examples we demonstrate that the developed algorithms are able to learn the localization parameters.Comment: shorter version is about to appear in IEEE Transactions of Signal Processing; 22 pages, 15 figure

Bayesian learning of noisy Markov decision processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller

A Backward Particle Interpretation of Feynman-Kac Formulae

We design a particle interpretation of Feynman-Kac measures on path spaces based on a backward Markovian representation combined with a traditional mean field particle interpretation of the flow of their final time marginals. In contrast to traditional genealogical tree based models, these new particle algorithms can be used to compute normalized additive functionals "on-the-fly" as well as their limiting occupation measures with a given precision degree that does not depend on the final time horizon. We provide uniform convergence results w.r.t. the time horizon parameter as well as functional central limit theorems and exponential concentration estimates. We also illustrate these results in the context of computational physics and imaginary time Schroedinger type partial differential equations, with a special interest in the numerical approximation of the invariant measure associated to $h$-processes

Uniform Stability of a Particle Approximation of the Optimal Filter Derivative

Sequential Monte Carlo methods, also known as particle methods, are a widely used set of computational tools for inference in non-linear non-Gaussian state-space models. In many applications it may be necessary to compute the sensitivity, or derivative, of the optimal filter with respect to the static parameters of the state-space model; for instance, in order to obtain maximum likelihood model parameters of interest, or to compute the optimal controller in an optimal control problem. In Poyiadjis et al. [2011] an original particle algorithm to compute the filter derivative was proposed and it was shown using numerical examples that the particle estimate was numerically stable in the sense that it did not deteriorate over time. In this paper we substantiate this claim with a detailed theoretical study. Lp bounds and a central limit theorem for this particle approximation of the filter derivative are presented. It is further shown that under mixing conditions these Lp bounds and the asymptotic variance characterized by the central limit theorem are uniformly bounded with respect to the time index. We demon- strate the performance predicted by theory with several numerical examples. We also use the particle approximation of the filter derivative to perform online maximum likelihood parameter estimation for a stochastic volatility model