Optimization viewpoint on Kalman smoothing, with applications to robust and sparse estimation

In this paper, we present the optimization formulation of the Kalman filtering and smoothing problems, and use this perspective to develop a variety of extensions and applications. We first formulate classic Kalman smoothing as a least squares problem, highlight special structure, and show that the classic filtering and smoothing algorithms are equivalent to a particular algorithm for solving this problem. Once this equivalence is established, we present extensions of Kalman smoothing to systems with nonlinear process and measurement models, systems with linear and nonlinear inequality constraints, systems with outliers in the measurements or sudden changes in the state, and systems where the sparsity of the state sequence must be accounted for. All extensions preserve the computational efficiency of the classic algorithms, and most of the extensions are illustrated with numerical examples, which are part of an open source Kalman smoothing Matlab/Octave package.Comment: 46 pages, 11 figure

Kalman Filtering With State Constraints: A Survey of Linear and Nonlinear Algorithms

The Kalman filter is the minimum-variance state estimator for linear dynamic systems with Gaussian noise. Even if the noise is non-Gaussian, the Kalman filter is the best linear estimator. For nonlinear systems it is not possible, in general, to derive the optimal state estimator in closed form, but various modifications of the Kalman filter can be used to estimate the state. These modifications include the extended Kalman filter, the unscented Kalman filter, and the particle filter. Although the Kalman filter and its modifications are powerful tools for state estimation, we might have information about a system that the Kalman filter does not incorporate. For example, we may know that the states satisfy equality or inequality constraints. In this case we can modify the Kalman filter to exploit this additional information and get better filtering performance than the Kalman filter provides. This paper provides an overview of various ways to incorporate state constraints in the Kalman filter and its nonlinear modifications. If both the system and state constraints are linear, then all of these different approaches result in the same state estimate, which is the optimal constrained linear state estimate. If either the system or constraints are nonlinear, then constrained filtering is, in general, not optimal, and different approaches give different results

Kalman Filtering With State Constraints: A Survey of Linear and Nonlinear Algorithms

The Kalman filter is the minimum-variance state estimator for linear dynamic systems with Gaussian noise. Even if the noise is non-Gaussian, the Kalman filter is the best linear estimator. For nonlinear systems it is not possible, in general, to derive the optimal state estimator in closed form, but various modifications of the Kalman filter can be used to estimate the state. These modifications include the extended Kalman filter, the unscented Kalman filter, and the particle filter. Although the Kalman filter and its modifications are powerful tools for state estimation, we might have information about a system that the Kalman filter does not incorporate. For example, we may know that the states satisfy equality or inequality constraints. In this case we can modify the Kalman filter to exploit this additional information and get better filtering performance than the Kalman filter provides. This paper provides an overview of various ways to incorporate state constraints in the Kalman filter and its nonlinear modifications. If both the system and state constraints are linear, then all of these different approaches result in the same state estimate, which is the optimal constrained linear state estimate. If either the system or constraints are nonlinear, then constrained filtering is, in general, not optimal, and different approaches give different results

Porovnání metod pro odhad omezených veličin s aplikací na ekonomická data

Diplomová práce představuje přehled základních technik pro filtrování nepozorovaných proměnných při použití stavové reprezentace modelu a stavových omezení ve tvaru nerovnic. Zabývá se především odvozením Kalmanova filtru, jeho rozšířením do nelineární podoby a použitím omezení na stavové proměnné. Alternativní přístupy pomocí stavového modelu s rovnoměrně rozloženým šumem a Sequential importance sampling jako jedna z metod Particle filtrů využívající Monte Carlo simulace jsou také popsány. Všechny tři metody jsou aplikovány na semistrukturální model použitelný pro analýzu měnové politiky. Filtrace používá makroekonomická data české ekonomiky a zohledňuje nezáporné omezení úrokových sazeb jako jeden z modelových stavů. Výsledky jsou navzájem srovnány a diskutovány.The thesis introduces an overview of techniques for filtering of unobserved variables using a state-space representation of a model and state inequality constraints. It is mainly aimed at a derivation of the linear Kalman filter, its extension into a form of a non-linear filter and imposing state constraints. The state uniform model with noise bounds and the sequential importance sampling, as a method of particle filters using Monte Carlo simulations, are described as alternative methods. These three methods are applied on a simple semi-structural model for a monetary policy analysis. The filtration is based on Czech macroeconomic data and reflects an imposed non-negative state constraint on the interest rate. Results of the algorithms are compared and discussed.Katedra pravděpodobnosti a matematické statistikyDepartment of Probability and Mathematical StatisticsMatematicko-fyzikální fakultaFaculty of Mathematics and Physic

A framework for non-Gaussian signal modeling and estimation

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1999.Includes bibliographical references (p. [235]-240).by Shawn Matthew Verbout.Ph.D

A framework for non-Gaussian signal modeling and estimation

"June 1999."Includes bibliographical references (p. [225]-240).Sponsored by the U.S. Air Force. F49620-96-1-0072 Supported by the U.S. Army Research Laboratory under Cooperative Agreement, DAAL01-96-2-001Shawn M. Verbout

Constrained expectation-maximization (EM), dynamic analysis, linear quadratic tracking, and nonlinear constrained expectation-maximation (EM) for the analysis of genetic regulatory networks and signal transduction networks

Despite the immense progress made by molecular biology in cataloging andcharacterizing molecular elements of life and the success in genome sequencing, therehave not been comparable advances in the functional study of complex phenotypes.This is because isolated study of one molecule, or one gene, at a time is not enough byitself to characterize the complex interactions in organism and to explain the functionsthat arise out of these interactions. Mathematical modeling of biological systems isone way to meet the challenge.My research formulates the modeling of gene regulation as a control problem andapplies systems and control theory to the identification, analysis, and optimal controlof genetic regulatory networks. The major contribution of my work includes biologicallyconstrained estimation, dynamical analysis, and optimal control of genetic networks.In addition, parameter estimation of nonlinear models of biological networksis also studied, as a parameter estimation problem of a general nonlinear dynamicalsystem. Results demonstrate the superior predictive power of biologically constrainedstate-space models, and that genetic networks can have differential dynamic propertieswhen subjected to different environmental perturbations. Application of optimalcontrol demonstrates feasibility of regulating gene expression levels. In the difficultproblem of parameter estimation, generalized EM algorithm is deployed, and a set of explicit formula based on extended Kalman filter is derived. Application of themethod to synthetic and real world data shows promising results