Optimization viewpoint on Kalman smoothing, with applications to robust and sparse estimation

In this paper, we present the optimization formulation of the Kalman filtering and smoothing problems, and use this perspective to develop a variety of extensions and applications. We first formulate classic Kalman smoothing as a least squares problem, highlight special structure, and show that the classic filtering and smoothing algorithms are equivalent to a particular algorithm for solving this problem. Once this equivalence is established, we present extensions of Kalman smoothing to systems with nonlinear process and measurement models, systems with linear and nonlinear inequality constraints, systems with outliers in the measurements or sudden changes in the state, and systems where the sparsity of the state sequence must be accounted for. All extensions preserve the computational efficiency of the classic algorithms, and most of the extensions are illustrated with numerical examples, which are part of an open source Kalman smoothing Matlab/Octave package.Comment: 46 pages, 11 figure

A Schur complement approach to preconditioning sparse linear least-squares problems with some dense rows

The effectiveness of sparse matrix techniques for directly solving large-scale linear least-squares problems is severely limited if the system matrix A has one or more nearly dense rows. In this paper, we partition the rows of A into sparse rows and dense rows (A s and A d ) and apply the Schur complement approach. A potential difficulty is that the reduced normal matrix AsTA s is often rank-deficient, even if A is of full rank. To overcome this, we propose explicitly removing null columns of A s and then employing a regularization parameter and using the resulting Cholesky factors as a preconditioner for an iterative solver applied to the symmetric indefinite reduced augmented system. We consider complete factorizations as well as incomplete Cholesky factorizations of the shifted reduced normal matrix. Numerical experiments are performed on a range of large least-squares problems arising from practical applications. These demonstrate the effectiveness of the proposed approach when combined with either a sparse parallel direct solver or a robust incomplete Cholesky factorization algorithm

The importance of component-wise stochasticity in particle swarm optimization

This paper illustrates the importance of independent, component-wise stochastic scaling values, from both a theoretical and empirical perspective. It is shown that a swarm employing scalar stochasticity is unable to express every point in the search space if the problem dimensionality is sufficiently large in comparison to the swarm size. The theoretical result is emphasized by an empirical experiment, comparing the performance of a scalar swarm on benchmarks with reachable and unreachable optima. It is shown that a swarm using scalar stochasticity performs significantly worse when the optimum is not in the span of its initial positions. Lastly, it is demonstrated that a scalar swarm performs significantly worse than a swarm with component-wise stochasticity on a large range of benchmark functions, even when the problem dimensionality allows the scalar swarm to reach the optima.The National Research Foundation (NRF) of South Africa (Grant Number 46712).http://link.springer.combookseries/5582019-10-03hj2018Computer Scienc

Fast interior point solution of quadratic programming problems arising from PDE-constrained optimization

Interior point methods provide an attractive class of approaches for solving linear, quadratic and nonlinear programming problems, due to their excellent efficiency and wide applicability. In this paper, we consider PDE-constrained optimization problems with bound constraints on the state and control variables, and their representation on the discrete level as quadratic programming problems. To tackle complex problems and achieve high accuracy in the solution, one is required to solve matrix systems of huge scale resulting from Newton iteration, and hence fast and robust methods for these systems are required. We present preconditioned iterative techniques for solving a number of these problems using Krylov subspace methods, considering in what circumstances one may predict rapid convergence of the solvers in theory, as well as the solutions observed from practical computations

The structure of iterative methods for symmetric linear discrete ill-posed problems

The iterative solution of large linear discrete ill-posed problems with an error contaminated data vector requires the use of specially designed methods in order to avoid severe error propagation. Range restricted minimal residual methods have been found to be well suited for the solution of many such problems. This paper discusses the structure of matrices that arise in a range restricted minimal residual method for the solution of large linear discrete ill-posed problems with a symmetric matrix. The exploitation of the structure results in a method that is competitive with respect to computer storage, number of iterations, and accuracy.Acknowledgments We would like to thank the referees for comments. The work of F. M. was supported by Dirección General de Investigación Científica y Técnica, Ministerio de Economía y Competitividad of Spain under grant MTM2012-36732-C03-01. Work of L. R. was supported by Universidad Carlos III de Madrid in the Department of Mathematics during the academic year 2010-2011 within the framework of the Chair of Excellence Program and by NSF grant DMS-1115385

Multidirectional Subspace Expansion for One-Parameter and Multiparameter Tikhonov Regularization

Tikhonov regularization is a popular method to approximate solutions of linear discrete ill-posed problems when the observed or measured data is contaminated by noise. Multiparameter Tikhonov regularization may improve the quality of the computed approximate solutions. We propose a new iterative method for large-scale multiparameter Tikhonov regularization with general regularization operators based on a multidirectional subspace expansion. The multidirectional subspace expansion may be combined with subspace truncation to avoid excessive growth of the search space. Furthermore, we introduce a simple and effective parameter selection strategy based on the discrepancy principle and related to perturbation results

Peaks and Troughs of Three-Dimensional Vestibulo-ocular Reflex in Humans

The three-dimensional vestibulo-ocular reflex (3D VOR) ideally generates compensatory ocular rotations not only with a magnitude equal and opposite to the head rotation but also about an axis that is collinear with the head rotation axis. Vestibulo-ocular responses only partially fulfill this ideal behavior. Because animal studies have shown that vestibular stimulation about particular axes may lead to suboptimal compensatory responses, we investigated in healthy subjects the peaks and troughs in 3D VOR stabilization in terms of gain and alignment of the 3D vestibulo-ocular response. Six healthy upright sitting subjects underwent whole body small amplitude sinusoidal and constant acceleration transients delivered by a six-degree-of-freedom motion platform. Subjects were oscillated about the vertical axis and about axes in the horizontal plane varying between roll and pitch at increments of 22.5° in azimuth. Transients were delivered in yaw, roll, and pitch and in the vertical canal planes. Eye movements were recorded in with 3D search coils. Eye coil signals were converted to rotation vectors, from which we calculated gain and misalignment. During horizontal axis stimulation, systematic deviations were found. In the light, misalignment of the 3D VOR had a maximum misalignment at about 45°. These deviations in misalignment can be explained by vector summation of the eye rotation components with a low gain for torsion and high gain for vertical. In the dark and in response to transients, gain of all components had lower values. Misalignment in darkness and for transients had different peaks and troughs than in the light: its minimum was during pitch axis stimulation and its maximum during roll axis stimulation. We show that the relatively large misalignment for roll in darkness is due to a horizontal eye movement component that is only present in darkness. In combination with the relatively low torsion gain, this horizontal component has a relative large effect on the alignment of the eye rotation axis with respect to the head rotation axis

Matching Schur complement approximations for certain saddle-point systems

The solution of many practical problems described by mathematical models requires approximation methods that give rise to linear(ized) systems of equations, solving which will determine the desired approximation. This short contribution describes a particularly effective solution approach for a certain class of so-called saddle-point linear systems which arises in different contexts