Estimation of Dynamic Gaussian Processes

Gaussian processes provide a compact representation for modeling and estimating an unknown function, that can be updated as new measurements of the function are obtained. This paper extends this powerful framework to the case where the unknown function dynamically changes over time. Specifically, we assume that the function evolves according to an integro-difference equation and that the measurements are obtained locally in a spatial sense. In this setting, we will provide the expressions for the conditional mean and covariance of the process given the measurements, which results in a generalized estimation framework, for which we coined the term Dynamic Gaussian Process (DGP) estimation. This new framework generalizes both Gaussian process regression and Kalman filtering. For a broad class of kernels, described by a set of basis functions, fast implementations are provided. We illustrate the results on a numerical example, demonstrating that the method can accurately estimate an evolving continuous function, even in the presence of noisy measurements and disturbances.Comment: 6 pages, 3 figures, to be presented at 62nd IEEE Conference on Decision and Control, CDC 2023, Singapore, Singapor

Estimation of Dynamic Gaussian Processes

Gaussian processes provide a compact representation for modeling and estimating an unknown function, that can be updated as new measurements of the function are obtained. This paper extends this powerful framework to the case where the unknown function dynamically changes over time. Specifically, we assume that the function evolves according to an integro-difference equation and that the measurements are obtained locally in a spatial sense. In this setting, we will provide the expressions for the conditional mean and covariance of the process given the measurements, which results in a generalized estimation framework, for which we coined the term Dynamic Gaussian Process (DGP) estimation. This new framework generalizes both Gaussian process regression and Kalman filtering. For a broad class of kernels, described by a set of basis functions, fast implementations are provided. We illustrate the results on a numerical example, demonstrating that the method can accurately estimate an evolving continuous function, even in the presence of noisy measurements and disturbances

Bayesian Probabilistic Numerical Methods in Time-Dependent State Estimation for Industrial Hydrocyclone Equipment

The use of high-power industrial equipment, such as large-scale mixing equipment or a hydrocyclone for separation of particles in liquid suspension, demands careful monitoring to ensure correct operation. The fundamental task of state-estimation for the liquid suspension can be posed as a time-evolving inverse problem and solved with Bayesian statistical methods. In this article, we extend Bayesian methods to incorporate statistical models for the error that is incurred in the numerical solution of the physical governing equations. This enables full uncertainty quantification within a principled computation-precision trade-off, in contrast to the over-confident inferences that are obtained when all sources of numerical error are ignored. The method is cast within a sequential Monte Carlo framework and an optimized implementation is provided in Python

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions

Inference and simulation in the context of high-dimensional dynamical systems remain computationally challenging problems. Some form of dimensionality reduction is required to make the problem tractable in general. In this paper, we propose a novel approximate Gaussian filtering and smoothing method which propagates low-rank approximations of the covariance matrices. This is accomplished by projecting the Lyapunov equations associated with the prediction step to a manifold of low-rank matrices, which are then solved by a recently developed, numerically stable, dynamical low-rank integrator. Meanwhile, the update steps are made tractable by noting that the covariance update only transforms the column space of the covariance matrix, which is low-rank by construction. The algorithm differentiates itself from existing ensemble-based approaches in that the low-rank approximations of the covariance matrices are deterministic, rather than stochastic. Crucially, this enables the method to reproduce the exact Kalman filter as the low-rank dimension approaches the true dimensionality of the problem. Our method reduces computational complexity from cubic (for the Kalman filter) to \emph{quadratic} in the state-space size in the worst-case, and can achieve \emph{linear} complexity if the state-space model satisfies certain criteria. Through a set of experiments in classical data-assimilation and spatio-temporal regression, we show that the proposed method consistently outperforms the ensemble-based methods in terms of error in the mean and covariance with respect to the exact Kalman filter. This comes at no additional cost in terms of asymptotic computational complexity.Comment: 12 pages main text (including references) + 9 pages appendix, 6 figure

Efficient spatio-temporal Gaussian regression via Kalman filtering

We study the non-parametric reconstruction of spatio-temporal dynamical processes via Gaussian Processes (GPs) regression from sparse and noisy data. GPs have been mainly applied to spatial regression where they represent one of the most powerful estimation approaches also thanks to their universal representing properties. Their extension to dynamical processes has been instead elusive so far since classical implementations lead to unscalable algorithms or require some sort of approximation. We propose a novel procedure to address this problem by coupling GPs regression and Kalman filtering. In particular, assuming space/time separability of the covariance (kernel) of the process and rational time spectrum, we build a finite-dimensional discrete-time state-space process representation amenable to Kalman filtering. With sampling over a finite set of fixed spatial locations, our major finding is that the current Kalman filter state represents a sufficient statistic to compute the minimum variance estimate of the process at any future time over the entire spatial domain. In machine learning, a representer theorem states that an important class of infinite-dimensional variational problems admits a computable and finite-dimensional exact solution. In view of this, our result can be interpreted as a novel Dynamic Representer Theorem for GPs. We then extend the study to situations where the spatial input locations set varies over time. The proposed algorithms are tested on both synthetic and real field data, providing comparisons with standard GP and truncated GP regression techniques

System Identification and Spatio-Temporal Kalman Filtering for Muscular Pressure Data

Denne masteroppgaven er et første skritt mot å oppnå ett glattet estimat av trykket som skapes av bekkenbunnsmuskulaturen. Metoden som presenteres kan deles inn i to skritt. Først gjøres system identifikasjon for å finne en polynomisk modell av systemets dynamikk. Signalene fra hjernen som styrer musklene approksimeres som et binært inngangssignal ved hjelp av en change detection algoritme. Dette inngangssignalet må tidsskiftes tilbake i tid, slik at inngangssignalet eksiteres før trykket. Trykkdataen som er samlet av FemFit, en vaginal trykksensor, kombineres med det estimerte inngangssignalet for å finne en diskret-tids ARX modell for hver av de 8 sensorene som FemFit består av. Dette gjøres ved å anta at hver av trykkmålingene har samme fordeling, med unntak av en skaleringsfaktor på inngangssignalet. Skaleringsfaktoren, ARX parametrene og tidsskiftet optimeres samtidig for ARX modeller med forskjellig orden. Alle de evaluerte ARX modellene oppnådde et simulerings-fit på omtrent 45% på valideringsdata, noe som indikerer at nøyaktigheten begrenses av den simple estimeringen av inngangssignalet. Den enkleste modellen, ARX211, konverteres til en tilstandsromrepresentasjon. Tilstandsromrepresentasjonen brukes i det andre skrittet til gaussisk regresjon i tid og rom ved hjelp av et Kalman filter. Kalman filteret er tilpasset fra artikkelen "Efficient Spatio-Temporal Gaussian Regression via Kalman Filtering" av Todesca et al. (2020) for bruk med muskeltrykk som data. I motsetning til den originale algoritmen, blir et system funnet i diskret-tid direkte. På grunn av dette må den initielle kovariansen estimeres, og støy kovariansene må stilles inn. Verdier for kovariansene som modellerer dynamikken tilstrekkelig ble ikke funnet, men den glattende egenskapen -- som glatter ut støy -- blir presentert. Til slutt blir noen forbedringer foreslått som fremtidig arbeid, og noen potensielle bruksområder nevnes