Periodic Splines and Gaussian Processes for the Resolution of Linear Inverse Problems

This paper deals with the resolution of inverse problems in a periodic setting or, in other terms, the reconstruction of periodic continuous-domain signals from their noisy measurements. We focus on two reconstruction paradigms: variational and statistical. In the variational approach, the reconstructed signal is solution to an optimization problem that establishes a tradeoff between fidelity to the data and smoothness conditions via a quadratic regularization associated to a linear operator. In the statistical approach, the signal is modeled as a stationary random process defined from a Gaussian white noise and a whitening operator; one then looks for the optimal estimator in the mean-square sense. We give a generic form of the reconstructed signals for both approaches, allowing for a rigorous comparison of the two.We fully characterize the conditions under which the two formulations yield the same solution, which is a periodic spline in the case of sampling measurements. We also show that this equivalence between the two approaches remains valid on simulations for a broad class of problems. This extends the practical range of applicability of the variational method

Functional Regression

Functional data analysis (FDA) involves the analysis of data whose ideal units of observation are functions defined on some continuous domain, and the observed data consist of a sample of functions taken from some population, sampled on a discrete grid. Ramsay and Silverman's 1997 textbook sparked the development of this field, which has accelerated in the past 10 years to become one of the fastest growing areas of statistics, fueled by the growing number of applications yielding this type of data. One unique characteristic of FDA is the need to combine information both across and within functions, which Ramsay and Silverman called replication and regularization, respectively. This article will focus on functional regression, the area of FDA that has received the most attention in applications and methodological development. First will be an introduction to basis functions, key building blocks for regularization in functional regression methods, followed by an overview of functional regression methods, split into three types: [1] functional predictor regression (scalar-on-function), [2] functional response regression (function-on-scalar) and [3] function-on-function regression. For each, the role of replication and regularization will be discussed and the methodological development described in a roughly chronological manner, at times deviating from the historical timeline to group together similar methods. The primary focus is on modeling and methodology, highlighting the modeling structures that have been developed and the various regularization approaches employed. At the end is a brief discussion describing potential areas of future development in this field

Modelling Periodic Measurement Data Having a Piecewise Polynomial Trend Using the Method of Variable Projection

This paper presents a new method for modelling periodic signals having an aperiodic trend, using the method of variable projection. It is a major extension to the IEEE-standard 1057 by permitting the background to be time varying; additionally, any number of harmonics of the periodic portion can be modelled. This paper focuses on using B-Splines to implement a piecewise polynomial model for the aperiodic portion of the signal. A thorough algebraic derivation of the method is presented, as well as a comparison to using global polynomial approximation. It is proven that B-Splines work better for modelling a more complicated aperiodic portion when compared to higher order polynomials. Furthermore, the piecewise polynomial model is capable of modelling the local signal variations produced by the interaction of a control system with a process in industrial applications. The method of variable projection reduces the problem to a one-dimensional nonlinear optimization, combined with a linear least-squares computation. An added benefit of using the method of variable projection is the possibility to calculate the covariances of the linear coefficients of the model, enabling the calculation of confidence and prediction intervals. The method is tested on both real measurement data acquired in industrial processes, as well as synthetic data. The method shows promising results for the precise characterization of periodic signals embedded in highly complex aperiodic backgrounds. Finally, snippets of the m-code are provided, together with a toolbox for B-Splines, which permit the implementation of the complete computation

Actually Sparse Variational Gaussian Processes

Gaussian processes (GPs) are typically criticised for their unfavourable scaling in both computational and memory requirements. For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data. In practice however, for large datasets requiring many inducing variables, such as low-lengthscale spatial data, even sparse GPs can become computationally expensive, limited by the number of inducing variables one can use. In this work, we propose a new class of inter-domain variational GP, constructed by projecting a GP onto a set of compactly supported B-spline basis functions. The key benefit of our approach is that the compact support of the B-spline basis functions admits the use of sparse linear algebra to significantly speed up matrix operations and drastically reduce the memory footprint. This allows us to very efficiently model fast-varying spatial phenomena with tens of thousands of inducing variables, where previous approaches failed

Actually Sparse Variational Gaussian Processes

Gaussian processes (GPs) are typically criticised for their unfavourable scaling in both computational and memory requirements. For large datasets, sparse GPs reduce these demands by conditioning on a small set of inducing variables designed to summarise the data. In practice however, for large datasets requiring many inducing variables, such as low-lengthscale spatial data, even sparse GPs can become computationally expensive, limited by the number of inducing variables one can use. In this work, we propose a new class of inter-domain variational GP, constructed by projecting a GP onto a set of compactly supported B-spline basis functions. The key benefit of our approach is that the compact support of the B-spline basis functions admits the use of sparse linear algebra to significantly speed up matrix operations and drastically reduce the memory footprint. This allows us to very efficiently model fast-varying spatial phenomena with tens of thousands of inducing variables, where previous approaches failed.Comment: 14 pages, 5 figures, published in AISTATS 202

Poisson inverse problems

In this paper we focus on nonparametric estimators in inverse problems for Poisson processes involving the use of wavelet decompositions. Adopting an adaptive wavelet Galerkin discretization, we find that our method combines the well-known theoretical advantages of wavelet--vaguelette decompositions for inverse problems in terms of optimally adapting to the unknown smoothness of the solution, together with the remarkably simple closed-form expressions of Galerkin inversion methods. Adapting the results of Barron and Sheu [Ann. Statist. 19 (1991) 1347--1369] to the context of log-intensity functions approximated by wavelet series with the use of the Kullback--Leibler distance between two point processes, we also present an asymptotic analysis of convergence rates that justifies our approach. In order to shed some light on the theoretical results obtained and to examine the accuracy of our estimates in finite samples, we illustrate our method by the analysis of some simulated examples.Comment: Published at http://dx.doi.org/10.1214/009053606000000687 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Self-similar prior and wavelet bases for hidden incompressible turbulent motion

This work is concerned with the ill-posed inverse problem of estimating turbulent flows from the observation of an image sequence. From a Bayesian perspective, a divergence-free isotropic fractional Brownian motion (fBm) is chosen as a prior model for instantaneous turbulent velocity fields. This self-similar prior characterizes accurately second-order statistics of velocity fields in incompressible isotropic turbulence. Nevertheless, the associated maximum a posteriori involves a fractional Laplacian operator which is delicate to implement in practice. To deal with this issue, we propose to decompose the divergent-free fBm on well-chosen wavelet bases. As a first alternative, we propose to design wavelets as whitening filters. We show that these filters are fractional Laplacian wavelets composed with the Leray projector. As a second alternative, we use a divergence-free wavelet basis, which takes implicitly into account the incompressibility constraint arising from physics. Although the latter decomposition involves correlated wavelet coefficients, we are able to handle this dependence in practice. Based on these two wavelet decompositions, we finally provide effective and efficient algorithms to approach the maximum a posteriori. An intensive numerical evaluation proves the relevance of the proposed wavelet-based self-similar priors.Comment: SIAM Journal on Imaging Sciences, 201

On the Uniqueness of Inverse Problems with Fourier-domain Measurements and Generalized TV Regularization

We study the super-resolution problem of recovering a periodic continuous-domain function from its low-frequency information. This means that we only have access to possibly corrupted versions of its Fourier samples up to a maximum cut-off frequency. The reconstruction task is specified as an optimization problem with generalized total-variation regularization involving a pseudo-differential operator. Our special emphasis is on the uniqueness of solutions. We show that, for elliptic regularization operators (e.g., the derivatives of any order), uniqueness is always guaranteed. To achieve this goal, we provide a new analysis of constrained optimization problems over Radon measures. We demonstrate that either the solutions are always made of Radon measures of constant sign, or the solution is unique. Doing so, we identify a general sufficient condition for the uniqueness of the solution of a constrained optimization problem with TV-regularization, expressed in terms of the Fourier samples.Comment: 20 page