Spherical Slepian functions and the polar gap in geodesy

The estimation of potential fields such as the gravitational or magnetic potential at the surface of a spherical planet from noisy observations taken at an altitude over an incomplete portion of the globe is a classic example of an ill-posed inverse problem. Here we show that the geodetic estimation problem has deep-seated connections to Slepian's spatiospectral localization problem on the sphere, which amounts to finding bandlimited spherical functions whose energy is optimally concentrated in some closed portion of the unit sphere. This allows us to formulate an alternative solution to the traditional damped least-squares spherical harmonic approach in geodesy, whereby the source field is now expanded in a truncated Slepian function basis set. We discuss the relative performance of both methods with regard to standard statistical measures as bias, variance and mean-square error, and pay special attention to the algorithmic efficiency of computing the Slepian functions on the region complementary to the axisymmetric polar gap characteristic of satellite surveys. The ease, speed, and accuracy of this new method makes the use of spherical Slepian functions in earth and planetary geodesy practical.Comment: 14 figures, submitted to the Geophysical Journal Internationa

Robust Network Topology Inference and Processing of Graph Signals

The abundance of large and heterogeneous systems is rendering contemporary data more pervasive, intricate, and with a non-regular structure. With classical techniques facing troubles to deal with the irregular (non-Euclidean) domain where the signals are defined, a popular approach at the heart of graph signal processing (GSP) is to: (i) represent the underlying support via a graph and (ii) exploit the topology of this graph to process the signals at hand. In addition to the irregular structure of the signals, another critical limitation is that the observed data is prone to the presence of perturbations, which, in the context of GSP, may affect not only the observed signals but also the topology of the supporting graph. Ignoring the presence of perturbations, along with the couplings between the errors in the signal and the errors in their support, can drastically hinder estimation performance. While many GSP works have looked at the presence of perturbations in the signals, much fewer have looked at the presence of perturbations in the graph, and almost none at their joint effect. While this is not surprising (GSP is a relatively new field), we expect this to change in the upcoming years. Motivated by the previous discussion, the goal of this thesis is to advance toward a robust GSP paradigm where the algorithms are carefully designed to incorporate the influence of perturbations in the graph signals, the graph support, and both. To do so, we consider different types of perturbations, evaluate their disruptive impact on fundamental GSP tasks, and design robust algorithms to address them.Comment: Dissertatio

Efficient Estimation of Signals via Non-Convex Approaches

This dissertation aims to highlight the importance of methodological development and the need for tailored algorithms in non-convex statistical problems. Specifically, we study three non-convex estimation problems with novel ideas and techniques in both statistical methodologies and algorithmic designs. Chapter 2 discusses my work with Zhou Fan on estimation of a piecewise-constant image, or a gradient-sparse signal on a general graph, from noisy linear measurements. In this work, we propose and study an iterative algorithm to minimize a penalized least-squares objective, with a penalty given by the ``$\ell_0$-norm\u27\u27 of the signal\u27s discrete graph gradient. The method uses a non-convex variant of proximal gradient descent, applying the alpha-expansion procedure to approximate the proximal mapping in each iteration, and using a geometric decay of the penalty parameter across iterations to ensure convergence. Under a cut-restricted isometry property for the measurement design, we prove global recovery guarantees for the estimated signal. For standard Gaussian designs, the required number of measurements is independent of the graph structure, and improves upon worst-case guarantees for total-variation (TV) compressed sensing on the 1-D line and 2-D lattice graphs by polynomial and logarithmic factors, respectively. The method empirically yields lower mean-squared recovery error compared with TV regularization in regimes of moderate undersampling and moderate to high signal-to-noise, for several examples of changepoint signals and gradient-sparse phantom images. Chapter 3 discusses my work with Zhou Fan and Sahand Negahban on tree-projected gradient descent for estimating gradient-sparse parameters. We consider estimating a gradient-sparse parameter $\boldsymbol{\theta}^*\in\mathbb{R}^p$, having strong gradient-sparsity $s^*:=\|\nabla_G \boldsymbol{\theta}^*\|_0$ on an underlying graph $G$. Given observations$Z_1,\ldots,Z_n$ and a smooth, convex loss function $\mathcal{L}$ for which our parameter of interest $\boldsymbol{\theta}^*$ minimizes the population risk \mathbb{E}[\mathcal{L}(\btheta;Z_1,\ldots,Z_n)], we propose to estimate $\boldsymbol{\theta}^*$ by a projected gradient descent algorithm that iteratively and approximately projects gradient steps onto spaces of vectors having small gradient-sparsity over low-degree spanning trees of $G$. We show that, under suitable restricted strong convexity and smoothness assumptions for the loss, the resulting estimator achieves the squared-error risk $\frac{s^*}{n} \log (1+\frac{p}{s^*})$ up to a multiplicative constant that is independent of $G$. In contrast, previous polynomial-time algorithms have only been shown to achieve this guarantee in more specialized settings, or under additional assumptions for $G$ and/or the sparsity pattern of $\nabla_G \boldsymbol{\theta}^*$. As applications of our general framework, we apply our results to the examples of linear models and generalized linear models with random design. Chapter 4 discusses my joint work with Zhou Fan, Roy R. Lederman, Yi Sun, and Tianhao Wang on maximum likelihood for high-noise group orbit estimation. Motivated by applications to single-particle cryo-electron microscopy (cryo-EM), we study several problems of function estimation in a low SNR regime, where samples are observed under random rotations of the function domain. In a general framework of group orbit estimation with linear projection, we describe a stratification of the Fisher information eigenvalues according to a sequence of transcendence degrees in the invariant algebra, and relate critical points of the log-likelihood landscape to a sequence of method-of-moments optimization problems. This extends previous results for a discrete rotation group without projection. We then compute these transcendence degrees and the forms of these moment optimization problems for several examples of function estimation under $\mathsf{SO}(2)$ and $\mathsf{SO}(3)$ rotations. For several of these examples, we affirmatively resolve numerical conjectures that $3^\text{rd}$-order moments are sufficient to locally identify a generic signal up to its rotational orbit, and also confirm the existence of spurious local optima for the landscape of the population log-likelihood. For low-dimensional approximations of the electric potential maps of two small protein molecules, we empirically verify that the noise-scalings of the Fisher information eigenvalues conform with these theoretical predictions over a range of SNR, in a model of $\mathsf{SO}(3)$ rotations without projection

Nonlinear and distributed sensory estimation

Methods to improve performance of sensors with regard to sensor nonlinearity, sensor noise and sensor bandwidths are investigated and new algorithms are developed. The necessity of the proposed research has evolved from the ever-increasing need for greater precision and improved reliability in sensor measurements. After describing the current state of the art of sensor related issues like nonlinearity and bandwidth, research goals are set to create a new trend on the usage of sensors. We begin the investigation with a detailed distortion analysis of nonlinear sensors. A need for efficient distortion compensation procedures is further justified by showing how a slight deviation from the linearity assumption leads to a very severe distortion in time and in frequency domains. It is argued that with a suitable distortion compensation technique the danger of having an infinite bandwidth nonlinear sensory operation, which is dictated by nonlinear distortion, can be avoided. Several distortion compensation techniques are developed and their performance is validated by simulation and experimental results. Like any other model-based technique, modeling errors or model uncertainty affects performance of the proposed scheme, this leads to the innovation of robust signal reconstruction. A treatment for this problem is given and a novel technique, which uses a nominal model instead of an accurate model and produces the results that are robust to model uncertainty, is developed. The means to attain a high operating bandwidth are developed by utilizing several low bandwidth pass-band sensors. It is pointed out that instead of using a single sensor to measure a high bandwidth signal, there are many advantages of using an array of several pass-band sensors. Having shown that employment of sensor arrays is an economic incentive and practical, several multi-sensor fusion schemes are developed to facilitate their implementation. Another aspect of this dissertation is to develop means to deal with outliers in sensor measurements. As fault sensor data detection is an essential element of multi-sensor network implementation, which is used to improve system reliability and robustness, several sensor scheduling configurations are derived to identify and to remove outliers

Two Studies in Representation of Signals

The thesis consists of two parts. In the first part deals with a multi-scale approach to vector quantization. An algorithm, dubbed reconstruction trees, is proposed and analyzed. Here the goal is parsimonious reconstruction of unsupervised data; the algorithm leverages a family of given partitions, to quickly explore the data in a coarse-to-fine multi-scale fashion. The main technical contribution is an analysis of the expected distortion achieved by the proposed algorithm, when the data are assumed to be sampled from a fixed unknown probability measure. Both asymptotic and finite sample results are provided, under suitable regularity assumptions on the probability measure. Special attention is devoted to the case in which the probability measure is supported on a smooth sub-manifold of the ambient space, and is absolutely continuous with respect to the Riemannian measure of it; in this case asymptotic optimal quantization is well understood and a benchmark for understanding the results is offered. The second part of the thesis deals with a novel approach to Graph Signal Processing which is based on Matroid Theory. Graph Signal Processing is the study of complex functions of the vertex set of a graph, based on the combinatorial Graph Laplacian operator of the underlying graph. This naturally gives raise to a linear operator, that to many regards resembles a Fourier transform, mirroring the graph domain into a frequency domain. On the one hand this structure asymptotically tends to mimic analysis on locally compact groups or manifolds, but on the other hand its discrete nature triggers a whole new scenario of algebraic phenomena. Hints towards making sense of this scenario are objects that already embody a discrete nature in continuous setting, such as measures with discrete support in time and frequency, also called Dirac combs. While these measures are key towards formulating sampling theorems and constructing wavelet frames in time-frequency Analysis, in the graph-frequency setting these boil down to distinguished combinatorial objects, the so called Circuits of a matroid, corresponding to the Fourier transform operator. In a particularly symmetric case, corresponding to Cayley graphs of finite abelian groups, the Dirac combs are proven to completely describe the so called lattice of cyclic flats, exhibiting the property of being atomistic, among other properties. This is a strikingly concise description of the matroid, that opens many questions concerning how this highly regular structure relaxes into more general instances. Lastly, a related problem concerning the combinatorial interplay between Fourier operator and its Spectrum is described, provided with some ideas towards its future development

CPGD: Cadzow Plug-and-Play Gradient Descent for Generalised FRI

Finite rate of innovation (FRI) is a powerful reconstruction framework enabling the recovery of sparse Dirac streams from uniform low-pass filtered samples. An extension of this framework, called generalised FRI (genFRI), has been recently proposed for handling cases with arbitrary linear measurement models. In this context, signal reconstruction amounts to solving a joint constrained optimisation problem, yielding estimates of both the Fourier series coefficients of the Dirac stream and its so-called annihilating filter, involved in the regularisation term. This optimisation problem is however highly non convex and non linear in the data. Moreover, the proposed numerical solver is computationally intensive and without convergence guarantee. In this work, we propose an implicit formulation of the genFRI problem. To this end, we leverage a novel regularisation term which does not depend explicitly on the unknown annihilating filter yet enforces sufficient structure in the solution for stable recovery. The resulting optimisation problem is still non convex, but simpler since linear in the data and with less unknowns. We solve it by means of a provably convergent proximal gradient descent (PGD) method. Since the proximal step does not admit a simple closed-form expression, we propose an inexact PGD method, coined as Cadzow plug-and-play gradient descent (CPGD). The latter approximates the proximal steps by means of Cadzow denoising, a well-known denoising algorithm in FRI. We provide local fixed-point convergence guarantees for CPGD. Through extensive numerical simulations, we demonstrate the superiority of CPGD against the state-of-the-art in the case of non uniform time samples.Comment: 16 pages, 8 figure

Applied Harmonic Analysis and Data Processing

Massive data sets have their own architecture. Each data source has an inherent structure, which we should attempt to detect in order to utilize it for applications, such as denoising, clustering, anomaly detection, knowledge extraction, or classification. Harmonic analysis revolves around creating new structures for decomposition, rearrangement and reconstruction of operators and functions—in other words inventing and exploring new architectures for information and inference. Two previous very successful workshops on applied harmonic analysis and sparse approximation have taken place in 2012 and in 2015. This workshop was the an evolution and continuation of these workshops and intended to bring together world leading experts in applied harmonic analysis, data analysis, optimization, statistics, and machine learning to report on recent developments, and to foster new developments and collaborations