Tyler's and Maronna's M-estimators: Non-Asymptotic Concentration Results

Tyler's and Maronna's M-estimators, as well as their regularized variants, are popular robust methods to estimate the scatter or covariance matrix of a multivariate distribution. In this work, we study the non-asymptotic behavior of these estimators, for data sampled from a distribution that satisfies one of the following properties: 1) independent sub-Gaussian entries, up to a linear transformation; 2) log-concave distributions; 3) distributions satisfying a convex concentration property. Our main contribution is the derivation of tight non-asymptotic concentration bounds of these M-estimators around a suitably scaled version of the data sample covariance matrix. Prior to our work, non-asymptotic bounds were derived only for Elliptical and Gaussian distributions. Our proof uses a variety of tools from non asymptotic random matrix theory and high dimensional geometry. Finally, we illustrate the utility of our results on two examples of practical interest: sparse covariance and sparse precision matrix estimation

Nonparametric consistency for maximum likelihood estimation and clustering based on mixtures of elliptically-symmetric distributions

The consistency of the maximum likelihood estimator for mixtures of elliptically-symmetric distributions for estimating its population version is shown, where the underlying distribution $P$ is nonparametric and does not necessarily belong to the class of mixtures on which the estimator is based. In a situation where $P$ is a mixture of well enough separated but nonparametric distributions it is shown that the components of the population version of the estimator correspond to the well separated components of $P$. This provides some theoretical justification for the use of such estimators for cluster analysis in case that $P$ has well separated subpopulations even if these subpopulations differ from what the mixture model assumes

Novel Computational Methods for State Space Filtering

The state-space formulation for time-dependent models has been long used invarious applications in science and engineering. While the classical Kalman filter(KF) provides optimal posterior estimation under linear Gaussian models, filteringin nonlinear and non-Gaussian environments remains challenging.Based on the Monte Carlo approximation, the classical particle filter (PF) can providemore precise estimation under nonlinear non-Gaussian models. However, it suffers fromparticle degeneracy. Drawing from optimal transport theory, the stochastic map filter(SMF) accommodates a solution to this problem, but its performance is influenced bythe limited flexibility of nonlinear map parameterisation. To account for these issues,a hybrid particle-stochastic map filter (PSMF) is first proposed in this thesis, wherethe two parts of the split likelihood are assimilated by the PF and SMF, respectively.Systematic resampling and smoothing are employed to alleviate the particle degeneracycaused by the PF. Furthermore, two PSMF variants based on the linear and nonlinearmaps (PSMF-L and PSMF-NL) are proposed, and their filtering performance is comparedwith various benchmark filters under different nonlinear non-Gaussian models.Although achieving accurate filtering results, the particle-based filters require expensive computations because of the large number of samples involved. Instead, robustKalman filters (RKFs) provide efficient solutions for the linear models with heavy-tailednoise, by adopting the recursive estimation framework of the KF. To exploit the stochasticcharacteristics of the noise, the use of heavy-tailed distributions which can fit variouspractical noises constitutes a viable solution. Hence, this thesis also introduces a novelRKF framework, RKF-SGαS, where the signal noise is assumed to be Gaussian and theheavy-tailed measurement noise is modelled by the sub-Gaussian α-stable (SGαS) distribution. The corresponding joint posterior distribution of the state vector and auxiliaryrandom variables is estimated by the variational Bayesian (VB) approach. Four differentminimum mean square error (MMSE) estimators of the scale function are presented.Besides, the RKF-SGαS is compared with the state-of-the-art RKFs under three kinds ofheavy-tailed measurement noises, and the simulation results demonstrate its estimationaccuracy and efficiency.One notable limitation of the proposed RKF-SGαS is its reliance on precise modelparameters, and substantial model errors can potentially impede its filtering performance. Therefore, this thesis also introduces a data-driven RKF method, referred to asRKFnet, which combines the conventional RKF framework with a deep learning technique. An unsupervised scheduled sampling technique (USS) is proposed to improve theistability of the training process. Furthermore, the advantages of the proposed RKFnetare quantified with respect to various traditional RKFs

Eigenvalues and constraints in mixture modeling: geometric and computational issues

This paper presents a review about the usage of eigenvalues restrictions for constrained parameter estimation in mixtures of elliptical distributions according to the likelihood approach. These restrictions serve a twofold purpose: to avoid convergence to degenerate solutions and to reduce the onset of non interesting (spurious) maximizers, related to complex likelihood surfaces. The paper shows how the constraints may play a key role in the theory of Euclidean data clustering. The aim here is to provide a reasoned review of the constraints and their applications, along the contributions of many authors, spanning the literature of the last thirty years.Spanish Ministerio de Economía y Competitividad (grant MTM2017-86061-C2-1-P)Junta de Castilla y León - Fondo Europeo de Desarrollo Regional (grant VA005P17 and VA002G18

BayesWave: Bayesian Inference for Gravitational Wave Bursts and Instrument Glitches

A central challenge in Gravitational Wave Astronomy is identifying weak signals in the presence of non-stationary and non-Gaussian noise. The separation of gravitational wave signals from noise requires good models for both. When accurate signal models are available, such as for binary Neutron star systems, it is possible to make robust detection statements even when the noise is poorly understood. In contrast, searches for "un-modeled" transient signals are strongly impacted by the methods used to characterize the noise. Here we take a Bayesian approach and introduce a multi-component, variable dimension, parameterized noise model that explicitly accounts for non-stationarity and non-Gaussianity in data from interferometric gravitational wave detectors. Instrumental transients (glitches) and burst sources of gravitational waves are modeled using a Morlet-Gabor continuous wavelet frame. The number and placement of the wavelets is determined by a trans-dimensional Reversible Jump Markov Chain Monte Carlo algorithm. The Gaussian component of the noise and sharp line features in the noise spectrum are modeled using the BayesLine algorithm, which operates in concert with the wavelet model.Comment: 36 pages, 15 figures, Version accepted by Class. Quant. Gra

Cleaning large correlation matrices: tools from random matrix theory

This review covers recent results concerning the estimation of large covariance matrices using tools from Random Matrix Theory (RMT). We introduce several RMT methods and analytical techniques, such as the Replica formalism and Free Probability, with an emphasis on the Marchenko-Pastur equation that provides information on the resolvent of multiplicatively corrupted noisy matrices. Special care is devoted to the statistics of the eigenvectors of the empirical correlation matrix, which turn out to be crucial for many applications. We show in particular how these results can be used to build consistent "Rotationally Invariant" estimators (RIE) for large correlation matrices when there is no prior on the structure of the underlying process. The last part of this review is dedicated to some real-world applications within financial markets as a case in point. We establish empirically the efficacy of the RIE framework, which is found to be superior in this case to all previously proposed methods. The case of additively (rather than multiplicatively) corrupted noisy matrices is also dealt with in a special Appendix. Several open problems and interesting technical developments are discussed throughout the paper.Comment: 165 pages, article submitted to Physics Report

Gravitational Lensing

Gravitational lensing has developed into one of the most powerful tools for the analysis of the dark universe. This review summarises the theory of gravitational lensing, its main current applications and representative results achieved so far. It has two parts. In the first, starting from the equation of geodesic deviation, the equations of thin and extended gravitational lensing are derived. In the second, gravitational lensing by stars and planets, galaxies, galaxy clusters and large-scale structures is discussed and summarised.Comment: Invited review article to appear in Classical and Quantum Gravity, 85 pages, 15 figure