Asymptotic Achievability of the Cramér–Rao Bound For Noisy Compressive Sampling

We consider a model of the form ${bf y}={bf Ax}+{bf n}$, where ${bf x}inBBC^{M}$ is sparse with at most $L$ nonzero coefficients in unknown locations, ${bf y}inBBC^{N}$ is the observation vector, ${bf A}inBBC^{Ntimes M}$ is the measurement matrix and ${bf n}inBBC^{N}$ is the Gaussian noise. We develop a CramÉr–Rao bound on the mean squared estimation error of the nonzero elements of ${bf x}$, corresponding to the genie-aided estimator (GAE) which is provided with the locations of the nonzero elements of ${bf x}$. Intuitively, the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of ${bf x}$ is no less than that of the GAE. Assuming that $L/N$ is fixed, we establish the existence of an estimator that asymptotically achieves the CramÉr–Rao bound without any knowledge of the locations of the nonzero elements of ${bf x}$ as $Nrightarrowinfty$ , for ${bf A}$ a random Gaussian matrix whose elements are drawn i.i.d. according to ${cal N}(0,1)$ .Engineering and Applied Science

Obtaining sparse distributions in 2D inverse problems

The mathematics of inverse problems has relevance across numerous estimation problems in science and engineering. L1 regularization has attracted recent attention in reconstructing the system properties in the case of sparse inverse problems; i.e., when the true property sought is not adequately described by a continuous distribution, in particular in Compressed Sensing image reconstruction. In this work, we focus on the application of L1 regularization to a class of inverse problems; relaxation-relaxation, T1–T2, and diffusion-relaxation, D–T2, correlation experiments in NMR, which have found widespread applications in a number of areas including probing surface interactions in catalysis and characterizing fluid composition and pore structures in rocks. We introduce a robust algorithm for solving the L1 regularization problem and provide a guide to implementing it, including the choice of the amount of regularization used and the assignment of error estimates. We then show experimentally that L1 regularization has significant advantages over both the Non-Negative Least Squares (NNLS) algorithm and Tikhonov regularization. It is shown that the L1 regularization algorithm stably recovers a distribution at a signal to noise ratio < 20 and that it resolves relaxation time constants and diffusion coefficients differing by as little as 10%. The enhanced resolving capability is used to measure the inter and intra particle concentrations of a mixture of hexane and dodecane present within porous silica beads immersed within a bulk liquid phase; neither NNLS nor Tikhonov regularization are able to provide this resolution. This experimental study shows that the approach enables discrimination between different chemical species when direct spectroscopic discrimination is impossible, and hence measurement of chemical composition within porous media, such as catalysts or rocks, is possible while still being stable to high levels of noise.A.R. acknowledges Gates Trust Cambridge for financial support. A.J.S. and L.F.G. would like to acknowledge support from EPSRC (EP/N009304/1)

Signal sampling and processing in magnetic resonance applications

In this thesis, signal sampling and processing techniques are developed for magnetic resonance applications, to improve the estimation of magnetic resonance parameters and to reduce experimental acquisition times. Two processing techniques are developed for Nuclear Magnetic Resonance (NMR) relaxation and diffusion experiments. $L_1$ regularization is recommended for extracting parameter distributions which are a priori known to be composed of sparse features. $L_1$ regularization is shown to be stable at signal-to-noise ratios < 20 and capable of resolving relaxation time constants and diffusion coefficients which differ by as little as 10%, such as in relaxation and diffusion studies of hexane/dodecane in porous media. Modified Total Generalized Variation (MTGV) regularization is recommended for extracting parameter distributions for which there is no prior knowledge of whether they are composed of sparse or smooth features. MTGV regularization is shown to perform better than conventional processing techniques and $L_1$ regularization over a range of simulated distributions. A method for optimising sampling patterns for relaxation and diffusion experiments, based on the Cramér-Rao Lower Bound theory, is presented. The method is validated against pulsed field gradient NMR diffusion data of two experimental systems. In the first experimental system, the sampling pattern is optimised for the most accurate estimation of the lognormal distribution parameters of an emulsion droplet size distribution of toluene in water. In the second experimental system, the sampling pattern is optimised for the most accurate estimation of the bi-exponential model parameters of a binary mixture of methane/ethane adsorbed in a zeolite. The proposed method predicts an uncertainty in estimating the model parameters which is < 10% different from the uncertainty estimated from the experimental data sampled using the same sampling pattern. Signal sampling and processing techniques are subsequently combined to reduce experimental acquisition times, which opens opportunities for studying unsteady systems over a long acquisition time and investigating fast-changing phenomena. A 32-fold decrease in the experimental acquisition time is achieved in extracting 3D spatially resolved spin spin relaxation maps. This is expected to be useful in investigating porous media systems. Three-component velocity maps on a 2D image, acquired every 4 ms, are used to capture, for the first time, the hydrodynamics of a bubble burst event. The experimental data are used to validate the predictions of numerical works.Gates Cambridge Scholarshi

Diversité et traitements non-linéaires pour les récepteurs modernes

Depuis le doctorat, les travaux de recherche auxquels j'ai contribué ont porté essentiellement sur des problèmes d'estimation d'un signal d'intérêt noyé dans du bruit. Les domaines d'application visés sont majoritairement le radar, mais aussi le GNSS et l'imagerie ultrasonore. Bien que différents, ces domaines sont soumis à des tendances similaires qui caractérisent ou caractériseront certainement les récepteurs modernes. En effet, les enjeux applicatifs requièrent de repousser sans cesse les limites de performance des traitements : le radariste cherche à détecter des petites cibles dans des environnements de plus en plus difficiles ; en GNSS, des solutions de positionnement haute précision sont recherchées dans des milieux très contraints tels les canyons urbains ; en imagerie médicale, une qualité accrue des images est recherchée pour améliorer les diagnostics, pour ne citer que quelques exemples. Parmi les tendances qui permettront de repousser les performances des récepteurs modernes, deux sont particulièrement présentes dans les travaux conduits jusqu'ici : la diversité des signaux et les traitements non linéaires. Le document illustre ceci en se focalisant sur deux des thématiques de recherche conduites jusqu’ici, à savoir « Le traitement du signal pour des radars de détection à large bande instantanée » et « La poursuite robuste de la phase d'un signal GNSS multifréquence ». Pour conclure, les perspectives de recherche d’un point de vue méthodologique et applicatif sont discutées

Asymptotic achievability of the Cramér-Rao bound for noisy compressive sampling

We consider a model of the form =Ax + n, where x ε CM is sparse with at most L nonzero coefficients in unknown locations, y ε CN is the observation vector, A CN×M is the measurement matrix and n ε CN is the Gaussian noise. We develop a Cramér-Rao bound on the mean squared estimation error of the nonzero elements of x, corresponding to the genie-aided estimator (GAE) which is provided with the locations of the nonzero elements of x. Intuitively, the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of x is no less than that of the GAE. Assuming that L/N is fixed, we establish the existence of an estimator that asymptotically achieves the Cramér-Rao bound without any knowledge of the locations of the nonzero elements of x as N → infinite;, for Aa random Gaussian matrix whose elements are drawn i.i.d. according to N (0,1). © 2009 IEEE

Information Theory and Machine Learning

The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems

Errata to " Asymptotic Achievability of the Cramér -Rao Bound for Noisy Compressive Sampling "

International audienceGiven N noisy measurements denoted by y and an overcom-plete Gaussian dictionary, A, the authors in [1] establish the existence and the asymptotic statistical efficiency of an unbiased estimator unaware of the locations of the non-zero entries, collected in set I, in the deterministic L-sparse signal x. More precisely, there exists an estimator x(y, A) unaware of set I with a variance reaching the oracle-CRB (Cramér-Rao Bound) in the doubly asymptotic scenario, i.e., for N, L → ∞ and L/N → α ∈ (0, 1). As was noted in [2] the result remains true even though the proposed closed-form expression of the variance of the estimator x(y, A) is incorrect. In this note, we correct this expression by providing an explicit formula and discuss its practical usefulness. Finally, the new expression allows to correct the misleading comprehension of the sparse signal estimation performance suggested in [1]. I. MAIN RESULT OF [1] Let y be the N × 1 noisy measurement vector given by y = Ax + n where A is a non-stochastic N × M matrix with controlled growing dimensions according to limN,L→∞ L/N = α ∈ (0, 1). An entry of matrix A is generated as a single realization of an i.i.d. Normal distribution N (0, 1), x is a deterministic L-sparse vector on set I and n is a centered circular white Gaussian noise of variance σ 2. The definition of the oracle (doubly) asymptotic CRB is given hereafter. Definition 1.1: The oracle-CRB in the doubly asymptotic scenario is defined according to