Dual-Space Analysis of the Sparse Linear Model

Sparse linear (or generalized linear) models combine a standard likelihood function with a sparse prior on the unknown coefficients. These priors can conveniently be expressed as a maximization over zero-mean Gaussians with different variance hyperparameters. Standard MAP estimation (Type I) involves maximizing over both the hyperparameters and coefficients, while an empirical Bayesian alternative (Type II) first marginalizes the coefficients and then maximizes over the hyperparameters, leading to a tractable posterior approximation. The underlying cost functions can be related via a dual-space framework from Wipf et al. (2011), which allows both the Type I or Type II objectives to be expressed in either coefficient or hyperparmeter space. This perspective is useful because some analyses or extensions are more conducive to development in one space or the other. Herein we consider the estimation of a trade-off parameter balancing sparsity and data fit. As this parameter is effectively a variance, natural estimators exist by assessing the problem in hyperparameter (variance) space, transitioning natural ideas from Type II to solve what is much less intuitive for Type I. In contrast, for analyses of update rules and sparsity properties of local and global solutions, as well as extensions to more general likelihood models, we can leverage coefficient-space techniques developed for Type I and apply them to Type II. For example, this allows us to prove that Type II-inspired techniques can be successful recovering sparse coefficients when unfavorable restricted isometry properties (RIP) lead to failure of popular L1 reconstructions. It also facilitates the analysis of Type II when non-Gaussian likelihood models lead to intractable integrations.Comment: 9 pages, 2 figures, submission to NIPS 201

Sparse regression algorithm for activity estimation in γ\gamma spectrometry

We consider the counting rate estimation of an unknown radioactive source, which emits photons at times modeled by an homogeneous Poisson process. A spectrometer converts the energy of incoming photons into electrical pulses, whose number provides a rough estimate of the intensity of the Poisson process. When the activity of the source is high, a physical phenomenon known as pileup effect distorts direct measurements, resulting in a significant bias to the standard estimators of the source activities used so far in the field. We show in this paper that the problem of counting rate estimation can be interpreted as a sparse regression problem. We suggest a post-processed, non-negative, version of the Least Absolute Shrinkage and Selection Operator (LASSO) to estimate the photon arrival times. The main difficulty in this problem is that no theoretical conditions can guarantee consistency in sparsity of LASSO, because the dictionary is not ideal and the signal is sampled. We therefore derive theoretical conditions and bounds which illustrate that the proposed method can none the less provide a good, close to the best attainable, estimate of the counting rate activity. The good performances of the proposed approach are studied on simulations and real datasets

Reconstruction par acquisition compressée en imagerie ultrasonore médicale 3D et Doppler

This thesis is dedicated to the application of the novel compressed sensing theory to the acquisition and reconstruction of 3D US images and Doppler signals. In 3D US imaging, one of the major difficulties concerns the number of RF lines that has to be acquired to cover the complete volume. The acquisition of each line takes an incompressible time due to the finite velocity of the ultrasound wave. One possible solution for increasing the frame rate consists in reducing the acquisition time by skipping some RF lines. The reconstruction of the missing information in post processing is then a typical application of compressed sensing. Another excellent candidate for this theory is the Doppler duplex imaging that implies alternating two modes of emission, one for B-mode imaging and the other for flow estimation. Regarding 3D imaging, we propose a compressed sensing framework using learned overcomplete dictionaries. Such dictionaries allow for much sparser representations of the signals since they are optimized for a particular class of images such as US images.We also focus on the measurement sensing setup and propose a line-wise sampling of entire RF lines which allows to decrease the amount of data and is feasible in a relatively simple setting of the 3D US equipment. The algorithm was validated on 3D simulated and experimental data. For the Doppler application, we proposed a CS based framework for randomly interleaving Doppler and US emissions. The proposed method reconstructs the Doppler signal using a block sparse Bayesian learning algorithm that exploits the correlation structure within a signal and has the ability of recovering partially sparse signals as long as they are correlated. This method is validated on simulated and experimental Doppler data.DL’objectif de cette thèse est le développement de techniques adaptées à l’application de la théorie de l’acquisition compressée en imagerie ultrasonore 3D et Doppler. En imagerie ultrasonore 3D une des principales difficultés concerne le temps d’acquisition très long lié au nombre de lignes RF à acquérir pour couvrir l’ensemble du volume. Afin d’augmenter la cadence d’imagerie une solution possible consiste à choisir aléatoirement des lignes RF qui ne seront pas acquises. La reconstruction des données manquantes est une application typique de l’acquisition compressée. Une autre application d’intérêt correspond aux acquisitions Doppler duplex où des stratégies d’entrelacement des acquisitions sont nécessaires et conduisent donc à une réduction de la quantité de données disponibles. Dans ce contexte, nous avons réalisé de nouveaux développements permettant l’application de l’acquisition compressée à ces deux modalités d’acquisition ultrasonore. Dans un premier temps, nous avons proposé d’utiliser des dictionnaires redondants construits à partir des signaux d’intérêt pour la reconstruction d’images 3D ultrasonores. Une attention particulière a aussi été apportée à la configuration du système d’acquisition et nous avons choisi de nous concentrer sur un échantillonnage des lignes RF entières, réalisable en pratique de façon relativement simple. Cette méthode est validée sur données 3D simulées et expérimentales. Dans un deuxième temps, nous proposons une méthode qui permet d’alterner de manière aléatoire les émissions Doppler et les émissions destinées à l’imagerie mode-B. La technique est basée sur une approche bayésienne qui exploite la corrélation et la parcimonie des blocs du signal. L’algorithme est validé sur des données Doppler simulées et expérimentales

Structure and Causality in Understanding Complex Systems

A central goal of science and engineering is to understand the causal structure of complex computational, physical, and social systems. Inferring this causal structure without performing experiments, however, is often extremely challenging. This thesis develops new mathematical approaches for exploiting the structure underlying many types of data to reveal insights about the causal relationships governing complex systems. The work consists of four aims, each of which leverages structure and causal modeling to understand a different type of system. In the first aim, we develop an algorithm based on the sparse Bayesian learning (SBL) framework for exploiting sparse and temporal structure in order to more efficiently collect data from time-varying high-dimensional systems. In the second aim, we develop a framework for explaining the operation of black-box machine learning classifiers using a causal model of how the data and classifier output are generated. In the third aim, we analyze a class of algorithms that use low-dimensional structure to infer causal interactions in coupled dynamical systems. In the final aim, we use surveys of the public and AI practitioners to model attitudes toward artificial intelligence adoption and governance, and employ the model to answer policy-relevant questions about AI governance.Ph.D