Extension of SBL Algorithms for the Recovery of Block Sparse Signals with Intra-Block Correlation

We examine the recovery of block sparse signals and extend the framework in two important directions; one by exploiting signals' intra-block correlation and the other by generalizing signals' block structure. We propose two families of algorithms based on the framework of block sparse Bayesian learning (BSBL). One family, directly derived from the BSBL framework, requires knowledge of the block structure. Another family, derived from an expanded BSBL framework, is based on a weaker assumption on the block structure, and can be used when the block structure is completely unknown. Using these algorithms we show that exploiting intra-block correlation is very helpful in improving recovery performance. These algorithms also shed light on how to modify existing algorithms or design new ones to exploit such correlation and improve performance.Comment: Matlab codes can be downloaded at: https://sites.google.com/site/researchbyzhang/bsbl, or http://dsp.ucsd.edu/~zhilin/BSBL.htm

Sparse signal recovery exploiting spatiotemporal correlation

Sparse signal recovery algorithms have significant impact on many fields. The core of these algorithms is to find a solution to an underdetermined inverse system of equations, where the solution is expected to be sparse or approximately sparse. However, most algorithms ignored correlation among nonzero entries of a solution, which is often encountered in a practical problem. Thus, it is unclear what role the correlation plays in signal recovery. This work aims to design algorithms which can exploit a variety of correlation structures in solutions and reveal the impact of these correlation structures on algorithms' recovery performance. First, a block sparse Bayesian learning (BSBL) framework is proposed. Based on it, a number of sparse Bayesian learning (SBL) algorithms are derived to exploit intra-block correlation in a block sparse model, temporal correlation in a multiple measurement vector model, spatiotemporal correlation in a spatiotemporal sparse model, and local temporal correlation in a time-varying sparse model. Several optimization approaches are employed in the algorithm development, such as the expectation-maximization method, the bound-optimization method, and a fixed-point method. Experimental results show that these algorithms have superior performance. With these algorithms, we find that different correlation structures affect the quality of estimated solutions to different degrees. However, if these correlation structures are present and exploited, algorithms' performance can be largely improved. Inspired by this, we connect these algorithms to Group-Lasso type algorithms and iterative reweighted [ell]₁ and [ell]₂ algorithms, and suggest strategies to modify them to exploit the correlation structures for better performance. The derived algorithms have been used with considerable success in various challenging applications such as wireless telemonitoring of raw physiological signals and prediction of patients' cognitive levels from their neuroimaging measures. In the former application, where raw physiological signals are neither sparse in the time domain nor sparse enough in transformed domains, the derived algorithms are the only algorithms so far that achieved satisfactory results. In the latter application, the derived algorithms achieved the highest prediction accuracy on common datasets, compared to published results around 201

Sparse signal recovery exploiting spatiotemporal correlation

Sparse signal recovery algorithms have significant impact on many fields. The core of these algorithms is to find a solution to an underdetermined inverse system of equations, where the solution is expected to be sparse or approximately sparse. However, most algorithms ignored correlation among nonzero entries of a solution, which is often encountered in a practical problem. Thus, it is unclear what role the correlation plays in signal recovery. This work aims to design algorithms which can exploit a variety of correlation structures in solutions and reveal the impact of these correlation structures on algorithms' recovery performance. First, a block sparse Bayesian learning (BSBL) framework is proposed. Based on it, a number of sparse Bayesian learning (SBL) algorithms are derived to exploit intra-block correlation in a block sparse model, temporal correlation in a multiple measurement vector model, spatiotemporal correlation in a spatiotemporal sparse model, and local temporal correlation in a time-varying sparse model. Several optimization approaches are employed in the algorithm development, such as the expectation-maximization method, the bound-optimization method, and a fixed-point method. Experimental results show that these algorithms have superior performance. With these algorithms, we find that different correlation structures affect the quality of estimated solutions to different degrees. However, if these correlation structures are present and exploited, algorithms' performance can be largely improved. Inspired by this, we connect these algorithms to Group-Lasso type algorithms and iterative reweighted [ell]₁ and [ell]₂ algorithms, and suggest strategies to modify them to exploit the correlation structures for better performance. The derived algorithms have been used with considerable success in various challenging applications such as wireless telemonitoring of raw physiological signals and prediction of patients' cognitive levels from their neuroimaging measures. In the former application, where raw physiological signals are neither sparse in the time domain nor sparse enough in transformed domains, the derived algorithms are the only algorithms so far that achieved satisfactory results. In the latter application, the derived algorithms achieved the highest prediction accuracy on common datasets, compared to published results around 201

Structure and Causality in Understanding Complex Systems

A central goal of science and engineering is to understand the causal structure of complex computational, physical, and social systems. Inferring this causal structure without performing experiments, however, is often extremely challenging. This thesis develops new mathematical approaches for exploiting the structure underlying many types of data to reveal insights about the causal relationships governing complex systems. The work consists of four aims, each of which leverages structure and causal modeling to understand a different type of system. In the first aim, we develop an algorithm based on the sparse Bayesian learning (SBL) framework for exploiting sparse and temporal structure in order to more efficiently collect data from time-varying high-dimensional systems. In the second aim, we develop a framework for explaining the operation of black-box machine learning classifiers using a causal model of how the data and classifier output are generated. In the third aim, we analyze a class of algorithms that use low-dimensional structure to infer causal interactions in coupled dynamical systems. In the final aim, we use surveys of the public and AI practitioners to model attitudes toward artificial intelligence adoption and governance, and employ the model to answer policy-relevant questions about AI governance.Ph.D