Low-Complexity Iterative Algorithms for (Discrete) Compressed Sensing

We consider iterative (`turbo') algorithms for compressed sensing. First, a unified exposition of the different approaches available in the literature is given, thereby enlightening the general principles and main differences. In particular we discuss i) the estimation step (matched filter vs. optimum MMSE estimator), ii) the unbiasing operation (implicitly or explicitly done and equivalent to the calculation of extrinsic information), and iii) thresholding vs. the calculation of soft values. Based on these insights we propose a low-complexity but well-performing variant utilizing a Krylov space approximation of the optimum linear MMSE estimator. The derivations are valid for any probability density of the signal vector. However, numerical results are shown for the discrete case. The novel algorithms shows very good performance and even slightly faster convergence compared to approximative message passing

Hardware Implementation of Compressed Sensing based Low Complex Video Encoder

This paper presents a memory efficient VLSI architecture of low complex video encoder using three dimensional (3-D) wavelet and Compressed Sensing (CS) is proposed for space and low power video applications. Majority of the conventional video coding schemes are based on hybrid model, which requires complex operations like transform coding (DCT), motion estimation and deblocking filter at the encoder. Complexity of the proposed encoder is reduced by replacing those complex operations by 3-D DWT and CS at the encoder. The proposed architecture uses 3-D DWT to enable the scalability with levels of wavelet decomposition and also to exploit the spatial and the temporal redundancies. CS provides the good error resilience and coding efficiency. At the first stage of the proposed architecture for encoder, 3-D DWT has been applied (Lifting based 2-D DWT in spatial domain and Haar wavelet in temporal domain) on each frame of the group of frames (GOF), and in the second stage CS module exploits the sparsity of the wavelet coefficients. Small set of linear measurements are extracted by projecting the sparse 3-D wavelet coefficients onto random Bernoulli matrix at the encoder. Compared with the best existing 3-D DWT architectures, the proposed architecture for 3-D DWT requires less memory and provide high throughput. For an N?N image, the proposed 3-D DWT architecture consumes a total of only 2?(3N +40P) words of on-chip memory for the one level of decomposition. The proposed architecture for an encoder is first of its kind and to the best of my knowledge, no architecture is noted for comparison. The proposed VLSI architecture of the encoder has been synthesized on 90-nm CMOS process technology and results show that it consumes 90.08 mW power and occupies an area equivalent to 416.799 K equivalent gate at frequency of 158 MHz.Comment: Submitted in IEEE transactions on VLS

A Unified Approach to Sparse Signal Processing

A unified view of sparse signal processing is presented in tutorial form by bringing together various fields. For each of these fields, various algorithms and techniques, which have been developed to leverage sparsity, are described succinctly. The common benefits of significant reduction in sampling rate and processing manipulations are revealed. The key applications of sparse signal processing are sampling, coding, spectral estimation, array processing, component analysis, and multipath channel estimation. In terms of reconstruction algorithms, linkages are made with random sampling, compressed sensing and rate of innovation. The redundancy introduced by channel coding in finite/real Galois fields is then related to sampling with similar reconstruction algorithms. The methods of Prony, Pisarenko, and MUSIC are next discussed for sparse frequency domain representations. Specifically, the relations of the approach of Prony to an annihilating filter and Error Locator Polynomials in coding are emphasized; the Pisarenko and MUSIC methods are further improvements of the Prony method. Such spectral estimation methods is then related to multi-source location and DOA estimation in array processing. The notions of sparse array beamforming and sparse sensor networks are also introduced. Sparsity in unobservable source signals is also shown to facilitate source separation in SCA; the algorithms developed in this area are also widely used in compressed sensing. Finally, the multipath channel estimation problem is shown to have a sparse formulation; algorithms similar to sampling and coding are used to estimate OFDM channels.Comment: 43 pages, 40 figures, 15 table

VLSI Friendly Framework for Scalable Video Coding based on Compressed Sensing

This paper presents a new VLSI friendly framework for scalable video coding based on Compressed Sensing (CS). It achieves scalability through 3-Dimensional Discrete Wavelet Transform (3-D DWT) and better compression ratio by exploiting the inherent sparsity of the high-frequency wavelet sub-bands through CS. By using 3-D DWT and a proposed adaptive measurement scheme called AMS at the encoder, one can succeed in improving the compression ratio and reducing the complexity of the decoder. The proposed video codec uses only 7% of the total number of multipliers needed in a conventional CS-based video coding system. A codebook of Bernoulli matrices with different sizes corresponding to the predefined sparsity levels is maintained at both the encoder and the decoder. Based on the calculated l0-norm of the input vector, one of the sixteen possible Bernoulli matrices will be selected for taking the CS measurements and its index will be transmitted along with the measurements. Based on this index, the corresponding Bernoulli matrix has been used in CS reconstruction algorithm to get back the high-frequency wavelet sub-bands at the decoder. At the decoder, a new Enhanced Approximate Message Passing (EAMP) algorithm has been proposed to reconstruct the wavelet coefficients and apply the inverse wavelet transform for restoring back the video frames. Simulation results have established the superiority of the proposed framework over the existing schemes and have increased its suitability for VLSI implementation. Moreover, the coded video is found to be scalable with an increase in a number of levels of wavelet decomposition

Projected Wirtinger Gradient Descent for Low-Rank Hankel Matrix Completion in Spectral Compressed Sensing

This paper considers reconstructing a spectrally sparse signal from a small number of randomly observed time-domain samples. The signal of interest is a linear combination of complex sinusoids at $R$ distinct frequencies. The frequencies can assume any continuous values in the normalized frequency domain $[0,1)$. After converting the spectrally sparse signal recovery into a low rank structured matrix completion problem, we propose an efficient feasible point approach, named projected Wirtinger gradient descent (PWGD) algorithm, to efficiently solve this structured matrix completion problem. We further accelerate our proposed algorithm by a scheme inspired by FISTA. We give the convergence analysis of our proposed algorithms. Extensive numerical experiments are provided to illustrate the efficiency of our proposed algorithm. Different from earlier approaches, our algorithm can solve problems of very large dimensions very efficiently.Comment: 12 page

Reconstruction of Sub-Nyquist Random Sampling for Sparse and Multi-Band Signals

As technology grows, higher frequency signals are required to be processed in various applications. In order to digitize such signals, conventional analog to digital convertors are facing implementation challenges due to the higher sampling rates. Hence, lower sampling rates (i.e., sub-Nyquist) are considered to be cost efficient. A well-known approach is to consider sparse signals that have fewer nonzero frequency components compared to the highest frequency component. For the prior knowledge of the sparse positions, well-established methods already exist. However, there are applications where such information is not available. For such cases, a number of approaches have recently been proposed. In this paper, we propose several random sampling recovery algorithms which do not require any anti-aliasing filter. Moreover, we offer certain conditions under which these recovery techniques converge to the signal. Finally, we also confirm the performance of the above methods through extensive simulations

Fast Compressed Sensing SAR Imaging based on Approximated Observation

In recent years, compressed sensing (CS) has been applied in the field of synthetic aperture radar (SAR) imaging and shows great potential. The existing models are, however, based on application of the sensing matrix acquired by the exact observation functions. As a result, the corresponding reconstruction algorithms are much more time consuming than traditional matched filter (MF) based focusing methods, especially in high resolution and wide swath systems. In this paper, we formulate a new CS-SAR imaging model based on the use of the approximated SAR observation deducted from the inverse of focusing procedures. We incorporate CS and MF within an sparse regularization framework that is then solved by a fast iterative thresholding algorithm. The proposed model forms a new CS-SAR imaging method that can be applied to high-quality and high-resolution imaging under sub-Nyquist rate sampling, while saving the computational cost substantially both in time and memory. Simulations and real SAR data applications support that the proposed method can perform SAR imaging effectively and efficiently under Nyquist rate, especially for large scale applications.Comment: Submitted To IEEE-JSTA

Compressed Sensing for Wireless Communications : Useful Tips and Tricks

As a paradigm to recover the sparse signal from a small set of linear measurements, compressed sensing (CS) has stimulated a great deal of interest in recent years. In order to apply the CS techniques to wireless communication systems, there are a number of things to know and also several issues to be considered. However, it is not easy to come up with simple and easy answers to the issues raised while carrying out research on CS. The main purpose of this paper is to provide essential knowledge and useful tips that wireless communication researchers need to know when designing CS-based wireless systems. First, we present an overview of the CS technique, including basic setup, sparse recovery algorithm, and performance guarantee. Then, we describe three distinct subproblems of CS, viz., sparse estimation, support identification, and sparse detection, with various wireless communication applications. We also address main issues encountered in the design of CS-based wireless communication systems. These include potentials and limitations of CS techniques, useful tips that one should be aware of, subtle points that one should pay attention to, and some prior knowledge to achieve better performance. Our hope is that this article will be a useful guide for wireless communication researchers and even non-experts to grasp the gist of CS techniques

Stronger L2/L2 Compressed Sensing; Without Iterating

We consider the extensively studied problem of $\ell_2/\ell_2$ compressed sensing. The main contribution of our work is an improvement over [Gilbert, Li, Porat and Strauss, STOC 2010] with faster decoding time and significantly smaller column sparsity, answering two open questions of the aforementioned work. Previous work on sublinear-time compressed sensing employed an iterative procedure, recovering the heavy coordinates in phases. We completely depart from that framework, and give the first sublinear-time $\ell_2/\ell_2$ scheme which achieves the optimal number of measurements without iterating; this new approach is the key step to our progress. Towards that, we satisfy the $\ell_2/\ell_2$ guarantee by exploiting the heaviness of coordinates in a way that was not exploited in previous work. Via our techniques we obtain improved results for various sparse recovery tasks, and indicate possible further applications to problems in the field, to which the aforementioned iterative procedure creates significant obstructions

Unveiling Bias Compensation in Turbo-Based Algorithms for (Discrete) Compressed Sensing

In Compressed Sensing, a real-valued sparse vector has to be recovered from an underdetermined system of linear equations. In many applications, however, the elements of the sparse vector are drawn from a finite set. Adapted algorithms incorporating this additional knowledge are required for the discrete-valued setup. In this paper, turbo-based algorithms for both cases are elucidated and analyzed from a communications engineering perspective, leading to a deeper understanding of the algorithm. In particular, we gain the intriguing insight that the calculation of extrinsic values is equal to the unbiasing of a biased estimate and present an improved algorithm