5 research outputs found
Multi Terminal Probabilistic Compressed Sensing
In this paper, the `Approximate Message Passing' (AMP) algorithm, initially
developed for compressed sensing of signals under i.i.d. Gaussian measurement
matrices, has been extended to a multi-terminal setting (MAMP algorithm). It
has been shown that similar to its single terminal counterpart, the behavior of
MAMP algorithm is fully characterized by a `State Evolution' (SE) equation for
large block-lengths. This equation has been used to obtain the rate-distortion
curve of a multi-terminal memoryless source. It is observed that by spatially
coupling the measurement matrices, the rate-distortion curve of MAMP algorithm
undergoes a phase transition, where the measurement rate region corresponding
to a low distortion (approximately zero distortion) regime is fully
characterized by the joint and conditional Renyi information dimension (RID) of
the multi-terminal source. This measurement rate region is very similar to the
rate region of the Slepian-Wolf distributed source coding problem where the RID
plays a role similar to the discrete entropy.
Simulations have been done to investigate the empirical behavior of MAMP
algorithm. It is observed that simulation results match very well with
predictions of SE equation for reasonably large block-lengths.Comment: 11 pages, 13 figures. arXiv admin note: text overlap with
arXiv:1112.0708 by other author
Polarization of the Renyi Information Dimension with Applications to Compressed Sensing
In this paper, we show that the Hadamard matrix acts as an extractor over the
reals of the Renyi information dimension (RID), in an analogous way to how it
acts as an extractor of the discrete entropy over finite fields. More
precisely, we prove that the RID of an i.i.d. sequence of mixture random
variables polarizes to the extremal values of 0 and 1 (corresponding to
discrete and continuous distributions) when transformed by a Hadamard matrix.
Further, we prove that the polarization pattern of the RID admits a closed form
expression and follows exactly the Binary Erasure Channel (BEC) polarization
pattern in the discrete setting. We also extend the results from the single- to
the multi-terminal setting, obtaining a Slepian-Wolf counterpart of the RID
polarization. We discuss applications of the RID polarization to Compressed
Sensing of i.i.d. sources. In particular, we use the RID polarization to
construct a family of deterministic -valued sensing matrices for
Compressed Sensing. We run numerical simulations to compare the performance of
the resulting matrices with that of random Gaussian and random Hadamard
matrices. The results indicate that the proposed matrices afford competitive
performances while being explicitly constructed.Comment: 12 pages, 2 figure
Achieving the Fundamental Limit of Lossless Analog Compression via Polarization
In this paper, we study the lossless analog compression for i.i.d.
nonsingular signals via the polarization-based framework. We prove that for
nonsingular source, the error probability of maximum a posteriori (MAP)
estimation polarizes under the Hadamard transform, which extends the
polarization phenomenon to analog domain. Building on this insight, we propose
partial Hadamard compression and develop the corresponding analog successive
cancellation (SC) decoder. The proposed scheme consists of deterministic
measurement matrices and non-iterative reconstruction algorithm, providing
benefits in both space and computational complexity. Using the polarization of
error probability, we prove that our approach achieves the
information-theoretical limit for lossless analog compression developed by Wu
and Verdu.Comment: 48 pages, 5 figures. This work was presented in part at the 2023 IEEE
Global Communications Conferenc
Compressed Sensing of Memoryless Sources:A Deterministic Hadamard Construction
Compressed sensing is a new trend in signal processing for efficient sampling and signal acquisition. The idea is that most real-world signals have a sparse representation in an appropriate basis and this can be exploited to capture the sparse signal by taking only a few linear projections. The recovery is possible by running appropriate low-complexity algorithms that exploit the sparsity (prior information) to reconstruct the signal from the linear projections (posterior information). The main benefit is that the required number of measurements is much smaller than the dimension of the signal. This results in a huge gain in sensor cost (in measurement devices) or a dramatic saving in data acquisition time. However, some difficulties naturally arise in applying the compressed sensing to real-world applications such as robustness issues in taking the linear projections and computational complexity of the recovery algorithm. In this thesis, we design structured matrices for compressed sensing. In particular, we claim that some of the practical difficulties can be reasonably solved by imposing some structure on the measurement matrices. The thesis evolves around the Hadamard matrices which are -valued matrices with many applications in signal processing, coding, optics and mathematics. As the title of the thesis implies, there are two main ingredients to this thesis. First, we use a memoryless assumption for the source, i.e., we assume that the nonzero components of the sparse signal are independently generated by a given probability distribution and their position is completely random. This allows us to use tools from probability, information theory and coding theory to rigorously assess the achievable performance. Second, using the mathematical properties of the Hadamard matrices, we design measurement matrices by selecting specific rows of a Hadamard matrix according to a deterministic criterion. We call the resulting matrices ``partial Hadamard matrices''. We design partial Hadamard matrices for three signal models: memoryless discrete signals and sparse signals with linear or sub-linear sparsity. A signal has linear sparsity if the number of its nonzero components is proportional to , the dimension of signal, whereas it has a sub-linear sparsity if scales like for some . We develop tools to rigorously analyze the performance of the proposed constructions by borrowing ideas from information theory and coding theory. We also extend our construction to distributed (multi-terminal) signals. Distributed compressed sensing is a ubiquitous problem in distributed data acquisition systems such as ad-hoc sensor networks. From both a theoretical and an engineering point of view, it is important to know how many measurement per dimension are necessary from different terminals in order to have a reliable estimate of the distributed data. We theoretically analyze this problem for a very simple setup where the components of the distributed signal are generated by a joint probability distribution which captures the spatial correlation among different terminals. We give an information-theoretic characterization of the measurements-rate region that results in a negligible recovery distortion. We also propose a low-complexity distributed message passing algorithm to achieve the theoretical limits