5 research outputs found
Lossless Linear Analog Compression
We establish the fundamental limits of lossless linear analog compression by
considering the recovery of random vectors
from the noiseless linear
measurements
with
measurement matrix . Specifically,
for a random vector of arbitrary
distribution we show that can be recovered with
zero error probability from
linear measurements,
where denotes the lower
modified Minkowski dimension and the infimum is over all sets
with . This achievability statement holds for Lebesgue almost all measurement
matrices . We then show that -rectifiable random vectors---a
stochastic generalization of -sparse vectors---can be recovered with zero
error probability from linear measurements. From classical compressed
sensing theory we would expect to be necessary for successful
recovery of . Surprisingly, certain classes of
-rectifiable random vectors can be recovered from fewer than
measurements. Imposing an additional regularity condition on the distribution
of -rectifiable random vectors , we do get the
expected converse result of measurements being necessary. The resulting
class of random vectors appears to be new and will be referred to as
-analytic random vectors
New Uniform Bounds for Almost Lossless Analog Compression
Wu and Verd\'u developed a theory of almost lossless analog compression,
where one imposes various regularity conditions on the compressor and the
decompressor with the input signal being modelled by a (typically
infinite-entropy) stationary stochastic process. In this work we consider all
stationary stochastic processes with trajectories in a prescribed set
of (bi)infinite sequences and find
uniform lower and upper bounds for certain compression rates in terms of metric
mean dimension and mean box dimension. An essential tool is the recent
Lindenstrauss-Tsukamoto variational principle expressing metric mean dimension
in terms of rate-distortion functions.Comment: This paper is going to be presented at 2019 IEEE International
Symposium on Information Theory. It is a short version of arXiv:1812.0045
Polarization of the Renyi Information Dimension with Applications to Compressed Sensing
In this paper, we show that the Hadamard matrix acts as an extractor over the
reals of the Renyi information dimension (RID), in an analogous way to how it
acts as an extractor of the discrete entropy over finite fields. More
precisely, we prove that the RID of an i.i.d. sequence of mixture random
variables polarizes to the extremal values of 0 and 1 (corresponding to
discrete and continuous distributions) when transformed by a Hadamard matrix.
Further, we prove that the polarization pattern of the RID admits a closed form
expression and follows exactly the Binary Erasure Channel (BEC) polarization
pattern in the discrete setting. We also extend the results from the single- to
the multi-terminal setting, obtaining a Slepian-Wolf counterpart of the RID
polarization. We discuss applications of the RID polarization to Compressed
Sensing of i.i.d. sources. In particular, we use the RID polarization to
construct a family of deterministic -valued sensing matrices for
Compressed Sensing. We run numerical simulations to compare the performance of
the resulting matrices with that of random Gaussian and random Hadamard
matrices. The results indicate that the proposed matrices afford competitive
performances while being explicitly constructed.Comment: 12 pages, 2 figure
Lossless Analog Compression
We establish the fundamental limits of lossless analog compression by
considering the recovery of arbitrary m-dimensional real random vectors x from
the noiseless linear measurements y=Ax with n x m measurement matrix A. Our
theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost
lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m
case, and considers zero error probability. Specifically, our achievability
result states that, for almost all A, the random vector x can be recovered with
zero error probability provided that n > K(x), where K(x) is given by the
infimum of the lower modified Minkowski dimension over all support sets U of x.
We then particularize this achievability result to the class of s-rectifiable
random vectors as introduced in Koliander et al. (2016); these are random
vectors of absolutely continuous distribution---with respect to the
s-dimensional Hausdorff measure---supported on countable unions of
s-dimensional differentiable submanifolds of the m-dimensional real coordinate
space. Countable unions of differentiable submanifolds include essentially all
signal models used in the compressed sensing literature. Specifically, we prove
that, for almost all A, s-rectifiable random vectors x can be recovered with
zero error probability from n>s linear measurements. This threshold is,
however, found not to be tight as exemplified by the construction of an
s-rectifiable random vector that can be recovered with zero error probability
from n<s linear measurements. This leads us to the introduction of the new
class of s-analytic random vectors, which admit a strong converse in the sense
of n greater than or equal to s being necessary for recovery with probability
of error smaller than one. The central conceptual tools in the development of
our theory are geometric measure theory and the theory of real analytic
functions
Achieving the Fundamental Limit of Lossless Analog Compression via Polarization
In this paper, we study the lossless analog compression for i.i.d.
nonsingular signals via the polarization-based framework. We prove that for
nonsingular source, the error probability of maximum a posteriori (MAP)
estimation polarizes under the Hadamard transform, which extends the
polarization phenomenon to analog domain. Building on this insight, we propose
partial Hadamard compression and develop the corresponding analog successive
cancellation (SC) decoder. The proposed scheme consists of deterministic
measurement matrices and non-iterative reconstruction algorithm, providing
benefits in both space and computational complexity. Using the polarization of
error probability, we prove that our approach achieves the
information-theoretical limit for lossless analog compression developed by Wu
and Verdu.Comment: 48 pages, 5 figures. This work was presented in part at the 2023 IEEE
Global Communications Conferenc