5,430 research outputs found
Universal polar coding and sparse recovery
This paper investigates universal polar coding schemes. In particular, a
notion of ordering (called convolutional path) is introduced between
probability distributions to determine when a polar compression (or
communication) scheme designed for one distribution can also succeed for
another one. The original polar decoding algorithm is also generalized to an
algorithm allowing to learn information about the source distribution using the
idea of checkers. These tools are used to construct a universal compression
algorithm for binary sources, operating at the lowest achievable rate
(entropy), with low complexity and with guaranteed small error probability. In
a second part of the paper, the problem of sketching high dimensional discrete
signals which are sparse is approached via the polarization technique. It is
shown that the number of measurements required for perfect recovery is
competitive with the bound (with optimal constant for binary
signals), meanwhile affording a deterministic low complexity measurement
matrix
Polarization of the Renyi Information Dimension with Applications to Compressed Sensing
In this paper, we show that the Hadamard matrix acts as an extractor over the
reals of the Renyi information dimension (RID), in an analogous way to how it
acts as an extractor of the discrete entropy over finite fields. More
precisely, we prove that the RID of an i.i.d. sequence of mixture random
variables polarizes to the extremal values of 0 and 1 (corresponding to
discrete and continuous distributions) when transformed by a Hadamard matrix.
Further, we prove that the polarization pattern of the RID admits a closed form
expression and follows exactly the Binary Erasure Channel (BEC) polarization
pattern in the discrete setting. We also extend the results from the single- to
the multi-terminal setting, obtaining a Slepian-Wolf counterpart of the RID
polarization. We discuss applications of the RID polarization to Compressed
Sensing of i.i.d. sources. In particular, we use the RID polarization to
construct a family of deterministic -valued sensing matrices for
Compressed Sensing. We run numerical simulations to compare the performance of
the resulting matrices with that of random Gaussian and random Hadamard
matrices. The results indicate that the proposed matrices afford competitive
performances while being explicitly constructed.Comment: 12 pages, 2 figure
Computational Complexity versus Statistical Performance on Sparse Recovery Problems
We show that several classical quantities controlling compressed sensing
performance directly match classical parameters controlling algorithmic
complexity. We first describe linearly convergent restart schemes on
first-order methods solving a broad range of compressed sensing problems, where
sharpness at the optimum controls convergence speed. We show that for sparse
recovery problems, this sharpness can be written as a condition number, given
by the ratio between true signal sparsity and the largest signal size that can
be recovered by the observation matrix. In a similar vein, Renegar's condition
number is a data-driven complexity measure for convex programs, generalizing
classical condition numbers for linear systems. We show that for a broad class
of compressed sensing problems, the worst case value of this algorithmic
complexity measure taken over all signals matches the restricted singular value
of the observation matrix which controls robust recovery performance. Overall,
this means in both cases that, in compressed sensing problems, a single
parameter directly controls both computational complexity and recovery
performance. Numerical experiments illustrate these points using several
classical algorithms.Comment: Final version, to appear in information and Inferenc
Geometric approach to error correcting codes and reconstruction of signals
We develop an approach through geometric functional analysis to error
correcting codes and to reconstruction of signals from few linear measurements.
An error correcting code encodes an n-letter word x into an m-letter word y in
such a way that x can be decoded correctly when any r letters of y are
corrupted. We prove that most linear orthogonal transformations Q from R^n into
R^m form efficient and robust robust error correcting codes over reals. The
decoder (which corrects the corrupted components of y) is the metric projection
onto the range of Q in the L_1 norm. An equivalent problem arises in signal
processing: how to reconstruct a signal that belongs to a small class from few
linear measurements? We prove that for most sets of Gaussian measurements, all
signals of small support can be exactly reconstructed by the L_1 norm
minimization. This is a substantial improvement of recent results of Donoho and
of Candes and Tao. An equivalent problem in combinatorial geometry is the
existence of a polytope with fixed number of facets and maximal number of
lower-dimensional facets. We prove that most sections of the cube form such
polytopes.Comment: 17 pages, 3 figure
Universal Source Polarization and an Application to a Multi-User Problem
We propose a scheme that universally achieves the smallest possible
compression rate for a class of sources with side information, and develop an
application of this result for a joint source channel coding problem over a
broadcast channel.Comment: to be presented at Allerton 201
- âŠ