30 research outputs found
The Velocity of the Propagating Wave for General Coupled Scalar Systems
We consider spatially coupled systems governed by a set of scalar density
evolution equations. Such equations track the behavior of message-passing
algorithms used, for example, in coding, sparse sensing, or
constraint-satisfaction problems. Assuming that the "profile" describing the
average state of the algorithm exhibits a solitonic wave-like behavior after
initial transient iterations, we derive a formula for the propagation velocity
of the wave. We illustrate the formula with two applications, namely
Generalized LDPC codes and compressive sensing.Comment: 5 pages, 5 figures, submitted to the Information Theory Workshop
(ITW) 2016 in Cambridge, U
Lossy Source Coding via Spatially Coupled LDGM Ensembles
We study a new encoding scheme for lossy source compression based on
spatially coupled low-density generator-matrix codes. We develop a
belief-propagation guided-decimation algorithm, and show that this algorithm
allows to approach the optimal distortion of spatially coupled ensembles.
Moreover, using the survey propagation formalism, we also observe that the
optimal distortions of the spatially coupled and individual code ensembles are
the same. Since regular low-density generator-matrix codes are known to achieve
the Shannon rate-distortion bound under optimal encoding as the degrees grow,
our results suggest that spatial coupling can be used to reach the
rate-distortion bound, under a {\it low complexity} belief-propagation
guided-decimation algorithm.
This problem is analogous to the MAX-XORSAT problem in computer science.Comment: Submitted to ISIT 201
Spatially-Coupled Random Access on Graphs
In this paper we investigate the effect of spatial coupling applied to the
recently-proposed coded slotted ALOHA (CSA) random access protocol. Thanks to
the bridge between the graphical model describing the iterative interference
cancelation process of CSA over the random access frame and the erasure
recovery process of low-density parity-check (LDPC) codes over the binary
erasure channel (BEC), we propose an access protocol which is inspired by the
convolutional LDPC code construction. The proposed protocol exploits the
terminations of its graphical model to achieve the spatial coupling effect,
attaining performance close to the theoretical limits of CSA. As for the
convolutional LDPC code case, large iterative decoding thresholds are obtained
by simply increasing the density of the graph. We show that the threshold
saturation effect takes place by defining a suitable counterpart of the
maximum-a-posteriori decoding threshold of spatially-coupled LDPC code
ensembles. In the asymptotic setting, the proposed scheme allows sustaining a
traffic close to 1 [packets/slot].Comment: To be presented at IEEE ISIT 2012, Bosto
Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula
Factorizing low-rank matrices has many applications in machine learning and
statistics. For probabilistic models in the Bayes optimal setting, a general
expression for the mutual information has been proposed using heuristic
statistical physics computations, and proven in few specific cases. Here, we
show how to rigorously prove the conjectured formula for the symmetric rank-one
case. This allows to express the minimal mean-square-error and to characterize
the detectability phase transitions in a large set of estimation problems
ranging from community detection to sparse PCA. We also show that for a large
set of parameters, an iterative algorithm called approximate message-passing is
Bayes optimal. There exists, however, a gap between what currently known
polynomial algorithms can do and what is expected information theoretically.
Additionally, the proof technique has an interest of its own and exploits three
essential ingredients: the interpolation method introduced in statistical
physics by Guerra, the analysis of the approximate message-passing algorithm
and the theory of spatial coupling and threshold saturation in coding. Our
approach is generic and applicable to other open problems in statistical
estimation where heuristic statistical physics predictions are available
Statistical physics-based reconstruction in compressed sensing
Compressed sensing is triggering a major evolution in signal acquisition. It
consists in sampling a sparse signal at low rate and later using computational
power for its exact reconstruction, so that only the necessary information is
measured. Currently used reconstruction techniques are, however, limited to
acquisition rates larger than the true density of the signal. We design a new
procedure which is able to reconstruct exactly the signal with a number of
measurements that approaches the theoretical limit in the limit of large
systems. It is based on the joint use of three essential ingredients: a
probabilistic approach to signal reconstruction, a message-passing algorithm
adapted from belief propagation, and a careful design of the measurement matrix
inspired from the theory of crystal nucleation. The performance of this new
algorithm is analyzed by statistical physics methods. The obtained improvement
is confirmed by numerical studies of several cases.Comment: 20 pages, 8 figures, 3 tables. Related codes and data are available
at http://aspics.krzakala.or
Compressed sensing with sparse, structured matrices
In the context of the compressed sensing problem, we propose a new ensemble
of sparse random matrices which allow one (i) to acquire and compress a
{\rho}0-sparse signal of length N in a time linear in N and (ii) to perfectly
recover the original signal, compressed at a rate {\alpha}, by using a message
passing algorithm (Expectation Maximization Belief Propagation) that runs in a
time linear in N. In the large N limit, the scheme proposed here closely
approaches the theoretical bound {\rho}0 = {\alpha}, and so it is both optimal
and efficient (linear time complexity). More generally, we show that several
ensembles of dense random matrices can be converted into ensembles of sparse
random matrices, having the same thresholds, but much lower computational
complexity.Comment: 7 pages, 6 figure