2,972 research outputs found
Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems
We consider chains of random constraint satisfaction models that are
spatially coupled across a finite window along the chain direction. We
investigate their phase diagram at zero temperature using the survey
propagation formalism and the interpolation method. We prove that the SAT-UNSAT
phase transition threshold of an infinite chain is identical to the one of the
individual standard model, and is therefore not affected by spatial coupling.
We compute the survey propagation complexity using population dynamics as well
as large degree approximations, and determine the survey propagation threshold.
We find that a clustering phase survives coupling. However, as one increases
the range of the coupling window, the survey propagation threshold increases
and saturates towards the phase transition threshold. We also briefly discuss
other aspects of the problem. Namely, the condensation threshold is not
affected by coupling, but the dynamic threshold displays saturation towards the
condensation one. All these features may provide a new avenue for obtaining
better provable algorithmic lower bounds on phase transition thresholds of the
individual standard model
Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems
We consider chains of random constraint satisfaction models that are spatially coupled across a finite window along the chain direction. We investigate their phase diagram at zero temperature using the survey propagation formalism and the interpolation method. We prove that the SAT-UNSAT phase transition threshold of an infinite chain is identical to the one of the individual standard model, and is therefore not affected by spatial coupling. We compute the survey propagation complexity using population dynamics as well as large degree approximations, and determine the survey propagation threshold. We find that a clustering phase survives coupling. However, as one increases the range of the coupling window, the survey propagation threshold increases and saturates towards the phase transition threshold. We also briefly discuss other aspects of the problem. Namely, the condensation threshold is not affected by coupling, but the dynamic threshold displays saturation towards the condensation one. All these features may provide a new avenue for obtaining better provable algorithmic lower bounds on phase transition thresholds of the individual standard mode
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
The Velocity of the Propagating Wave for General Coupled Scalar Systems
We consider spatially coupled systems governed by a set of scalar density
evolution equations. Such equations track the behavior of message-passing
algorithms used, for example, in coding, sparse sensing, or
constraint-satisfaction problems. Assuming that the "profile" describing the
average state of the algorithm exhibits a solitonic wave-like behavior after
initial transient iterations, we derive a formula for the propagation velocity
of the wave. We illustrate the formula with two applications, namely
Generalized LDPC codes and compressive sensing.Comment: 5 pages, 5 figures, submitted to the Information Theory Workshop
(ITW) 2016 in Cambridge, U
The Space of Solutions of Coupled XORSAT Formulae
The XOR-satisfiability (XORSAT) problem deals with a system of Boolean
variables and clauses. Each clause is a linear Boolean equation (XOR) of a
subset of the variables. A -clause is a clause involving distinct
variables. In the random -XORSAT problem a formula is created by choosing
-clauses uniformly at random from the set of all possible clauses on
variables. The set of solutions of a random formula exhibits various
geometrical transitions as the ratio varies.
We consider a {\em coupled} -XORSAT ensemble, consisting of a chain of
random XORSAT models that are spatially coupled across a finite window along
the chain direction. We observe that the threshold saturation phenomenon takes
place for this ensemble and we characterize various properties of the space of
solutions of such coupled formulae.Comment: Submitted to ISIT 201
Lossy Source Coding via Spatially Coupled LDGM Ensembles
We study a new encoding scheme for lossy source compression based on
spatially coupled low-density generator-matrix codes. We develop a
belief-propagation guided-decimation algorithm, and show that this algorithm
allows to approach the optimal distortion of spatially coupled ensembles.
Moreover, using the survey propagation formalism, we also observe that the
optimal distortions of the spatially coupled and individual code ensembles are
the same. Since regular low-density generator-matrix codes are known to achieve
the Shannon rate-distortion bound under optimal encoding as the degrees grow,
our results suggest that spatial coupling can be used to reach the
rate-distortion bound, under a {\it low complexity} belief-propagation
guided-decimation algorithm.
This problem is analogous to the MAX-XORSAT problem in computer science.Comment: Submitted to ISIT 201
Sparse Probabilistic Models:Phase Transitions and Solutions via Spatial Coupling
This thesis is concerned with a number of novel uses of spatial coupling, applied to a class of probabilistic graphical models. These models include error correcting codes, random constraint satisfaction problems (CSPs) and statistical physics models called diluted spin systems. Spatial coupling is a technique initially developed for channel coding, which provides a recipe to transform a class of sparse linear codes into codes that are longer but more robust at high noise level. In fact it was observed that for coupled codes there are efficient algorithms whose decoding threshold is the optimal one, a phenomenon called threshold saturation. The main aim of this thesis is to explore alternative applications of spatial coupling. The goal is to study properties of uncoupled probabilistic models (not just coding) through the use of the corresponding spatially coupled models. The methods employed are ranging from the mathematically rigorous to the purely experimental. We first explore spatial coupling as a proof technique in the realm of LDPC codes. The Maxwell conjecture states that for arbitrary BMS channels the optimal (MAP) threshold of the standard (uncoupled) LDPC codes is given by the Maxwell construction. We are able to prove the Maxwell Conjecture for any smooth family of BMS channels by using (i) the fact that coupled codes perform optimally (which was already proved) and (ii) that the optimal thresholds of the coupled and uncoupled LDPC codes coincide. The method is used to derive two more results, namely the equality of GEXIT curves above the MAP threshold and the exactness of the averaged Bethe free energy formula derived under the RS cavity method from statistical physics. As a second application of spatial coupling we show how to derive novel bounds on the phase transitions in random constraint satisfaction problems, and possibly a general class of diluted spin systems. In the case of coloring, we investigate what happens to the dynamic and freezing thresholds. The phenomenon of threshold saturation is present also in this case, with the dynamic threshold moving to the condensation threshold, and the freezing moving to colorability. These claims are supported by experimental evidence, but in some cases, such as the saturation of the freezing threshold it is possible to make part of this claim more rigorous. This allows in principle for the computation of thresholds by use of spatial coupling. The proof is in the spirit of the potential method introduced by Kumar, Young, Macris and Pfister for LDPC codes. Finally, we explore how to find solutions in (uncoupled) probabilistic models. To test this, we start with a typical instance of random K-SAT (the base problem), and we build a spatially coupled structure that locally inherits the structure of the base problem. The goal is to run an algorithm for finding a suitable solution in the coupled structure and then "project" this solution to obtain a solution for the base problem. Experimental evidence points to the fact it is indeed possible to use a form of unit-clause propagation (UCP), a simple algorithm, to achieve this goal. This approach works also in regimes where the standard UCP fails on the base problem
- …