5,458 research outputs found
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
Lossy Source Coding via Spatially Coupled LDGM Ensembles
We study a new encoding scheme for lossy source compression based on
spatially coupled low-density generator-matrix codes. We develop a
belief-propagation guided-decimation algorithm, and show that this algorithm
allows to approach the optimal distortion of spatially coupled ensembles.
Moreover, using the survey propagation formalism, we also observe that the
optimal distortions of the spatially coupled and individual code ensembles are
the same. Since regular low-density generator-matrix codes are known to achieve
the Shannon rate-distortion bound under optimal encoding as the degrees grow,
our results suggest that spatial coupling can be used to reach the
rate-distortion bound, under a {\it low complexity} belief-propagation
guided-decimation algorithm.
This problem is analogous to the MAX-XORSAT problem in computer science.Comment: Submitted to ISIT 201
Syndrome source coding and its universal generalization
A method of using error-correcting codes to obtain data compression, called syndrome-source-coding, is described in which the source sequence is treated as an error pattern whose syndrome forms the compressed data. It is shown that syndrome-source-coding can achieve arbitrarily small distortion with the number of compressed digits per source digit arbitrarily close to the entropy of a binary memoryless source. A universal generalization of syndrome-source-coding is formulated which provides robustly-effective, distortionless, coding of source ensembles
Message passing algorithms for non-linear nodes and data compression
The use of parity-check gates in information theory has proved to be very
efficient. In particular, error correcting codes based on parity checks over
low-density graphs show excellent performances. Another basic issue of
information theory, namely data compression, can be addressed in a similar way
by a kind of dual approach. The theoretical performance of such a Parity Source
Coder can attain the optimal limit predicted by the general rate-distortion
theory. However, in order to turn this approach into an efficient compression
code (with fast encoding/decoding algorithms) one must depart from parity
checks and use some general random gates. By taking advantage of analytical
approaches from the statistical physics of disordered systems and SP-like
message passing algorithms, we construct a compressor based on low-density
non-linear gates with a very good theoretical and practical performance.Comment: 13 pages, European Conference on Complex Systems, Paris (Nov 2005
Fundamentals of Large Sensor Networks: Connectivity, Capacity, Clocks and Computation
Sensor networks potentially feature large numbers of nodes that can sense
their environment over time, communicate with each other over a wireless
network, and process information. They differ from data networks in that the
network as a whole may be designed for a specific application. We study the
theoretical foundations of such large scale sensor networks, addressing four
fundamental issues- connectivity, capacity, clocks and function computation.
To begin with, a sensor network must be connected so that information can
indeed be exchanged between nodes. The connectivity graph of an ad-hoc network
is modeled as a random graph and the critical range for asymptotic connectivity
is determined, as well as the critical number of neighbors that a node needs to
connect to. Next, given connectivity, we address the issue of how much data can
be transported over the sensor network. We present fundamental bounds on
capacity under several models, as well as architectural implications for how
wireless communication should be organized.
Temporal information is important both for the applications of sensor
networks as well as their operation.We present fundamental bounds on the
synchronizability of clocks in networks, and also present and analyze
algorithms for clock synchronization. Finally we turn to the issue of gathering
relevant information, that sensor networks are designed to do. One needs to
study optimal strategies for in-network aggregation of data, in order to
reliably compute a composite function of sensor measurements, as well as the
complexity of doing so. We address the issue of how such computation can be
performed efficiently in a sensor network and the algorithms for doing so, for
some classes of functions.Comment: 10 pages, 3 figures, Submitted to the Proceedings of the IEE
- …