18,105 research outputs found
Opportunistic scheduling with limited channel state information: A rate distortion approach
We consider an opportunistic communication system in which a transmitter selects one of multiple channels over which to schedule a transmission, based on partial knowledge of the network state. We characterize a fundamental limit on the rate that channel state information must be conveyed to the transmitter in order to meet a constraint on expected throughput. This problem is modeled as a causal rate distortion optimization of a Markov source. We introduce a novel distortion metric capturing the impact of imperfect channel state information on throughput. We compute a closed-form expression for the causal information rate distortion function for the case of two channels, as well as an algorithmic upper bound on the causal rate distortion function. Finally, we characterize the gap between the causal information rate distortion and the causal entropic rate-distortion functions.National Science Foundation (U.S.) (Grant CNS-0915988)National Science Foundation (U.S.) (Grant CNS-1217048)United States. Army Research Office. Multidisciplinary University Research Initiative (Grant W911NF-08-1-0238)United States. Office of Naval Research (Grant N00014-12-1-0064)National Science Foundation (U.S.). Center for Science of Information (Grant CCF-09-39370
On the Computation of the Gaussian Rate-Distortion-Perception Function
In this paper, we study the computation of the rate-distortion-perception
function (RDPF) for a multivariate Gaussian source under mean squared error
(MSE) distortion and, respectively, Kullback-Leibler divergence, geometric
Jensen-Shannon divergence, squared Hellinger distance, and squared
Wasserstein-2 distance perception metrics. To this end, we first characterize
the analytical bounds of the scalar Gaussian RDPF for the aforementioned
divergence functions, also providing the RDPF-achieving forward "test-channel"
realization. Focusing on the multivariate case, we establish that, for
tensorizable distortion and perception metrics, the optimal solution resides on
the vector space spanned by the eigenvector of the source covariance matrix.
Consequently, the multivariate optimization problem can be expressed as a
function of the scalar Gaussian RDPFs of the source marginals, constrained by
global distortion and perception levels. Leveraging this characterization, we
design an alternating minimization scheme based on the block nonlinear
Gauss-Seidel method, which optimally solves the problem while identifying the
Gaussian RDPF-achieving realization. Furthermore, the associated algorithmic
embodiment is provided, as well as the convergence and the rate of convergence
characterization. Lastly, for the "perfect realism" regime, the analytical
solution for the multivariate Gaussian RDPF is obtained. We corroborate our
results with numerical simulations and draw connections to existing results.Comment: This paper has been submitted for journal publicatio
Shannon Information and Kolmogorov Complexity
We compare the elementary theories of Shannon information and Kolmogorov
complexity, the extent to which they have a common purpose, and where they are
fundamentally different. We discuss and relate the basic notions of both
theories: Shannon entropy versus Kolmogorov complexity, the relation of both to
universal coding, Shannon mutual information versus Kolmogorov (`algorithmic')
mutual information, probabilistic sufficient statistic versus algorithmic
sufficient statistic (related to lossy compression in the Shannon theory versus
meaningful information in the Kolmogorov theory), and rate distortion theory
versus Kolmogorov's structure function. Part of the material has appeared in
print before, scattered through various publications, but this is the first
comprehensive systematic comparison. The last mentioned relations are new.Comment: Survey, LaTeX 54 pages, 3 figures, Submitted to IEEE Trans
Information Theor
Approaching the Rate-Distortion Limit with Spatial Coupling, Belief propagation and Decimation
We investigate an encoding scheme for lossy compression of a binary symmetric
source based on simple spatially coupled Low-Density Generator-Matrix codes.
The degree of the check nodes is regular and the one of code-bits is Poisson
distributed with an average depending on the compression rate. The performance
of a low complexity Belief Propagation Guided Decimation algorithm is
excellent. The algorithmic rate-distortion curve approaches the optimal curve
of the ensemble as the width of the coupling window grows. Moreover, as the
check degree grows both curves approach the ultimate Shannon rate-distortion
limit. The Belief Propagation Guided Decimation encoder is based on the
posterior measure of a binary symmetric test-channel. This measure can be
interpreted as a random Gibbs measure at a "temperature" directly related to
the "noise level of the test-channel". We investigate the links between the
algorithmic performance of the Belief Propagation Guided Decimation encoder and
the phase diagram of this Gibbs measure. The phase diagram is investigated
thanks to the cavity method of spin glass theory which predicts a number of
phase transition thresholds. In particular the dynamical and condensation
"phase transition temperatures" (equivalently test-channel noise thresholds)
are computed. We observe that: (i) the dynamical temperature of the spatially
coupled construction saturates towards the condensation temperature; (ii) for
large degrees the condensation temperature approaches the temperature (i.e.
noise level) related to the information theoretic Shannon test-channel noise
parameter of rate-distortion theory. This provides heuristic insight into the
excellent performance of the Belief Propagation Guided Decimation algorithm.
The paper contains an introduction to the cavity method
Lossy data compression with random gates
We introduce a new protocol for a lossy data compression algorithm which is
based on constraint satisfaction gates. We show that the theoretical capacity
of algorithms built from standard parity-check gates converges exponentially
fast to the Shannon's bound when the number of variables seen by each gate
increases. We then generalize this approach by introducing random gates. They
have theoretical performances nearly as good as parity checks, but they offer
the great advantage that the encoding can be done in linear time using the
Survey Inspired Decimation algorithm, a powerful algorithm for constraint
satisfaction problems derived from statistical physics
Reducing the loss of information through annealing text distortion
Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Granados, A. ;Cebrian, M. ; Camacho, D. ; de Borja Rodriguez, F. "Reducing the Loss of Information through Annealing Text Distortion". IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 7 pp. 1090 - 1102, July 2011Compression distances have been widely used in knowledge discovery and data mining. They are parameter-free, widely applicable, and very effective in several domains. However, little has been done to interpret their results or to explain their behavior. In this paper, we take a step toward understanding compression distances by performing an experimental evaluation of the impact of several kinds of information distortion on compression-based text clustering. We show how progressively removing words in such a way that the complexity of a document is slowly reduced helps the compression-based text clustering and improves its accuracy. In fact, we show how the nondistorted text clustering can be improved by means of annealing text distortion. The experimental results shown in this paper are consistent using different data sets, and different compression algorithms belonging to the most important compression families: Lempel-Ziv, Statistical and Block-Sorting.This work was supported by the Spanish Ministry of Education and Science under TIN2010-19872 and TIN2010-19607 projects
- …