252 research outputs found
Neural Estimation of the Rate-Distortion Function With Applications to Operational Source Coding
A fundamental question in designing lossy data compression schemes is how
well one can do in comparison with the rate-distortion function, which
describes the known theoretical limits of lossy compression. Motivated by the
empirical success of deep neural network (DNN) compressors on large, real-world
data, we investigate methods to estimate the rate-distortion function on such
data, which would allow comparison of DNN compressors with optimality. While
one could use the empirical distribution of the data and apply the
Blahut-Arimoto algorithm, this approach presents several computational
challenges and inaccuracies when the datasets are large and high-dimensional,
such as the case of modern image datasets. Instead, we re-formulate the
rate-distortion objective, and solve the resulting functional optimization
problem using neural networks. We apply the resulting rate-distortion
estimator, called NERD, on popular image datasets, and provide evidence that
NERD can accurately estimate the rate-distortion function. Using our estimate,
we show that the rate-distortion achievable by DNN compressors are within
several bits of the rate-distortion function for real-world datasets.
Additionally, NERD provides access to the rate-distortion achieving channel, as
well as samples from its output marginal. Therefore, using recent results in
reverse channel coding, we describe how NERD can be used to construct an
operational one-shot lossy compression scheme with guarantees on the achievable
rate and distortion. Experimental results demonstrate competitive performance
with DNN compressors
Syndrome-Based Encoding of Compressible Sources for M2M Communication
Data originating from many devices and sensors can be modeled as sparse signals. Hence, efficient compression techniques of such data are essential to reduce bandwidth and transmission power, especially for energy constrained devices within machine to machine communication scenarios. This paper provides accurate analysis of the operational distortion-rate function (ODR) for syndrome-based source encoders of noisy sparse sources. We derive the probability density function of error due to both quantization and pre- quantization noise for a type of mixed distributed source comprising Bernoulli and an arbitrary continuous distribution, e.g., Bernoulli- uniform sources. Then, we derive the ODR for two encoding schemes based on the syndromes of Reed-Solomon (RS) and Bose, Chaudhuri, and Hocquenghem (BCH) codes. The presented analysis allows designing a quantizer such that a target average distortion is achieved. As confirmed by numerical results, the closed-form expression for ODR perfectly coincides with the simulation. Also, the performance loss compared to an entropy based encoder is tolerable
- …