294 research outputs found
Lecture Notes on Network Information Theory
These lecture notes have been converted to a book titled Network Information
Theory published recently by Cambridge University Press. This book provides a
significantly expanded exposition of the material in the lecture notes as well
as problems and bibliographic notes at the end of each chapter. The authors are
currently preparing a set of slides based on the book that will be posted in
the second half of 2012. More information about the book can be found at
http://www.cambridge.org/9781107008731/. The previous (and obsolete) version of
the lecture notes can be found at http://arxiv.org/abs/1001.3404v4/
LACE: A light-weight, causal model for enhancing coded speech through adaptive convolutions
Classical speech coding uses low-complexity postfilters with zero lookahead
to enhance the quality of coded speech, but their effectiveness is limited by
their simplicity. Deep Neural Networks (DNNs) can be much more effective, but
require high complexity and model size, or added delay. We propose a DNN model
that generates classical filter kernels on a per-frame basis with a model of
just 300~K parameters and 100~MFLOPS complexity, which is a practical
complexity for desktop or mobile device CPUs. The lack of added delay allows it
to be integrated into the Opus codec, and we demonstrate that it enables
effective wideband encoding for bitrates down to 6 kb/s.Comment: 5 pages, accepted at WASPAA 202
Rate-distortion function for finite block codes: Analysis of symmetric binary hamming problem
Shannon's rate-distortion theory provides an asymptotic analysis, where delays are allowed to grow unbounded. In practice, real-time applications, such as video streaming and network storage, are subject to certain maximum delay. Accordingly, it is imperative to develop a finite-delay framework for analyzing the rate-distortion limit. In this backdrop, we propose an intuitive generalization of Shannon's asymptotic operational framework to finite block codes. In view of the extreme complexity of such framework, we obtain insight by specializing to the symmetric binary hamming problem. Even upon such specialization, the proposed framework is computationally so intensive that accurate evaluation of the finite-delay rate-distortion function is practical only upto a block length of three. In order to obtain further insight, we then propose a lower-complexity lower bound, based on the partition function of natural numbers, whose computation is practical upto a block length of six. Finally, using a simple combinatorial argument, we propose an upper bound to localize the desired rate-distortion function between our lower and upper bounds
File compression using probabilistic grammars and LR parsing
Data compression, the reduction in size of the physical representation
of data being stored or transmitted, has long been of interest both as a research topic and as a practical technique. Different methods are used
for encoding different classes of data files. The purpose of this research
is to compress a class of highly redundant data files whose contents are
partially described by a context-free grammar (i.e. text files containing
computer programs).
An encoding technique is developed for the removal of structural
dependancy due to the context-free structure of such files. The technique
depends on a type of LR parsing method called LALR(K) (Lookahead LRM).
The encoder also pays particular attention to the encoding of editing
characters, comments, names and constants.
The encoded data maintains the exact information content of the
original data. Hence, a decoding technique (depending on the same
parsing method) is developed to recover the original information from
its compressed representation.
The technique is demonstrated by compressing Pascal programs. An
optimal coding scheme (based on Huffman codes) is used to encode the
parsing alternatives in each parsing state. The decoder uses these codes
during the decoding phase. Also Huffman codes, based on the probability
of the symbols c oncerned, are used when coding editing characterst
comments, names and constants. The sizes of the parsing tables (and
subsequently the encoding tables) were considerably reduced by splitting
them into a number of sub-tables.
The minimum and the average code length of the average program are
derived from two different matrices. These matrices are constructed
from a probabilistic grammar, and the language generated by this grammar.
Finally, various comparisons are made with a related encoding method by
using a simple context-free language
The Capacity of Online (Causal) -ary Error-Erasure Channels
In the -ary online (or "causal") channel coding model, a sender wishes to
communicate a message to a receiver by transmitting a codeword symbol by symbol via a channel
limited to at most errors and/or erasures. The channel is
"online" in the sense that at the th step of communication the channel
decides whether to corrupt the th symbol or not based on its view so far,
i.e., its decision depends only on the transmitted symbols .
This is in contrast to the classical adversarial channel in which the
corruption is chosen by a channel that has a full knowledge on the sent
codeword .
In this work we study the capacity of -ary online channels for a combined
corruption model, in which the channel may impose at most {\em errors} and
at most {\em erasures} on the transmitted codeword. The online
channel (in both the error and erasure case) has seen a number of recent
studies which present both upper and lower bounds on its capacity. In this
work, we give a full characterization of the capacity as a function of ,
and .Comment: This is a new version of the binary case, which can be found at
arXiv:1412.637
- …