9,889 research outputs found
Lossless and near-lossless source coding for multiple access networks
A multiple access source code (MASC) is a source code designed for the following network configuration: a pair of correlated information sequences {X-i}(i=1)(infinity), and {Y-i}(i=1)(infinity) is drawn independent and identically distributed (i.i.d.) according to joint probability mass function (p.m.f.) p(x, y); the encoder for each source operates without knowledge of the other source; the decoder jointly decodes the encoded bit streams from both sources. The work of Slepian and Wolf describes all rates achievable by MASCs of infinite coding dimension (n --> infinity) and asymptotically negligible error probabilities (P-e((n)) --> 0). In this paper, we consider the properties of optimal instantaneous MASCs with finite coding dimension (n 0) performance. The interest in near-lossless codes is inspired by the discontinuity in the limiting rate region at P-e((n)) = 0 and the resulting performance benefits achievable by using near-lossless MASCs as entropy codes within lossy MASCs. Our central results include generalizations of Huffman and arithmetic codes to the MASC framework for arbitrary p(x, y), n, and P-e((n)) and polynomial-time design algorithms that approximate these optimal solutions
Source Coding for Quasiarithmetic Penalties
Huffman coding finds a prefix code that minimizes mean codeword length for a
given probability distribution over a finite number of items. Campbell
generalized the Huffman problem to a family of problems in which the goal is to
minimize not mean codeword length but rather a generalized mean known as a
quasiarithmetic or quasilinear mean. Such generalized means have a number of
diverse applications, including applications in queueing. Several
quasiarithmetic-mean problems have novel simple redundancy bounds in terms of a
generalized entropy. A related property involves the existence of optimal
codes: For ``well-behaved'' cost functions, optimal codes always exist for
(possibly infinite-alphabet) sources having finite generalized entropy. Solving
finite instances of such problems is done by generalizing an algorithm for
finding length-limited binary codes to a new algorithm for finding optimal
binary codes for any quasiarithmetic mean with a convex cost function. This
algorithm can be performed using quadratic time and linear space, and can be
extended to other penalty functions, some of which are solvable with similar
space and time complexity, and others of which are solvable with slightly
greater complexity. This reduces the computational complexity of a problem
involving minimum delay in a queue, allows combinations of previously
considered problems to be optimized, and greatly expands the space of problems
solvable in quadratic time and linear space. The algorithm can be extended for
purposes such as breaking ties among possibly different optimal codes, as with
bottom-merge Huffman coding.Comment: 22 pages, 3 figures, submitted to IEEE Trans. Inform. Theory, revised
per suggestions of reader
Prefix Codes: Equiprobable Words, Unequal Letter Costs
Describes a near-linear-time algorithm for a variant of Huffman coding, in
which the letters may have non-uniform lengths (as in Morse code), but with the
restriction that each word to be encoded has equal probability. [See also
``Huffman Coding with Unequal Letter Costs'' (2002).]Comment: proceedings version in ICALP (1994
Cryptanalysis of an MPEG-Video Encryption Scheme Based on Secret Huffman Tables
This paper studies the security of a recently-proposed MPEG-video encryption
scheme based on secret Huffman tables. Our cryptanalysis shows that: 1) the key
space of the encryption scheme is not sufficiently large against
divide-and-conquer (DAC) attack and known-plaintext attack; 2) it is possible
to decrypt a cipher-video with a partially-known key, thus dramatically
reducing the complexity of the DAC brute-force attack in some cases; 3) its
security against the chosen-plaintext attack is very weak. Some experimental
results are included to support the cryptanalytic results with a brief discuss
on how to improve this MPEG-video encryption scheme.Comment: 8 pages, 4 figure
Weighted universal bit allocation: optimal multiple quantization matrix coding
We introduce a two-stage bit allocation algorithm analogous to the algorithm for weighted universal vector quantization (WUVQ). The encoder uses a collection of possible bit allocations (typically in the form of a collection of quantization matrices) rather than a single bit allocation (or single quantization matrix). We describe both an encoding algorithm for achieving optimal compression using a collection of bit allocations and a technique for designing locally optimal collections of bit allocations. We demonstrate performance on a JPEG style coder using the mean squared error (MSE) distortion measure. On a sequence of medical brain scans, the algorithm achieves up to 2.5 dB improvement over a single bit allocation system, up to 5 dB improvement over a WUVQ with first- and second-stage vector dimensions equal to 16 and 4 respectively, and up to 12 dB improvement over an entropy constrained vector quantizer (ECVQ) using 4 dimensional vectors
The map equation
Many real-world networks are so large that we must simplify their structure
before we can extract useful information about the systems they represent. As
the tools for doing these simplifications proliferate within the network
literature, researchers would benefit from some guidelines about which of the
so-called community detection algorithms are most appropriate for the
structures they are studying and the questions they are asking. Here we show
that different methods highlight different aspects of a network's structure and
that the the sort of information that we seek to extract about the system must
guide us in our decision. For example, many community detection algorithms,
including the popular modularity maximization approach, infer module
assignments from an underlying model of the network formation process. However,
we are not always as interested in how a system's network structure was formed,
as we are in how a network's extant structure influences the system's behavior.
To see how structure influences current behavior, we will recognize that links
in a network induce movement across the network and result in system-wide
interdependence. In doing so, we explicitly acknowledge that most networks
carry flow. To highlight and simplify the network structure with respect to
this flow, we use the map equation. We present an intuitive derivation of this
flow-based and information-theoretic method and provide an interactive on-line
application that anyone can use to explore the mechanics of the map equation.
We also describe an algorithm and provide source code to efficiently decompose
large weighted and directed networks based on the map equation.Comment: 9 pages and 3 figures, corrected typos. For associated Flash
application, see http://www.tp.umu.se/~rosvall/livemod/mapequation
First-Come-First-Served for Online Slot Allocation and Huffman Coding
Can one choose a good Huffman code on the fly, without knowing the underlying
distribution? Online Slot Allocation (OSA) models this and similar problems:
There are n slots, each with a known cost. There are n items. Requests for
items are drawn i.i.d. from a fixed but hidden probability distribution p.
After each request, if the item, i, was not previously requested, then the
algorithm (knowing the slot costs and the requests so far, but not p) must
place the item in some vacant slot j(i). The goal is to minimize the sum, over
the items, of the probability of the item times the cost of its assigned slot.
The optimal offline algorithm is trivial: put the most probable item in the
cheapest slot, the second most probable item in the second cheapest slot, etc.
The optimal online algorithm is First Come First Served (FCFS): put the first
requested item in the cheapest slot, the second (distinct) requested item in
the second cheapest slot, etc. The optimal competitive ratios for any online
algorithm are 1+H(n-1) ~ ln n for general costs and 2 for concave costs. For
logarithmic costs, the ratio is, asymptotically, 1: FCFS gives cost opt + O(log
opt).
For Huffman coding, FCFS yields an online algorithm (one that allocates
codewords on demand, without knowing the underlying probability distribution)
that guarantees asymptotically optimal cost: at most opt + 2 log(1+opt) + 2.Comment: ACM-SIAM Symposium on Discrete Algorithms (SODA) 201
- …