7,698 research outputs found
Lower Bounds on the Redundancy of Huffman Codes with Known and Unknown Probabilities
In this paper we provide a method to obtain tight lower bounds on the minimum
redundancy achievable by a Huffman code when the probability distribution
underlying an alphabet is only partially known. In particular, we address the
case where the occurrence probabilities are unknown for some of the symbols in
an alphabet. Bounds can be obtained for alphabets of a given size, for
alphabets of up to a given size, and for alphabets of arbitrary size. The
method operates on a Computer Algebra System, yielding closed-form numbers for
all results. Finally, we show the potential of the proposed method to shed some
light on the structure of the minimum redundancy achievable by the Huffman
code
The placement of the head that maximizes predictability. An information theoretic approach
The minimization of the length of syntactic dependencies is a
well-established principle of word order and the basis of a mathematical theory
of word order. Here we complete that theory from the perspective of information
theory, adding a competing word order principle: the maximization of
predictability of a target element. These two principles are in conflict: to
maximize the predictability of the head, the head should appear last, which
maximizes the costs with respect to dependency length minimization. The
implications of such a broad theoretical framework to understand the
optimality, diversity and evolution of the six possible orderings of subject,
object and verb are reviewed.Comment: in press in Glottometric
Optimal Prefix Codes for Infinite Alphabets with Nonlinear Costs
Let be a measure of strictly positive probabilities on the set
of nonnegative integers. Although the countable number of inputs prevents usage
of the Huffman algorithm, there are nontrivial for which known methods find
a source code that is optimal in the sense of minimizing expected codeword
length. For some applications, however, a source code should instead minimize
one of a family of nonlinear objective functions, -exponential means,
those of the form , where is the length of
the th codeword and is a positive constant. Applications of such
minimizations include a novel problem of maximizing the chance of message
receipt in single-shot communications () and a previously known problem of
minimizing the chance of buffer overflow in a queueing system (). This
paper introduces methods for finding codes optimal for such exponential means.
One method applies to geometric distributions, while another applies to
distributions with lighter tails. The latter algorithm is applied to Poisson
distributions and both are extended to alphabetic codes, as well as to
minimizing maximum pointwise redundancy. The aforementioned application of
minimizing the chance of buffer overflow is also considered.Comment: 14 pages, 6 figures, accepted to IEEE Trans. Inform. Theor
`The frozen accident' as an evolutionary adaptation: A rate distortion theory perspective on the dynamics and symmetries of genetic coding mechanisms
We survey some interpretations and related issues concerning the frozen hypothesis due to F. Crick and how it can be explained in terms of several natural mechanisms involving error correction codes, spin glasses, symmetry breaking and the characteristic robustness of genetic networks. The approach to most of these questions involves using elements of Shannon's rate distortion theory incorporating a semantic system which is meaningful for the relevant alphabets and vocabulary implemented in transmission of the genetic code. We apply the fundamental homology between information source uncertainty with the free energy density of a thermodynamical system with respect to transcriptional regulators and the communication channels of sequence/structure in proteins. This leads to the suggestion that the frozen accident may have been a type of evolutionary adaptation
A spin glass model for reconstructing nonlinearly encrypted signals corrupted by noise
An encryption of a signal is a random mapping which can be corrupted
by an additive noise. Given the Encryption Redundancy Parameter (ERP)
, the signal strength parameter , and
the ('bare') noise-to-signal ratio (NSR) , we consider the problem
of reconstructing from its corrupted image by a Least Square Scheme
for a certain class of random Gaussian mappings. The problem is equivalent to
finding the configuration of minimal energy in a certain version of spherical
spin glass model, with squared Gaussian-distributed random potential. We use
the Parisi replica symmetry breaking scheme to evaluate the mean overlap
between the original signal and its recovered image
(known as 'estimator') as , which is a measure of the quality of
the signal reconstruction. We explicitly analyze the general case of
linear-quadratic family of random mappings and discuss the full curve. When nonlinearity exceeds a certain threshold but redundancy
is not yet too big, the replica symmetric solution is necessarily broken in
some interval of NSR. We show that encryptions with a nonvanishing linear
component permit reconstructions with for any and any
, with as . In
contrast, for the case of purely quadratic nonlinearity, for any ERP
there exists a threshold NSR value such that for
making the reconstruction impossible. The behaviour
close to the threshold is given by and
is controlled by the replica symmetry breaking mechanism.Comment: 33 pages, 5 figure
- …