10,944 research outputs found
Relations between random coding exponents and the statistical physics of random codes
The partition function pertaining to finite--temperature decoding of a
(typical) randomly chosen code is known to have three types of behavior,
corresponding to three phases in the plane of rate vs. temperature: the {\it
ferromagnetic phase}, corresponding to correct decoding, the {\it paramagnetic
phase}, of complete disorder, which is dominated by exponentially many
incorrect codewords, and the {\it glassy phase} (or the condensed phase), where
the system is frozen at minimum energy and dominated by subexponentially many
incorrect codewords. We show that the statistical physics associated with the
two latter phases are intimately related to random coding exponents. In
particular, the exponent associated with the probability of correct decoding at
rates above capacity is directly related to the free energy in the glassy
phase, and the exponent associated with probability of error (the error
exponent) at rates below capacity, is strongly related to the free energy in
the paramagnetic phase. In fact, we derive alternative expressions of these
exponents in terms of the corresponding free energies, and make an attempt to
obtain some insights from these expressions. Finally, as a side result, we also
compare the phase diagram associated with a simple finite-temperature universal
decoder for discrete memoryless channels, to that of the finite--temperature
decoder that is aware of the channel statistics.Comment: 26 pages, 2 figures, submitted to IEEE Transactions on Information
Theor
A Tutorial on Time-Evolving Dynamical Bayesian Inference
In view of the current availability and variety of measured data, there is an
increasing demand for powerful signal processing tools that can cope
successfully with the associated problems that often arise when data are being
analysed. In practice many of the data-generating systems are not only
time-variable, but also influenced by neighbouring systems and subject to
random fluctuations (noise) from their environments. To encompass problems of
this kind, we present a tutorial about the dynamical Bayesian inference of
time-evolving coupled systems in the presence of noise. It includes the
necessary theoretical description and the algorithms for its implementation.
For general programming purposes, a pseudocode description is also given.
Examples based on coupled phase and limit-cycle oscillators illustrate the
salient features of phase dynamics inference. State domain inference is
illustrated with an example of coupled chaotic oscillators. The applicability
of the latter example to secure communications based on the modulation of
coupling functions is outlined. MatLab codes for implementation of the method,
as well as for the explicit examples, accompany the tutorial.Comment: Matlab codes can be found on http://py-biomedical.lancaster.ac.uk
- …