8,784 research outputs found
A generalized risk approach to path inference based on hidden Markov models
Motivated by the unceasing interest in hidden Markov models (HMMs), this
paper re-examines hidden path inference in these models, using primarily a
risk-based framework. While the most common maximum a posteriori (MAP), or
Viterbi, path estimator and the minimum error, or Posterior Decoder (PD), have
long been around, other path estimators, or decoders, have been either only
hinted at or applied more recently and in dedicated applications generally
unfamiliar to the statistical learning community. Over a decade ago, however, a
family of algorithmically defined decoders aiming to hybridize the two standard
ones was proposed (Brushe et al., 1998). The present paper gives a careful
analysis of this hybridization approach, identifies several problems and issues
with it and other previously proposed approaches, and proposes practical
resolutions of those. Furthermore, simple modifications of the classical
criteria for hidden path recognition are shown to lead to a new class of
decoders. Dynamic programming algorithms to compute these decoders in the usual
forward-backward manner are presented. A particularly interesting subclass of
such estimators can be also viewed as hybrids of the MAP and PD estimators.
Similar to previously proposed MAP-PD hybrids, the new class is parameterized
by a small number of tunable parameters. Unlike their algorithmic predecessors,
the new risk-based decoders are more clearly interpretable, and, most
importantly, work "out of the box" in practice, which is demonstrated on some
real bioinformatics tasks and data. Some further generalizations and
applications are discussed in conclusion.Comment: Section 5: corrected denominators of the scaled beta variables (pp.
27-30), => corrections in claims 1, 3, Prop. 12, bottom of Table 1. Decoder
(49), Corol. 14 are generalized to handle 0 probabilities. Notation is more
closely aligned with (Bishop, 2006). Details are inserted in eqn-s (43); the
positivity assumption in Prop. 11 is explicit. Fixed typing errors in
equation (41), Example
Multiresolution vector quantization
Multiresolution source codes are data compression algorithms yielding embedded source descriptions. The decoder of a multiresolution code can build a source reproduction by decoding the embedded bit stream in part or in whole. All decoding procedures start at the beginning of the binary source description and decode some fraction of that string. Decoding a small portion of the binary string gives a low-resolution reproduction; decoding more yields a higher resolution reproduction; and so on. Multiresolution vector quantizers are block multiresolution source codes. This paper introduces algorithms for designing fixed- and variable-rate multiresolution vector quantizers. Experiments on synthetic data demonstrate performance close to the theoretical performance limit. Experiments on natural images demonstrate performance improvements of up to 8 dB over tree-structured vector quantizers. Some of the lessons learned through multiresolution vector quantizer design lend insight into the design of more sophisticated multiresolution codes
The benefit of a 1-bit jump-start, and the necessity of stochastic encoding, in jamming channels
We consider the problem of communicating a message in the presence of a
malicious jamming adversary (Calvin), who can erase an arbitrary set of up to
bits, out of transmitted bits . The capacity of such
a channel when Calvin is exactly causal, i.e. Calvin's decision of whether or
not to erase bit depends on his observations was
recently characterized to be . In this work we show two (perhaps)
surprising phenomena. Firstly, we demonstrate via a novel code construction
that if Calvin is delayed by even a single bit, i.e. Calvin's decision of
whether or not to erase bit depends only on (and
is independent of the "current bit" ) then the capacity increases to
when the encoder is allowed to be stochastic. Secondly, we show via a novel
jamming strategy for Calvin that, in the single-bit-delay setting, if the
encoding is deterministic (i.e. the transmitted codeword is a deterministic
function of the message ) then no rate asymptotically larger than is
possible with vanishing probability of error, hence stochastic encoding (using
private randomness at the encoder) is essential to achieve the capacity of
against a one-bit-delayed Calvin.Comment: 21 pages, 4 figures, extended draft of submission to ISIT 201
Upper Bounds on the Capacity of Binary Channels with Causal Adversaries
In this work we consider the communication of information in the presence of
a causal adversarial jammer. In the setting under study, a sender wishes to
communicate a message to a receiver by transmitting a codeword
bit-by-bit over a communication channel. The sender and the receiver do not
share common randomness. The adversarial jammer can view the transmitted bits
one at a time, and can change up to a -fraction of them. However, the
decisions of the jammer must be made in a causal manner. Namely, for each bit
the jammer's decision on whether to corrupt it or not must depend only on
for . This is in contrast to the "classical" adversarial
jamming situations in which the jammer has no knowledge of , or
knows completely. In this work, we present upper bounds (that
hold under both the average and maximal probability of error criteria) on the
capacity which hold for both deterministic and stochastic encoding schemes.Comment: To appear in the IEEE Transactions on Information Theory; shortened
version appeared at ISIT 201
- …