2,451 research outputs found
An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities
We describe an extension of Earley's parser for stochastic context-free
grammars that computes the following quantities given a stochastic context-free
grammar and an input string: a) probabilities of successive prefixes being
generated by the grammar; b) probabilities of substrings being generated by the
nonterminals, including the entire string being generated by the grammar; c)
most likely (Viterbi) parse of the string; d) posterior expected number of
applications of each grammar production, as required for reestimating rule
probabilities. (a) and (b) are computed incrementally in a single left-to-right
pass over the input. Our algorithm compares favorably to standard bottom-up
parsing methods for SCFGs in that it works efficiently on sparse grammars by
making use of Earley's top-down control structure. It can process any
context-free rule format without conversion to some normal form, and combines
computations for (a) through (d) in a single algorithm. Finally, the algorithm
has simple extensions for processing partially bracketed inputs, and for
finding partial parses and their likelihoods on ungrammatical inputs.Comment: 45 pages. Slightly shortened version to appear in Computational
Linguistics 2
KL-Divergence Guided Two-Beam Viterbi Algorithm on Factorial HMMs
This thesis addresses the problem of the high computation complexity issue that arises when decoding hidden Markov models (HMMs) with a large number of states. A novel approach, the two-beam Viterbi, with an extra forward beam, for decoding HMMs is implemented on a system that uses factorial HMM to simultaneously recognize a pair of isolated digits on one audio channel. The two-beam Viterbi algorithm uses KL-divergence and hierarchical clustering to reduce the overall decoding complexity. This novel approach achieves 60% less computation compared to the baseline algorithm, the Viterbi beam search, while maintaining 82.5% recognition accuracy.Ope
Fast, Robust, and Versatile Event Detection through HMM Belief State Gradient Measures
Event detection is a critical feature in data-driven systems as it assists
with the identification of nominal and anomalous behavior. Event detection is
increasingly relevant in robotics as robots operate with greater autonomy in
increasingly unstructured environments. In this work, we present an accurate,
robust, fast, and versatile measure for skill and anomaly identification. A
theoretical proof establishes the link between the derivative of the
log-likelihood of the HMM filtered belief state and the latest emission
probabilities. The key insight is the inverse relationship in which gradient
analysis is used for skill and anomaly identification. Our measure showed
better performance across all metrics than related state-of-the art works. The
result is broadly applicable to domains that use HMMs for event detection.Comment: 8 pages, 7 figures, double col, ieee conference forma
Inference with Constrained Hidden Markov Models in PRISM
A Hidden Markov Model (HMM) is a common statistical model which is widely
used for analysis of biological sequence data and other sequential phenomena.
In the present paper we show how HMMs can be extended with side-constraints and
present constraint solving techniques for efficient inference. Defining HMMs
with side-constraints in Constraint Logic Programming have advantages in terms
of more compact expression and pruning opportunities during inference.
We present a PRISM-based framework for extending HMMs with side-constraints
and show how well-known constraints such as cardinality and all different are
integrated. We experimentally validate our approach on the biologically
motivated problem of global pairwise alignment
Low-Complexity LP Decoding of Nonbinary Linear Codes
Linear Programming (LP) decoding of Low-Density Parity-Check (LDPC) codes has
attracted much attention in the research community in the past few years. LP
decoding has been derived for binary and nonbinary linear codes. However, the
most important problem with LP decoding for both binary and nonbinary linear
codes is that the complexity of standard LP solvers such as the simplex
algorithm remains prohibitively large for codes of moderate to large block
length. To address this problem, two low-complexity LP (LCLP) decoding
algorithms for binary linear codes have been proposed by Vontobel and Koetter,
henceforth called the basic LCLP decoding algorithm and the subgradient LCLP
decoding algorithm.
In this paper, we generalize these LCLP decoding algorithms to nonbinary
linear codes. The computational complexity per iteration of the proposed
nonbinary LCLP decoding algorithms scales linearly with the block length of the
code. A modified BCJR algorithm for efficient check-node calculations in the
nonbinary basic LCLP decoding algorithm is also proposed, which has complexity
linear in the check node degree.
Several simulation results are presented for nonbinary LDPC codes defined
over Z_4, GF(4), and GF(8) using quaternary phase-shift keying and
8-phase-shift keying, respectively, over the AWGN channel. It is shown that for
some group-structured LDPC codes, the error-correcting performance of the
nonbinary LCLP decoding algorithms is similar to or better than that of the
min-sum decoding algorithm.Comment: To appear in IEEE Transactions on Communications, 201
- …