59,798 research outputs found
Tropical Geometry of Statistical Models
This paper presents a unified mathematical framework for inference in
graphical models, building on the observation that graphical models are
algebraic varieties.
From this geometric viewpoint, observations generated from a model are
coordinates of a point in the variety, and the sum-product algorithm is an
efficient tool for evaluating specific coordinates. The question addressed here
is how the solutions to various inference problems depend on the model
parameters. The proposed answer is expressed in terms of tropical algebraic
geometry. A key role is played by the Newton polytope of a statistical model.
Our results are applied to the hidden Markov model and to the general Markov
model on a binary tree.Comment: 14 pages, 3 figures. Major revision. Applications now in companion
paper, "Parametric Inference for Biological Sequence Analysis
Learning Dynamic Feature Selection for Fast Sequential Prediction
We present paired learning and inference algorithms for significantly
reducing computation and increasing speed of the vector dot products in the
classifiers that are at the heart of many NLP components. This is accomplished
by partitioning the features into a sequence of templates which are ordered
such that high confidence can often be reached using only a small fraction of
all features. Parameter estimation is arranged to maximize accuracy and early
confidence in this sequence. Our approach is simpler and better suited to NLP
than other related cascade methods. We present experiments in left-to-right
part-of-speech tagging, named entity recognition, and transition-based
dependency parsing. On the typical benchmarking datasets we can preserve POS
tagging accuracy above 97% and parsing LAS above 88.5% both with over a
five-fold reduction in run-time, and NER F1 above 88 with more than 2x increase
in speed.Comment: Appears in The 53rd Annual Meeting of the Association for
Computational Linguistics, Beijing, China, July 201
- …