91,923 research outputs found
Conditional Random Field Autoencoders for Unsupervised Structured Prediction
We introduce a framework for unsupervised learning of structured predictors
with overlapping, global features. Each input's latent representation is
predicted conditional on the observable data using a feature-rich conditional
random field. Then a reconstruction of the input is (re)generated, conditional
on the latent structure, using models for which maximum likelihood estimation
has a closed-form. Our autoencoder formulation enables efficient learning
without making unrealistic independence assumptions or restricting the kinds of
features that can be used. We illustrate insightful connections to traditional
autoencoders, posterior regularization and multi-view learning. We show
competitive results with instantiations of the model for two canonical NLP
tasks: part-of-speech induction and bitext word alignment, and show that
training our model can be substantially more efficient than comparable
feature-rich baselines
A common misapplication of statistical inference: nuisance control with null-hypothesis significance tests
Experimental research on behavior and cognition frequently rests on stimulus
or subject selection where not all characteristics can be fully controlled,
even when attempting strict matching. For example, when contrasting patients to
controls, variables such as intelligence or socioeconomic status are often
correlated with patient status. Similarly, when presenting word stimuli,
variables such as word frequency are often correlated with primary variables of
interest. One procedure very commonly employed to control for such nuisance
effects is conducting inferential tests on confounding stimulus or subject
characteristics. For example, if word length is not significantly different for
two stimulus sets, they are considered as matched for word length. Such a test
has high error rates and is conceptually misguided. It reflects a common
misunderstanding of statistical tests: interpreting significance not to refer
to inference about a particular population parameter, but about 1. the sample
in question, 2. the practical relevance of a sample difference (so that a
nonsignificant test is taken to indicate evidence for the absence of relevant
differences). We show inferential testing for assessing nuisance effects to be
inappropriate both pragmatically and philosophically, present a survey showing
its high prevalence, and briefly discuss an alternative in the form of
regression including nuisance variables
A Survey of Word Reordering in Statistical Machine Translation: Computational Models and Language Phenomena
Word reordering is one of the most difficult aspects of statistical machine
translation (SMT), and an important factor of its quality and efficiency.
Despite the vast amount of research published to date, the interest of the
community in this problem has not decreased, and no single method appears to be
strongly dominant across language pairs. Instead, the choice of the optimal
approach for a new translation task still seems to be mostly driven by
empirical trials. To orientate the reader in this vast and complex research
area, we present a comprehensive survey of word reordering viewed as a
statistical modeling challenge and as a natural language phenomenon. The survey
describes in detail how word reordering is modeled within different
string-based and tree-based SMT frameworks and as a stand-alone task, including
systematic overviews of the literature in advanced reordering modeling. We then
question why some approaches are more successful than others in different
language pairs. We argue that, besides measuring the amount of reordering, it
is important to understand which kinds of reordering occur in a given language
pair. To this end, we conduct a qualitative analysis of word reordering
phenomena in a diverse sample of language pairs, based on a large collection of
linguistic knowledge. Empirical results in the SMT literature are shown to
support the hypothesis that a few linguistic facts can be very useful to
anticipate the reordering characteristics of a language pair and to select the
SMT framework that best suits them.Comment: 44 pages, to appear in Computational Linguistic
- …