25 research outputs found

    A Tutorial on the Expectation-Maximization Algorithm Including Maximum-Likelihood Estimation and EM Training of Probabilistic Context-Free Grammars

    Full text link
    The paper gives a brief review of the expectation-maximization algorithm (Dempster 1977) in the comprehensible framework of discrete mathematics. In Section 2, two prominent estimation methods, the relative-frequency estimation and the maximum-likelihood estimation are presented. Section 3 is dedicated to the expectation-maximization algorithm and a simpler variant, the generalized expectation-maximization algorithm. In Section 4, two loaded dice are rolled. A more interesting example is presented in Section 5: The estimation of probabilistic context-free grammars.Comment: Presented at the 15th European Summer School in Logic, Language and Information (ESSLLI 2003). Example 5 extended (and partially corrected

    Using a Probabilistic Class-Based Lexicon for Lexical Ambiguity Resolution

    Full text link
    This paper presents the use of probabilistic class-based lexica for disambiguation in target-word selection. Our method employs minimal but precise contextual information for disambiguation. That is, only information provided by the target-verb, enriched by the condensed information of a probabilistic class-based lexicon, is used. Induction of classes and fine-tuning to verbal arguments is done in an unsupervised manner by EM-based clustering techniques. The method shows promising results in an evaluation on real-world translations.Comment: 7 pages, uses colacl.st

    Inducing a Semantically Annotated Lexicon via EM-Based Clustering

    Full text link
    We present a technique for automatic induction of slot annotations for subcategorization frames, based on induction of hidden classes in the EM framework of statistical estimation. The models are empirically evalutated by a general decision test. Induction of slot labeling for subcategorization frames is accomplished by a further application of EM, and applied experimentally on frame observations derived from parsing large corpora. We outline an interpretation of the learned representations as theoretical-linguistic decompositional lexical entries.Comment: 8 pages, uses colacl.sty. Proceedings of the 37th Annual Meeting of the ACL, 199

    Inside-outside estimation meets dynamic EM : gold

    No full text
    It is an interesting fact that most of the stochastic models used by linguists can be interpreted as probabilistic context-free grammars (Prescher 2001). In this paper, this result will be accompanied by the formal proof that the inside-outside algorithm, the standard training method for probabilistic context-free grammars, can be regarded as dynamic-programming variant of the EM algorithm. Even if this result is considered in isolation this means that most of the probabilistic models used by linguists are trained by a version of the EM algorithm. However, this result is even more interesting when considered in a theoretical context because the well-known convergence behavior of the inside-outside algorithm has been confirmed by many experiments but it seems that it never has been formally proved. Furthermore, being a version of the EM algorithm, the inside-outside algorithm also inherits the good convergence behavior of EM. We therefore contend that the yet imperfect line of argumentation can be transformed into a coherent proof

    Environmental Policy (2nd ed.)

    No full text
    Evidence of climate change, resource shortages and biodiversity loss is growing in significance every year. This second edition of Environmental Policy explains how policy can resond and bring about greater sustainability in individual lifestyles, corporate strategies, national policies and international relations. The book discusses the interaction between environmental and human systems, suggesting environmental policy as a way to steer human systems to function within environmental constraints

    Treebank Conversion – Converting the NEGRA treebank to an LTAG grammar –

    No full text
    We present a method for rule-based structure conversion of existing treebanks, which aims at the extraction of linguistically sound, corpus-based grammars in a specific grammatical framework. We apply this method to the NEGRA treebank to derive an LTAG grammar of German. We describe the methodology and tools for structure conversion and LTAG extraction. The conversion and grammar extraction process imports linguistic generalisations that are missing the in original treebank. This supports the extraction of a linguistically sound grammar with maximal generalisation, as well as grammar induction techniques to capture unseen data in stochastic parsing. We further illustrate the flexibility of our conversion method by deriving an alternative representation in terms of topological field marking from the NEGRA treebank, which can be used as input for stochastic topological parsing approaches. On a broader perspective our approach contributes to a better understanding on where corpuslinguistics and theoretical linguistics can meet and enrich each other. 1

    On the statistical consistency of DOP estimators

    No full text
    Given a sample from an unknown probability distribution, a statistical estimator uses the sample in order to guess the unknown distribution. One desired property of an estimator is that its guess is increasingly likely to get arbitrarily close to the unknown distribution as the samples become larger. This property of an estimator is called consistency. Data Oriented Parsing (DOP) employs all fragments of the trees in a training treebank, including the full parse-trees themselves, as the rewrite rules of a probabilistic treesubstitution grammar. Since the most popular DOP-estimator (DOP1) was shown to be inconsistent, there is an outstanding theoretical question concerning the possibility of DOPestimators with reasonable statistical properties. This question constitutes the topic of the current paper. First, we show that, contrary to common wisdom, any unbiased estimator for DOP is futile because it will not generalize over the training treebank. Subsequently, we show that a consistent estimator that generalizes over the treebank should involve a local smoothing technique. This exposes the relation between DOP and existing memory-based models that work with full memory and an analogical function such as k-nearest neighbor, which is known to implement backoff smoothing. Finally, we present a new consistent backoffbased estimator for DOP and discuss how it combines the memory-based preference for the longest match with the probabilistic preference for the most frequent match.

    Inducing Probabilistic Syllable Classes Using Multivariate Clustering

    No full text
    An approach to automatic detection of syllable structure is presented. We demonstrate a novel application of EM-based clustering to multivariate data, exemplied by the induction of 3- and 5-dimensional probabilistic syllable classes. The qualitative evaluation shows that the method yields phonologically meaningful syllable classes. We then propose a novel approach to grapheme-to-phoneme conversion and show that syllable structure represents valuable information for pronunciation systems
    corecore