Inducing Probabilistic Grammars by Bayesian Model Merging

We describe a framework for inducing probabilistic grammars from corpora of positive samples. First, samples are {\em incorporated} by adding ad-hoc rules to a working grammar; subsequently, elements of the model (such as states or nonterminals) are {\em merged} to achieve generalization and a more compact representation. The choice of what to merge and when to stop is governed by the Bayesian posterior probability of the grammar given the data, which formalizes a trade-off between a close fit to the data and a default preference for simpler models (`Occam's Razor'). The general scheme is illustrated using three types of probabilistic grammars: Hidden Markov models, class-based $n$-grams, and stochastic context-free grammars.Comment: To appear in Grammatical Inference and Applications, Second International Colloquium on Grammatical Inference; Springer Verlag, 1994. 13 page

Optimization of the Morpher Morphology Engine Using Knowledge Base Reduction Techniques

Morpher is a novel morphological rule induction engine designed and developed for agglutinative languages. The Morpher engine models inflection using general string-based transformation rules and it can learn multiple arbitrary affix types, too. In order to scale the engine to training sets containing millions of examples, we need an efficient management of the generated rule base. In this paper we investigate and present several optimization techniques using rule elimination based on context length, support and cardinality parameters. The performed evaluation tests show that using the proposed optimization techniques, we can reduce the average inflection time to 0.52 %, the average lemmatization time to 2.59 % and the number of rules to 2.25 % of the original values, while retaining a high correctness ratio of 98 %. The optimized model can execute inflection and lemmatization in acceptable time after training millions of items, unlike other existing methods like Morfessor, MORSEL or MorphoChain

Constructing Deterministic ?-Automata from Examples by an Extension of the RPNI Algorithm

The RPNI algorithm (Oncina, Garcia 1992) constructs deterministic finite automata from finite sets of negative and positive example words. We propose and analyze an extension of this algorithm to deterministic ?-automata with different types of acceptance conditions. In order to obtain this generalization of RPNI, we develop algorithms for the standard acceptance conditions of ?-automata that check for a given set of example words and a deterministic transition system, whether these example words can be accepted in the transition system with a corresponding acceptance condition. Based on these algorithms, we can define the extension of RPNI to infinite words. We prove that it can learn all deterministic ?-automata with an informative right congruence in the limit with polynomial time and data. We also show that the algorithm, while it can learn some automata that do not have an informative right congruence, cannot learn deterministic ?-automata for all regular ?-languages in the limit. Finally, we also prove that active learning with membership and equivalence queries is not easier for automata with an informative right congruence than for general deterministic ?-automata

Constrained domain maximum likelihood estimation and the loss function in statistical pattern recognition

In this thesis we present a new estimation algorithm for statistical models which does not incurs in the over-trainning problems. This new estimation techinque, the so-called, constrained domain maximum likelihood estimation (CDMLE) holds all the theoretical properties of the maximum likelihood estimation and furthermore it does not provides overtrained parameter sets. On the other hand, the impliations of the the 0-1 loss function assumption are analysed in the pattern recognition tasks. Specifically, more versatile functions are designed without increasing the optimal classification rule costs. This approach is applied to the statistical machine translation problem.Andrés Ferrer, J. (2008). Constrained domain maximum likelihood estimation and the loss function in statistical pattern recognition. http://hdl.handle.net/10251/13638Archivo delegad