Estimation of Stochastic Attribute-Value Grammars using an Informative Sample

We argue that some of the computational complexity associated with estimation of stochastic attribute-value grammars can be reduced by training upon an informative subset of the full training set. Results using the parsed Wall Street Journal corpus show that in some circumstances, it is possible to obtain better estimation results using an informative sample than when training upon all the available material. Further experimentation demonstrates that with unlexicalised models, a Gaussian Prior can reduce overfitting. However, when models are lexicalised and contain overlapping features, overfitting does not seem to be a problem, and a Gaussian Prior makes minimal difference to performance. Our approach is applicable for situations when there are an infeasibly large number of parses in the training set, or else for when recovery of these parses from a packed representation is itself computationally expensive.Comment: 6 pages, 2 figures. Coling 2000, Saarbr\"{u}cken, Germany. pp 586--59

Stochastic Attribute-Value Grammars

Probabilistic analogues of regular and context-free grammars are well-known in computational linguistics, and currently the subject of intensive research. To date, however, no satisfactory probabilistic analogue of attribute-value grammars has been proposed: previous attempts have failed to define a correct parameter-estimation algorithm. In the present paper, I define stochastic attribute-value grammars and give a correct algorithm for estimating their parameters. The estimation algorithm is adapted from Della Pietra, Della Pietra, and Lafferty (1995). To estimate model parameters, it is necessary to compute the expectations of certain functions under random fields. In the application discussed by Della Pietra, Della Pietra, and Lafferty (representing English orthographic constraints), Gibbs sampling can be used to estimate the needed expectations. The fact that attribute-value grammars generate constrained languages makes Gibbs sampling inapplicable, but I show how a variant of Gibbs sampling, the Metropolis-Hastings algorithm, can be used instead.Comment: 23 pages, 21 Postscript figures, uses rotate.st

Content-based Video Retrieval by Integrating Spatio-Temporal and Stochastic Recognition of Events

As amounts of publicly available video data grow the need to query this data efficiently becomes significant. Consequently content-based retrieval of video data turns out to be a challenging and important problem. We address the specific aspect of inferring semantics automatically from raw video data. In particular, we introduce a new video data model that supports the integrated use of two different approaches for mapping low-level features to high-level concepts. Firstly, the model is extended with a rule-based approach that supports spatio-temporal formalization of high-level concepts, and then with a stochastic approach. Furthermore, results on real tennis video data are presented, demonstrating the validity of both approaches, as well us advantages of their integrated us

Reversible stochastic attribute-value grammars

Een bekende vraag in de taalkunde is de vraag of de mens twee onafhankelijke modules heeft voor taalbegrip en taalproductie. In de computertaalkunde zijn taalbegrip (ontleding) en taalproductie (generatie) in de recente geschiedenis eigenlijk altijd als twee afzonderlijke taken en dus modules behandeld. De hoofdstelling van dit proefschrift is dat ontleding en generatie op een computer door één component uitgevoerd kan worden, zonder slechter te presteren dan afzonderlijke componenten voor ontleding en generatie. De onderliggende redenering is dat veel voorkeuren gedeeld moeten zijn tussen productie en begrip, omdat het anders niet mogelijk zou zijn om een geproduceerde zin te begrijpen. Om deze stelling te onderbouwen is er eerst een generator voor het Nederlands ontwikkeld. Deze generator is vervolgens geïntegreerd met een bestaande ontleder voor het Nederlands. Het proefschrift toont aan dat er inderdaad geen significant verschil is tussen de prestaties van de geïntegreerde module en afzonderlijke begrips- en productiecomponenten. Om een beter begrip te krijgen hoe het gecombineerde model werkt, wordt er zogenaamde `feature selectie’ toegepast. Dit is een techniek om de belangrijkste eigenschappen die een begrijpelijke en vloeiende zin karakteriseren op te sporen. Het proefschrift toont aan dat dit met een klein aantal, voornamelijk taalkundig geïnformeerde eigenschappen bepaald kan worden

A Machine learning approach to POS tagging

We have applied inductive learning of statistical decision trees and relaxation labelling to the Natural Language Processing (NLP) task of morphosyntactic disambiguation (Part Of Speech Tagging). The learning process is supervised and obtains a language model oriented to resolve POS ambiguities. This model consists of a set of statistical decision trees expressing distribution of tags and words in some relevant contexts. The acquired language models are complete enough to be directly used as sets of POS disambiguation rules, and include more complex contextual information than simple collections of n-grams usually used in statistical taggers. We have implemented a quite simple and fast tagger that has been tested and evaluated on the Wall Street Journal (WSJ) corpus with a remarkable accuracy. However, better results can be obtained by translating the trees into rules to feed a flexible relaxation labelling based tagger. In this direction we describe a tagger which is able to use information of any kind (n-grams, automatically acquired constraints, linguistically motivated manually written constraints, etc.), and in particular to incorporate the machine learned decision trees. Simultaneously, we address the problem of tagging when only small training material is available, which is crucial in any process of constructing, from scratch, an annotated corpus. We show that quite high accuracy can be achieved with our system in this situation.Postprint (published version