Position Models and Language Modeling

International audienceIn statistical language modelling the classic model used is $n$-gram. This model is not able however to capture long term dependencies, \emph{i.e.} dependencies larger than $n$. An alternative to this model is the probabilistic automaton. Unfortunately, it appears that preliminary experiments on the use of this model in language modelling is not yet competitive, partly because it tries to model too long term dependencies. We propose here to improve the use of this model by restricting the dependency to a more reasonable value. Experiments shows an improvement of 45\% reduction in the perplexity obtained on the Wall Street Journal language modeling task

Efficient Pruning of Probabilistic Automata

International audienceApplications of probabilistic grammatical inference are limited due to time and space consuming constraints. In statistical language modeling, for example, large corpora are now available and lead to managing automata with millions of states. We propose in this article a method for pruning automata (when restricted to tree based structures) which is not only efficient (sub-quadratic) but that allows to dramatically reduce the size of the automaton with a small impact on the underlying distribution. Results are evaluated on a language modeling task

Use of Grammatical Inference in Natural Speech Recognition

This paper presents the application of stochastic grammatical inference to speech recognition. In speech recognition, the acoustic signal process produces a set of words which are combinating to build sentences. Language models are then used to lead the speech recognition application to the most pertinent combination. Up to now, statistical language models are used. We suggest to use stochastic formal grammars instead of statistical models. Theses stochastic grammars will be build by machine learning algorithms. We will first show that unaided grammatical inference cannot be used for speech recognition. We will then make manifest that smoothing is necessary and show the gain that one can obtain by using a basic smoothing. We finally put up a smoothing technic dedicates to stochastic formal grammars. 2 THE QUALITY CRITERION 1 Introduction Our aim is to use stochastic grammatical inference for natural speech recognition. The main difference between validations of grammatical inference..

Pautomac: A probabilistic automata and hidden markov models learning competition,PAutomaC : a PFA/HMM Learning Competition

Contains fulltext : 122433.pdf (preprint version ) (Open Access

A Markovian approach to the induction of regular string distributions

We propose in this paper a novel approach to the induction of the structure of Hidden Markov Models (HMMs). The notion of partially observable Markov models (POMMs) is introduced. POMMs form a particular case of HMMs where any state emits a single letter with probability one, but several states can emit the same letter. It is shown that any HMM can be represented by an equivalent POMM. The proposed induction algorithm aims at finding a POMM fitting a sample drawn from an unknown target POMM. The induced model is built to fit the dynamics of the target machine observed in the sample. A POMM is seen as a lumped process of a Markov chain and the induced POMM is constructed to best approximate the stationary distribution and the mean first passage times (MFPT) observed in the sample. The induction relies on iterative state splitting from an initial maximum likelihood model. The transition probabilities of the updated model are found by solving an optimization problem to minimize the difference between the observed MFPT and their values computed in the induced model

Přibližná redukce konečných automatů pro detekci útoků ve vysokorychlostních sítích

We consider the problem of approximate reduction of non-deterministic automata that appear in hardware-accelerated network intrusion detection systems (NIDSes). We define an error distance of a reduced automaton from the original one as the probability of packets being incorrectly classified by the reduced automaton (wrt the probabilistic distribution of packets in the network traffic). We use this notion to design an approximate reduction procedure that achieves a great size reduction (much beyond the state-of-the-art language preserving techniques) with a controlled and small error. We have implemented our approach and evaluated it on use cases from Snort , a popular NIDS. Our results provide experimental evidence that the method can be highly efficient in practice, allowing NIDSes to follow the rapid growth in the speed of networks.Článek se zaobírá přibližnou redukcí konečných automatů pro detekci útoků ve vysokorychlostních sítích