40,182 research outputs found
A linear memory algorithm for Baum-Welch training
Background: Baum-Welch training is an expectation-maximisation algorithm for
training the emission and transition probabilities of hidden Markov models in a
fully automated way.
Methods and results: We introduce a linear space algorithm for Baum-Welch
training. For a hidden Markov model with M states, T free transition and E free
emission parameters, and an input sequence of length L, our new algorithm
requires O(M) memory and O(L M T_max (T + E)) time for one Baum-Welch
iteration, where T_max is the maximum number of states that any state is
connected to. The most memory efficient algorithm until now was the
checkpointing algorithm with O(log(L) M) memory and O(log(L) L M T_max) time
requirement. Our novel algorithm thus renders the memory requirement completely
independent of the length of the training sequences. More generally, for an
n-hidden Markov model and n input sequences of length L, the memory requirement
of O(log(L) L^(n-1) M) is reduced to O(L^(n-1) M) memory while the running time
is changed from O(log(L) L^n M T_max + L^n (T + E)) to O(L^n M T_max (T + E)).
Conclusions: For the large class of hidden Markov models used for example in
gene prediction, whose number of states does not scale with the length of the
input sequence, our novel algorithm can thus be both faster and more
memory-efficient than any of the existing algorithms.Comment: 14 pages, 1 figure version 2: fixed some errors, final version of
pape
A statistical multiresolution approach for face recognition using structural hidden Markov models
This paper introduces a novel methodology that combines the multiresolution feature of the discrete wavelet transform (DWT) with the local interactions of the facial structures expressed through the structural hidden Markov model (SHMM). A range of wavelet filters such as Haar, biorthogonal 9/7, and Coiflet, as well as Gabor, have been implemented in order to search for the best performance. SHMMs perform a thorough probabilistic analysis of any sequential pattern by revealing both its inner and outer structures simultaneously. Unlike traditional HMMs, the SHMMs do not perform the state conditional independence of the visible observation sequence assumption. This is achieved via the concept of local structures introduced by the SHMMs. Therefore, the long-range dependency problem inherent to traditional HMMs has been drastically reduced. SHMMs have not previously been applied to the problem of face identification. The results reported in this application have shown that SHMM outperforms the traditional hidden Markov model with a 73% increase in accuracy
- …