The enormous popularity of Hidden Markov models (HMMs) in spatio-temporal pattern recognition is largely due to the ability to 'learn' model parameters from observation sequences through the Baum-Welch and other re-estimation procedures. In this study, HMM parameters are estimated from an ensemble of models trained on individual observation sequences. The proposed methods are shown to provide superior classification performance to competing methods

Davis, R. I. A.

Lovell, Brian C.

Walder, C. J.

University of Queensland eSpace

IMPROVED CLASSIFICATION USING HIDDEN MARKOV AVERAGING FROM
MULTIPLE OBSERVATION SEQUENCES
Richard I. A. Davis, Christian J. Walder and Brian C. Lovell
Intelligent Real-Time Imaging and Sensing Group,
School of Information Technology and Electrical Engineering,
The University of Queensland, Australia, 4072
{riadavis, walder, lovell}@itee.uq.edu.au
ABSTRACT
The enormous popularity of Hidden Markov models (HMMs)
in spatio-temporal pattern recognition is largely due to the
ability to “learn” model parameters from observation se-
quences through the Baum-Welch and other re-estimation
procedures. In this study, HMM parameters are estimated
from an ensemble of models trained on individual observa-
tion sequences. The proposed methods are shown to provide
superior classification performance to competing methods.
1. INTRODUCTION
The successful application of Hidden Markov Models (HMMs)
to diverse applications such as speech recognition [1, 2]
and gene sequence analysis using profile HMMs [3] demon-
strates the immense utility of the HMM as a workhorse
for spatio-temporal pattern recognition. The usefulness of
the HMM stems from the ability to learn HMM parame-
ters through the Baum-Welch re-estimation procedure, and
to provide a form of context handling in pattern recognition
tasks.
In 1993, Rabiner and Juang [1] described a method where
K observation sequences are used at each step of the Baum-
Welch re-estimation procedure to produce a single HMM
parameter estimate. This and other methods only guarantee
a local maxima in model quality, leaving open the possi-
bility of finding superior methods, especially for HMMs of
specific structure.
In this paper a class of new estimation methods are pro-
posed where the Baum-Welch re-estimation procedure is
run separately to completion on the K observations and the
parameters then combined to yield a single estimate. This
technique expected to yield improvements because many
convergence runs were used, with the results being com-
bined using a range of methods. Such a methodology was
suggested by Mackay [5] in his study of ensemble learn-
ing, but does not appear to have been investigated further.
The methods used here were motivated by techniques for
avoiding local minima in the context of Bayesian networks
investigated by Elidan et al. [6].
Previous work by the authors [4] had investigated the
performance of a range of algorithms for matching the out-
puts of a given HMM. Because classification is such an im-
portant application of HMMs, this paper focuses on the per-
fomance of the trained models on a classification task. The
results are in agreement with those in [4].
2. HMM PARAMETER ESTIMATION FROM
MULTIPLE OBSERVATIONS
A hidden Markov model ([1], Chapter 6) consists of a set
of N nodes, each of which is associated with a set of M
possible observations. The parameters of the model include
an initial state
pi = [p1, p2, p3, ..., pN ]
T
with elements pn, n ∈ [1, N ] which describes the distribu-
tion over the initial node set, a transition matrix
A =


a11 a12 . . . a1N
a21 a22 . . . a2N
.
.
.
.
.
.
.
.
.
.
.
.
aN1 aN2 . . . aNN


with elements aij with i, j ∈ [1, N ] for the transition prob-
ability from node i to node j conditional on node i, and an
observation matrix
B =


b11 b12 . . . b1M
b21 b22 . . . b2M
.
.
.
.
.
.
.
.
.
.
.
.
bN1 bN2 . . . bNM


with elements bim for the probability of observing symbol
m ∈ [1,M ] given that the system is in state i ∈ [1, N ].
Rabiner and Juang denote the HMM model parameter set
by λ = (A,B, pi).
The model order pair (N,M) together with additional
restrictions on allowed transitions and emissions defines the
structure of the model (see figure 1 for an illustration of two
different transition structures).
The Baum-Welch algorithm is an “iterative update” al-
gorithm which re-estimates parameters of a given HMM to
produce a new model which has a higher probability of gen-
erating the given observation sequence. This re-estimation
procedure is continued until no more significant improve-
ment in probability can be obtained and the local maxi-
mum is thus found. The re-estimation procedure was ini-
tiated with a random set of HMM parameters that matched
the known structural constraints on the HMM (specifically
Left-Right models only). This guarantees that every step of
the convergence will generate a left-right model. However
similar results were obtained when no transition constraints
were placed upon the initial generating model.
For example, if it is known that the model is left-right
and always starts in state 1, set pi = [1, 0, 0, . . . , 0]T and A
to upper-triangular. Other elements of A and B are then set
to random values with the constraint that the row sums must
equal unity to ensure that rows can be treated as probability
mass functions. Note that elements of the matrices A and B
that are set to zero will remain zero after re-estimation due
to the nature of the algorithm. Hence the structure is pre-
served throughout the procedure. If the starting node is not
known, then initially the elements of pi can be set randomly
with the constraint that the sum must equal unity as before.
After the re-estimation runs to completion, one element, pij
say, usually becomes close to unity and all other elements
usually become negligible. The index j would then repre-
sent an estimate of the starting node for this sequence.
2.1. Rabiner and Juang Method and Variants
Now consider the case where K observation sequences are
known to be generated by the same HMM and the objective
is to determine the HMM parameters that yield high proba-
bility of generating all K observed sequences.
Rabiner and Juang [1] proposed just such a multi-sequence
training algorithm using the K observation sequences at
each stage of the Baum-Welch re-estimation to iteratively
update a single HMM parameter set. The re-estimation for-
mula for this type of iterative update method is as follows
(reproduced from [1]):
aij =
∑
k Wk
∑Tk
t=1 α
k
i aijbj(O
(k)
t+1)β
(k)
t+1(j)∑
k Wk
∑Tk
t=1 a
k
t (i)β
k
t (i)
(1)
bij =
∑
k Wk
∑
Ot(k)=vj
αkt (i)β
k
t (i)∑
k Wk
∑Tk
t=1 α
k
t (i)β
k
t (i)
(2)
where Wk = 1/Pk, k ∈ [1 . . . k] is the inverse of the prob-
ability of the current model estimate generating training se-
quence k, evaluated using the forward algorithm [1].
a) Cyclic Bias b) Left-Right
Figure 1: Cyclic and Left-Right structures showing allowed
transitions. Bold arrows indicate higher probabilities.
The form of the re-estimation relation for pi depends
upon the model structure and is trivial in the case of left-
right models (the only case covered in [1]), assuming pi =
[1, 0, 0, . . . , 0]T . For other models, such as cyclic (see figure
1) one method is to run the Baum-Welch re-estimation pro-
cedure to completion on each of the K observations to es-
timate the starting node as above and take a simple average
of the starting node distribution over all sequences. (Note
that if the source model were known, it would be simple use
the Viterbi algorithm [1] to estimate the starting node)
2.2. Proposed Ensemble Methods
The second approach described here is a special case of
the method suggested by Mackay [5] where an ensemble
of models is trained. In this paper one model is estimated
for each of the K observation sequences (other approaches
are possible but are not considered here). This enables the
formation of K independent model estimates from the train-
ing sequences. From these the next step is to examine the
efficacy of combining the independent parameter estimates
using a range of simple averaging techniques of the follow-
ing form:
aij =
∑
k Wka
(k)
ij∑
k Wk
(3)
bij =
∑
k Wkb
(k)
ik∑
k Wk
(4)
pii =
∑
k Wkpi
(k)
i∑
k Wk
(5)
where Wk is the weighting factor for each sequence and
λ(k) = (A(k), B(k), pi(k)). The quality of all model esti-
mates is judged by the probability of that model generating
an unseen set of Q test sequences from the same source as
the K training sequences as described below.
An alternative involving averaging numerator and de-
nominator before combining and normalising was also in-
vestigated, but produced estimators with performance much
the same as Mackay’s proposed method above.
3. METHODS INVESTIGATED AND
PERFORMANCE
Simple benchmark methods:
• Random: Model constructed from randomly selected
parameters
• Rabiner’s Wk = 1/Pk Estimation Rabiner and Juang’s
standard method [1] with their suggested Wk = 1/Pk
weighting.
• Unit Weight: Simple ensemble averaging over all K
models,
• Permuted Unit Weight: Random permutations ap-
plied to the nodes of the HMM ensemble prior to
paramter averaging
Method evaluation is performed using Monte Carlo sim-
ulations to produce random sources, training sequences, and
test sequences. Only the performance of the models on the
unseen test sequences is presented here as a measure of true
learning ability.
For each model λˆk inferred from a sequence Sk, k ∈
[1 . . . K] through the Baum-Welch restimation procedure,
there is an associated probability Pˆk of that inferred model
producing the sequence Sk. Similarly, define Pˆ allk as the
probability (calculated using the Forward algorithm) of that
model λk generating all sequences Sk. Then define the fol-
lowing probabilities:
Pˆ lk = P (l
thtraining seq. | model for seq.k)
Pˆk = P (k
thtraining seq. | model for seq.k)
Pˆ allk = P (all K training seq. | model for seq.k)
=
K∏
l=1
Pˆ lk
Pˆ all = P (all Q test seq. | estimated model)
Ptrue = P (all Q test seq. | true source model)
Note that for Rabiner and Juang’s single model tech-
nique of (1) and (2), Pk is calculated during the re-estimation
procedure is thus not the same as Pˆk above which arises af-
ter convergence of the re-estimation procedure on each of
the K sequences.
It is important to distinguish between Pˆall for the test
sequence set, and Pˆall for the training sequence set. This
distinction is always made clear in the text of this paper.
A broader comparison of averaging methods on a se-
quence set approximation task may be found at [4].
4. PERFORMANCE OF NEW METHODS ON A
CLASSIFICATION TASK
A study of the performance of the new HMM estimation
procedure on a simple classification task was also conducted
using the Bayes Net Toolbox [7]. The true and test models
had 4 states, 8 observation symbols, and observations of se-
quence length 14 were used for both training and testing.
The trial was conducted by generating two true mod-
els and corresponding training and testing sets. Succes-
sively larger subsets of the training set were formed, and the
models were trained on these subsets. Classification perfor-
mance was then tested (with a testing set size of 200) on the
models trained on each subset. This enabled the plotting of
the number of correct classifications against training set size
on our curve for each training method.
All the above steps were completed five times for five
randomly chosen source models, and the mean of the curves
is shown in figure 2. No structure was imposed on any of
the models: all parameters were uniformly randomly chosen
(an ergodic model distribution).
Unit merge training was done with random initialisa-
tion of each model before training and averaging. Another
method was to randomly permute nodes relative to the nodes
of other models before averaging. This experiment was mo-
tivated by the fact that we can often relabel (permute) the
states of HMMs to produce equivalent models with identical
probability but with quite different parameters (A,B, pi).
We found that if we randomly relabel in this manner before
parameter averaging, we obtain HMMs that perform just as
well. We believe that this is due to the fact that random ini-
tialization effectively includes a random permutation step -
so a random permutation after initialization is complete will
not change the average result overall. This issue will be in-
vestigated further in later work to see if small gains can be
made by maximizing over the set of all relative permuta-
tions.
The random untrained models achieved about 50% cor-
rect as expected. The true generating model only achieved
about 80% correct. Averaging methods approach true clas-
sification performance with about only 10 sequences; Ra-
biner’s method required about 70 sequences to achieve the
same level of performance. Hence learning for classifica-
tion is more efficient for the proposed averaging methods in
terms of the required number of training sequences but the
asymptotic performance is similar (this was expected since
it is known that the Rabiner method is quite effective at clas-
sification).
The results of this trial are consistent with the results of
learning for maximum Pˆall described in the previous sec-
tion.
0 10 20 30 40 50 60 70 80 90 100
40
45
50
55
60
65
70
75
80
M
ea
n 
Pe
rc
en
t C
or
re
ct
Number of Training Examples
True
Rabiner
Unit Weight
Permuted Unit Weight
Random
Figure 2: Classification trial (Lseq = 14).
10 20 30 40 50 60 70 80 90 100
−1110
−1100
−1090
−1080
−1070
−1060
−1050
−1040
−1030
Top percentile of models used in ensemble average
Lo
g 
pr
ob
ab
ilit
y o
n 
un
se
en
 d
at
a
Quality of averaged model against number of models in ensemble
Figure 3: Mean values of Pˆall on test sequences when train-
ing on long training sequences (Lseq = 30).
5. EFFECT OF INCLUDING MORE MODELS
A trial was designed to study the benefits of including extra
converged models in the ensemble to be averaged. Figure 3
plots the Pˆall values (defined as the log probability of gen-
erating a set of unseen test sequences against the number of
models in the ensemble as additional (poorer) models are
added to the ensemble. Models were combined using the
unit weighting technique.
The continual improvement in performance with increas-
ing ensemble size is a general overall trend which was con-
sistently seen. However, in this and other cases, including a
single new model or even up to 5 models in rank sequence
sometimes reduces the overall model quality. It is hypothe-
sized that this is due to the potential ambiguity for relative
permutations of the HMM nodes.
6. CONCLUSIONS
The proposed Unit Weighted Ensemble averaging method
for HMM parameter estimation from multiple observation
sequences appears to offer significantly more probable model
estimates on unseen data than the well-known method of
Rabiner and Juang [1]. The classification trial demonstrated
that fewer training sequences were required to achieve the
same level of classification accuracy with the averaging method.
7. FUTURE WORK
Significant work remains to be done on the structure of a
Windsorized algorithm and its interaction with weighting
schemes and relative node permutation. It may be possible
to design an algorithm which selects which method to use
based upon sequence length and other parameters, thereby
producing a method optimised for all sequence lengths. Fi-
nally these methods will be tested on a range of important
practical problems in fields such as gene sequence analysis
where precise learning (particularly from limited numbers
of sample sequences) is important. Ultimately, this should
lead to a comprehensive study of the parallel re-estimation
problem and provide a theoretical framework to underpin
future development.
8. REFERENCES
[1] L. R. Rabiner, B. H. Juang, Fundamentals of Speech
Recognition New Jersey Prentice Hall, 1993.
[2] L. R. Rabiner, S. E. Levinson, and M. M. Sondhi, An
Introduction to the Application of the Theory of Prob-
abilistic Functions of a Markov Process to Automatic
Speech Recognition, The Bell System Technical Jour-
nal 1035-1074, Vol. 62, No. 4, April 1983.
[3] S. R. Eddy, “Profile hidden markov models.” Bioinfor-
matics Vol 14:755-763, 1998.
[4] R. I. A. Davis and B. C. Lovell, “Improved Estima-
tion of Hidden Markov Model Parameters from Mul-
tiple Observation Sequences”, International Congress
on Pattern Recognition, Quebec City (2002)
[5] D. J. C. Mackay, “Ensemble Learning for Hidden
Markov Models”, Technical report, Cavendish Labo-
ratory, University of Cambridge, 1997.
[6] G. Elidan, M. Ninio, N. Friedman, and D. Schuurmans
“Data perturbation for escaping local maxima in learn-
ing.” Proceedings of the Eighteenth National Confer-
ence on Artificial Intelligence (AAAI-02), (2002)
[7] IRIS source for estimating Hidden Markov Models.
http://www.itee.uq.edu.au/ iris/CVsource/source.html


Improved Classification Using Hidden Markov Averaging From Multiple Observation Sequences

The enormous popularity of Hidden Markov models (HMMs)  in spatio-temporal pattern recognition is largely due to the ability to &quot;learn&quot; model parameters from observation sequences through the Baum-Welch and other re-estimation procedures. In this study, HMM parameters are estimated from an ensemble of models trained on individual observation sequences. The proposed methods are shown to provide superior classification performance to competing methods

Richard I. A. Davis

Christian J. Walder

Brian C. Lovell

CiteSeerX

Improved Classification Using Hidden Markov Averaging from Multiple Observation Sequences

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.58.8480

Improved Classification Using Hidden Markov Averaging From Multiple Observation Sequences

Abstract

Similar works

Full text

Available Versions

University of Queensland eSpace

CiteSeerX