In this paper we present a new approach for topic spotting based on subword units and feature vectors instead of words. In our first approach, we only use vector quantized feature vectors and polygram language models for topic representation. In the second approach, we use phonemes instead of the vector quantized feature vectors and model the topics again using polygram language models. We trained and tested the two methods on two different corpora. The first is a part of a media corpus which contains data from TV shows for three different topics. The second is the VERBMOBIL-corpus where we used 18 dialog acts as topics. Each corpus was splitted into disjunctive test and training sets. We achieved recognition rates up to 82% for the three topics of the media corpus and up to 64% using 18 dialog acts of the VERBMOBIL-corpus as topics

Harbeck, Stefan

Niemann, Heinrich

Nöth, Elmar

Warnke, V.

Scientific documents from the Saarland University,

Gafiirdart yom 
.bmb+f 
Bundasministerium fur 
Bildung. WissanschBft. 
Forschung und Technologie 
erbmobil 
Verbundvorhaben 
Topic Spotting Using Subword Units. 
V. Warnke, S. Harbeck, 
H. Niemann, E. N6th 
F.-A.-Universitat Erlangen-Nurnberg 
Report 205 
Marz 1997 
Marz 1997 
V. Warnke, S. Harbeck, 
H. Niemann, E. Noth 
Lehrstuhl flir Mustererkennung (Inf. 5) 
Friedrich-Alexander-U niversi tat Erlangen-N iirnberg 
Martensstr. 3 
D-91058 Erlangen 
Tel.: (09131) 85 - 7883 
e-mall: warnke(Qinformatik. uni -erlangen. de 
Gehort zum Antragsabschnitt: 4.8 
Das diesem Bericht zugrundeliegende Forschungsvorhaben wurde mit Mitteln 
des Bundesministers flir Bildung, Wissenschaft, Forschung und Technologie 
unter dem Forderkennzeichen 01 IV 701 K/5 gef6rdert. Die Verantwortung 
flir den Inhalt dieser Arbeit liegt bei den Autoren. 
From: Tagung,band de, 9. Aachener Kolloquiums "Signaltheorie", pages 2.87 - 2.90 
Topic Spotting Using Subword Units 
V. Warnke, S. Harbeck, E. Noth, H. Niemann 
Universitat Erlangen-Niirnberg, Lehrstuhl fUr Mustererkennung, 
Martensstr. 3, 
91058 Erlangen, Germany 
Tel.: +49 9131 / 857883 
Fax.: +49 9131 / 303811 
warnke@informatik.uni-erlangen.de http://www5.informatik.uni-erlangen.de 
Abstract 
In this paper we present a new approach for 
topic spotting based on subword units and fea-
ture vectors instead of words. In our first 
approach, we only use vector quantized fea-
ture vectors and polygram language models 
for topic representation. In the second ap-
proach, we use phonemes instead of the vec-
tor quantized feature vectors and model the 
t9pics again using polygram language models. 
We trained and tested the two methods on two 
different corpora. The first is a part of a me-
dia corpus which contains data from TV shows 
for three different topics. The second is the 
VERBMOBIL-corpus where we used 18 dialog 
acts as topics. Each corpus was splitted into 
disjunctive test and training sets. We achieved 
recognition rates up to 82% for the three top-
ics of the media corpus and up to 64% using 
18 dialog acts of the VERBMOBIL-corpus as 
topics. 
1. Introduction 
In most approaches in the field of topic spot-
ting, words or word sequences are used for 
identifying a topic [5J. This is done by word 
recognition with large vocabularies or special 
word spotters [9]. To train such recognizers in 
both cases a huge amount of tediously labeled 
data has to be available. 
For the training of our topic spotter with 
phonemes and vector quantized feature vec-
tors we do not need the data to be labeled as 
exactly as for training a word spotter or a word 
recognition system. We only need the speech 
signals labeled with their topic rather than a 
word-to-word transliteration. Using either a 
vector quantizer (see section 2.1.) or phoneme 
segmentizer (see section 2.2.), we segment the 
speech signal into a symbol sequence. With 
these sequences we train stochastic language 
models (LMs) for each of the topics to be iden-
tified. In test phase we run all LMs in parallel 
and decide for the topic with the maximum a 
posteriori probability (see section 2.3 .). 
The advantage of our approach is evident 
when changing to a new domain. Doing topic 
spotting with a large vocabulary speech rec-
ognizer, one has to adapt the lexicon if not 
retrain all the acoustic models with domain 
dependent transliterated speech. With a word 
spotter) new keywords have to be identified 
and trained. In our approach only the lan-
guage models have to be retrained. The train-
ing labels, i.e. the assignment of the topic 
to the training utterances, can be done very 
fast. Using the vector quantizer; one can even 
switch the language without the need for a 
more detailed labelling of the training data. 
For our experiments we understand a topic in 
two different ways: in the first case a topic 
is a theme of a TV show (IDS-corpus) and in 
the second we understand a dialog act [2] as a 
topic (VERB MOBIL-COrpus) (see section 3.). 
2. Methods Used 
2.1. Feature extraction and vec-
tor quantization 
For representing the speech signal as feature 
vectors, we use the mel-frequency-cepstrum-
coefficients (MFCCs). A feature vector c is 
calculated for a 10ms part of the speech sig-
nal and contains the energy and the first 11 
MFCCs. These features and their first deriva-
tives are used as an input to a phoneme seg-
mentizer in the phoneme based approach. In 
the vector quantization approach eight neigh-
boring feature vectors c (without the deriva-
tives) are concatenated to a new feature vec-
tor c, which describes a context of 80ms with 
96 coefficients. We use this time window, be-
cause the average length of a german phoneme 
is about 80ms. With this feature vector we 
calculate an initial codebook q( 0) with 240 
classes. Using the linear discriminant analysis 
(LDA) on the codebook q(O) we optimize the 
intra- and inter-class-distance of the 240 code-
book classes and transform the feature vector 
c with the 96 components in a new smaller vec-
training data feature quantization 
I extraction LDA topic 1 codebook 
q(C) 
I with K topic N classes 
Figure 1: Partioning of the feature space with fea-
ture reduction using LDA 
tor c with 24 components. This feature vector 
is used to train a new codebook q( C) with 65 
classes (see figure 1). For training the code-
book we use the well known LEG-algorithm 
[4], which minimizes the expected quantiza-
tion error 
E = £[d(C,q(C))J (1) 
With the resulting codebook we are able to 
segment the utterances of the trainmg and test 
data into equidistant code book class sequences 
(CCSs) with a segment length of 80ms. Using 
these CCSs we train a LM for every topic of 
interest . 
2.2. Phoneme segmentation 
In the phoneme based approach we use 
phoneme sequences instead of CCSs. Our 
phoneme segmentizer works HMM-based. Ev-
ery phoneme is modeled by one left-right-
semicontinous HMM. All HMMs are merged 
together to a compound-HMM-network with 
one start and one end state. Thus it is possi-
ble to segment an utterance and calculate the 
corresponding phonemes in one pass using the 
one-pass-algorithm as described in [3J . The 
inventory of our phoneme segmentizer is 62 
phonemes and 3 silence models . We trained 
this phoneme segmentizer with the training 
data from the VERBMOBIL-corpus and used 
it to segment the data from the IDS-corpus as 
well. 
We can see the phoneme segmentizer (just aE 
the vector quantizer) as a signal-to-symbol 
transformation. The differences between the 
phoneme classes and the codebook classes is 
that codebook classes have no direct acoustic-
phonetic meaning. Notice, that we keep the 
size of the input feature vector (24), the num-
ber of phonemes/codebook classes (65), and 
the average window size (80ms) in the same 
order. Thus we can see how much we loose 
in performance, when working with the unsu-
pervised vector quantization rather than the 
supervised phoneme segmentation. 
2.3. Polygram Language Model 
In most cases language models are used to 
calculate the probability of a word sequence 
W = WI ... Wm in a given language or context. 
We use polygram language models [7J, which 
are a special kind of stochastic language mod-
els to calculate the probability of a symbol se-
quence where a symbol could be a phoneme or 
a codebook class. 
Using polygrams the probability of the symbol 
sequence WI . . . Wm is calculated with the help 
of 
m 
P(WI) . II P(wn I WIW2 ... Wn-2Wn-l) (2) 
n=2 ' v ' 
history 
Because the younger history Wm-n+1 ... Wm-I 
of the symbol sequence WI " . Wm is more im-
portant for modeling and to restrict the num-
ber of free parameters inside the LM, we only 
use the last n - 1 symbols instead of the whole 
history. 
P(Wm I WI .. , Wm-I) ~ 
P(wm I Wm - n +1 ... Wm-I) (3) 
, v ~ 
(n-I) 
With this shorter history we can estimate 
P(wm I wm- n+1'" wm-d from a given train-
ing corpus using the interpolation scheme 
A (I ) #(WI ..• wm ) 
P Wm WI··· Wm-I = ( )' # WI" ,Wm-I (4) 
where # is a function which counts how often 
a symbol sequence is seen in the training data. 
To handle symbol sequences that were not seen 
in the training data we need an interpolation 
formalism. 
Linear interpolation 
The first interpolation method we use is the 
linear interpolation [7J (L = lexicon size): 
n 
- 1 
P(wm I WI" .wm-d = Po' L 
+PI . p(wm } 
+P2 . p(wm I Wm-I) 
+ L Pi . p(Wm I Wm-i+l ... Wm-I). (5) 
i=3 
The interpolation coefficients Pi can be es-
timated using the Expectation Maximization 
(EM) algorithm [6J on a given validation set. 
U sing this method an unseen symbol sequence 
is modeled by its subsequences weighted with 
the interpolation coefficients. 
Rational interpolation 
The rational interpolation method is the sec-
ond interpolation we apply [7]: 
P(wm I WI .. , Wm-l) = 
utterance 
I ...-____ ----icqdebook class sequence h 1 
I I I 
,------ - --, 
12 1 45 37 23 62 4 
37 23 62 4 43 17 23 
4 43 17 23 46 57 54 
46 57 54 3 4 21 25 
LM training sequence 
, 
I I 
Figure 2: SI?litting a ees w into subsequences 
wlth step S = 3 and window W = 7 
Lf=lPi' (l/L)n-i. #dWI" ,Wm-IWm) 
E?=IPi' (l/L)n-i. #i(WI ... wm-d ' (6) 
where #i counts the i last predecessors 
(Wm-i ... Wm-l) in a given sequence w. In 
this interpolation formalism it is also possi-
ble to estimate the interpolation coefficients 
using the EM-algorithm on a given validation 
set. This interpolation gives more weight to 
the symbol sequences which have often been 
present in the training data and are in the 
nearest neighborhood of the observed symbol. 
3. Corpora 
3.1. IDS-Corpus 
We performed experiments for both ap-
proaches with the data from the IDS-corpus 
(Institut fUr Deutsche Sprache). This corpus 
contains data from German TV shows for the 
three topics speech, politics, and culture. It 
is a small part of the IDS-media-corpus with 
more than about 200 hours of spoken data. 
It was not easy to say, which TV show cor-
responds to which topic. So we assigned, for 
example, one TV show to the topic "politics", 
if it was announced that it is a political dis-
cussion about the "gulf war". All utterances of 
this TV show were then assigned to the topic 
"politics" . 316 utterances from 11 different 
TV shows were divided into speaker disjunc-
tive training (250 utterances) and test sets (66 
utterances). The length of an utterance is 39 
seconds in the average. 
3.2. VERBMOBIL-Corpus 
VERBMOBIL is a speech-to- speech transla-
tion project [1 J in the domain of appointment 
scheduling, i.e. two persons try to fix a meet-
ing date, time, and place. One of the tasks 
of the dialog module within VERBMOBIL is to 
LM training sequences multigram model 
1" ...... -:===::. polygram count ..-----... 
[ i~~~:;~f. }--I '-__ L_M_l _-,I 
l i~~~~~~~. }-- LI __ L_M_N_--,I 
Figure 3: Training of LMs for the topics Ti 
keep track of the state of the dialog in terms 
of dialog acts. Dialog acts are e.g., "greet-
ing", "confirmation of a date", l'suggestion of 
a place" . In VERBMOBIL a turn of one user 
can consist of more than one dialog act. Cur-
rently, the processing is done in two steps: 
first, the utterance is segmented into dialog 
act units. Second, these units are classified 
into dialog acts. In VERBMOBIL there are 19 
dialog acts. In our experiments we recognize 
18 of these dialog acts as topics l • We used 
3284 dialog acts for training and 1064 dialog 
acts for testing. For the experiments described 
below we used a hand segmentation of turns 
into dialog acts. 
4. Topic spotting 
4.1. Preparing the Sequences 
When we transform an utterance into a sym-
bol sequence W = WI ... Wm we can train and 
test our topic spotter with the whole sym-
bol sequence or we can observe the seluence 
through a sliding window with window ength 
Wand step S (see figure 2) to get subse-
quences w. Thus we are able to get a result 
at each step of the recognition task and ob-
serve the provisional results. 
4.2. Using Language Models 
Using the sequence W or w we trained a poly-
gram language model for each topic (see fig-
ure 3). For the classification task, we combine 
all the polygram models together to one multi-
gram model. This multigram model contains 
the polygram models for all topics. Running 
the multigram model means running all poly-
gram models in parallel to calculate the a pos-
teriori probabilities P(Ti I w) for each topic 
Ti E T of interest 
We decide for the topic with the maximum a 
posteriori probability. For the whole sequence 
W (if we obsere w) we accumulate the hits for 
each subsequence wand decide for the topic 
which receives the most hits. 
lOne dialog act was only present a few times in the training 
set and could thus not be trained. 
5. Experiments and Results 
Using the data from the IDS-corpus we cal-
culated cess and used different length W for 
our sliding window. The step was in all ex-
periments set to S = 1. So we were able 
to get a result for every 80ms of the ut-
terance with the actual context of the win-
dow W. Table 1 shows the best results we 
achieved for different sizes of Wand interpola-
tion methods using polygrams of length three. 
We performed the same experiments with the 
code book class sequences wdh trtgram - LM 
74 77 70 81 70 74 78 85 
68 68 56 75 56 63 50 87 
II 67 I 65 II 64 I 71 II 65 65 71 80 
Table 1: Recognition results in percent for cess 
(L =linear interpolation, R =rational 
interpolation) 
phoneme sequences. Here the window size W 
(W phonemes) is only approximately constant 
over time. We again reached the best results 
when we used the maximum context of three. 
The results for this experiment are given in ta-
ble 2. We compared the results for our auto-
phoneme sequences w~th trtgram - LM 
81 81 85 89 79 74 81 
81 100 81 94 81 75 81 
II 79 I 73 II 82 I 77 82 76 79 77 
Table 2: Recognition results in percent for 
phoneme sequences 
matically segmentized symbol sequences with 
LMs trained on the transliterated word se-
quences (simulating 100% word recognition) 
which reached a recognition result of 89% for 
a sliding window of W = 10 and 85% for the 
whole utterance. Thus we can see that is pos-
sible to classify the three topics of the IDS-
corpus using subword units and polygram lan-
guage models. 
Using the VERBMOBIL-corpus as database we 
only did experiments with the phoneme se-
quences. At this point of our research we 
only used the spoken phoneme sequences of 
the 18 dialog acts for training and testing. We 
calculated the a posteriori probability for the 
whole symbol sequences of an utterance. Thus 
we reached a recognition rate of 64% for the 
18 dialog acts when we used the phoneme se-
quences, a rational interpolation method and 
maximum poly_gram size of four for the lan-
guage models. We also trained a topic spotter 
on the transliterated word sequence as we did 
for the IDS-corpus. Here we reached a recog-
nition rate of 63%. This shows that (for the 
given training data) we can model the topics 
better with phoneme sequences than with the 
word sequences. One reason is that, e.g. the 
words appointment and appointments are dif-
ferent for the word based LMs, but the stem 
(appointment) is equal for the phoneme based 
LMs. A more detailed discussion of the exper-
iments is given in [8]. 
6. Conclusion 
We presented two approaches for topic spot-
ting with the use of subword units. Because 
we use vector quantization and a domain in-
dependent phoneme segmentizer, we only need 
the utterances of the training set labeled with 
their topics. The main difference between 
both approaches is the fact, that we use cess 
of equidistant length in our vector quantizer 
and a symbol sequence of variable length in 
our phoneme based approach. In both ap-
proaches the topics are modeled with the help 
of stochastic language models. During the 
recognition task for all topics the a posteri-
ori probability is calculated in parallel and the 
topic with the maximum probability is cho-
sen. We showed that depending on the defini-
tion of topic and the amount of training data 
our approach performed almost as good or bet-
ter than spotter which uses a "perfect" word 
recognition module. 
7. References 
1. T. Bub and J. Schwinn. Verbmobil: The Evolution of a 
Complex Large Speech-to-Speech Translation System. In 
Int . Conf. on Spoken Language Proces&ing, volume 4, pages 
1026-1029 , Philadelphia, 1996. 
2. S. J ekat , A. Klein, E. Maier, 1. Maleck, M. Mast, and 
J. Quantz . Dialogue Acts in Verbmobil. Verbmobil Re-
port 65 , 1995. 
3. T. Kuhn. Die Erkennungsphase in einem Dialogsllstem, vol-
ume 80 of Dissertationen zur Kunstlichen Intelligenz. Infix, 
St. Augusti n, 1995. 
4. Y . Linde, A. Buzo, and R .M. Gray. An Algorithm for Vec-
tor Quantizer Design. IEEE Trans. on Communications, 
28(1): 84- 95, 1980. 
5. E . Parris and M. Carrey. Topic spotting with task indepen-
dent Models. In Proc. European Conf. on Speech Commu-
nication and Technology, pages 2133-2136, Madrid, Spain, 
September 1995. 
6. E . G. Schukat-Talamazzini . Automatische Spracherkennung 
- Grundlagen, statistische Modelle und ejJiziente Algorith-
men. Vieweg, Braunschweig, 1995. 
7. E.G . Schukat-Talamazzini . Stochastic Language Models. In 
Electrotechnical and Computer Science Conference, Por-
toroi, Slovenia, 1995. 
8. V. Warnke. Topik- und Topikgrenzen-Spotting. Diplomar-
beit , Lehrstuhl fur Mustererkennung (Informatik 5), Univer-
sita t Erlangen ·Nurnberg, 1996. 
9. M. Wint raub . Keyword-spotting using sri's decipher large 
vocabulary speech recognition system. In Proceedings Inter-
national Conference on Automatic Speech and Signal Pro-
cessing, pages 463-466, 1993. 


Topic spotting using subword units

Universaar

Acronym

Gafiirdart yom .bmb+f Bundasministerium fur Bildung. WissanschBft. Forschung und Technologie erbmobil Verbundvorhaben Topic Spotting Using Subword Units. V. Warnke, S. Harbeck, H. Niemann, E. N6th F.-A.-Universitat Erlangen-Nurnberg Report 205 Marz 1997 Marz 1997 V. Warnke, S. Harbeck, H. Niemann, E. Noth Lehrstuhl flir Mustererkennung (Inf. 5) Friedrich-Alexander-U niversi tat Erlangen-N iirnberg Martensstr. 3 D-91058 Erlangen Tel.: (09131) 85 - 7883 e-mall: warnke(Qinformatik. uni -erlangen. de Gehort zum Antragsabschnitt: 4.8 Das diesem Bericht zugrundeliegende Forschungsvorhaben wurde mit Mitteln des Bundesministers flir Bildung, Wissenschaft, Forschung und Technologie unter dem Forderkennzeichen 01 IV 701 K/5 gef6rdert. Die Verantwortung flir den Inhalt dieser Arbeit liegt bei den Autoren. From: Tagung,band de, 9. Aachener Kolloquiums "Signaltheorie", pages 2.87 - 2.90 Topic Spotting Using Subword Units V. Warnke, S. Harbeck, E. Noth, H. Niemann Universitat Erlangen-Niirnberg, Lehrstuhl fUr Mustererkennung, Martensstr. 3, 91058 Erlangen, Germany Tel.: +49 9131 / 857883 Fax.: +49 9131 / 303811 warnke@informatik.uni-erlangen.de http://www5.informatik.uni-erlangen.de Abstract In this paper we present a new approach for topic spotting based on subword units and fea-ture vectors instead of words. In our first approach, we only use vector quantized fea-ture vectors and polygram language models for topic representation. In the second ap-proach, we use phonemes instead of the vec-tor quantized feature vectors and model the t9pics again using polygram language models. We trained and tested the two methods on two different corpora. The first is a part of a me-dia corpus which contains data from TV shows for three different topics. The second is the VERBMOBIL-corpus where we used 18 dialog acts as topics. Each corpus was splitted into disjunctive test and training sets. We achieved recognition rates up to 82% for the three top-ics of the media corpus and up to 64% using 18 dialog acts of the VERBMOBIL-corpus as topics. 1. Introduction In most approaches in the field of topic spot-ting, words or word sequences are used for identifying a topic [5J. This is done by word recognition with large vocabularies or special word spotters [9]. To train such recognizers in both cases a huge amount of tediously labeled data has to be available. For the training of our topic spotter with phonemes and vector quantized feature vec-tors we do not need the data to be labeled as exactly as for training a word spotter or a word recognition system. We only need the speech signals labeled with their topic rather than a word-to-word transliteration. Using either a vector quantizer (see section 2.1.) or phoneme segmentizer (see section 2.2.), we segment the speech signal into a symbol sequence. With these sequences we train stochastic language models (LMs) for each of the topics to be iden-tified. In test phase we run all LMs in parallel and decide for the topic with the maximum a posteriori probability (see section 2.3 .). The advantage of our approach is evident when changing to a new domain. Doing topic spotting with a large vocabulary speech rec-ognizer, one has to adapt the lexicon if not retrain all the acoustic models with domain dependent transliterated speech. With a word spotter) new keywords have to be identified and trained. In our approach only the lan-guage models have to be retrained. The train-ing labels, i.e. the assignment of the topic to the training utterances, can be done very fast. Using the vector quantizer; one can even switch the language without the need for a more detailed labelling of the training data. For our experiments we understand a topic in two different ways: in the first case a topic is a theme of a TV show (IDS-corpus) and in the second we understand a dialog act [2] as a topic (VERB MOBIL-COrpus) (see section 3.). 2. Methods Used 2.1. Feature extraction and vec-tor quantization For representing the speech signal as feature vectors, we use the mel-frequency-cepstrum-coefficients (MFCCs). A feature vector c is calculated for a 10ms part of the speech sig-nal and contains the energy and the first 11 MFCCs. These features and their first deriva-tives are used as an input to a phoneme seg-mentizer in the phoneme based approach. In the vector quantization approach eight neigh-boring feature vectors c (without the deriva-tives) are concatenated to a new feature vec-tor c, which describes a context of 80ms with 96 coefficients. We use this time window, be-cause the average length of a german phoneme is about 80ms. With this feature vector we calculate an initial codebook q( 0) with 240 classes. Using the linear discriminant analysis (LDA) on the codebook q(O) we optimize the intra- and inter-class-distance of the 240 code-book classes and transform the feature vector c with the 96 components in a new smaller vec-training data feature quantization I extraction LDA topic 1 codebook q(C) I with K topic N classes Figure 1: Partioning of the feature space with fea-ture reduction using LDA tor c with 24 components. This feature vector is used to train a new codebook q( C) with 65 classes (see figure 1). For training the code-book we use the well known LEG-algorithm [4], which minimizes the expected quantiza-tion error E = £[d(C,q(C))J (1) With the resulting codebook we are able to segment the utterances of the trainmg and test data into equidistant code book class sequences (CCSs) with a segment length of 80ms. Using these CCSs we train a LM for every topic of interest . 2.2. Phoneme segmentation In the phoneme based approach we use phoneme sequences instead of CCSs. Our phoneme segmentizer works HMM-based. Ev-ery phoneme is modeled by one left-right-semicontinous HMM. All HMMs are merged together to a compound-HMM-network with one start and one end state. Thus it is possi-ble to segment an utterance and calculate the corresponding phonemes in one pass using the one-pass-algorithm as described in [3J . The inventory of our phoneme segmentizer is 62 phonemes and 3 silence models . We trained this phoneme segmentizer with the training data from the VERBMOBIL-corpus and used it to segment the data from the IDS-corpus as well. We can see the phoneme segmentizer (just aE the vector quantizer) as a signal-to-symbol transformation. The differences between the phoneme classes and the codebook classes is that codebook classes have no direct acoustic-phonetic meaning. Notice, that we keep the size of the input feature vector (24), the num-ber of phonemes/codebook classes (65), and the average window size (80ms) in the same order. Thus we can see how much we loose in performance, when working with the unsu-pervised vector quantization rather than the supervised phoneme segmentation. 2.3. Polygram Language Model In most cases language models are used to calculate the probability of a word sequence W = WI ... Wm in a given language or context. We use polygram language models [7J, which are a special kind of stochastic language mod-els to calculate the probability of a symbol se-quence where a symbol could be a phoneme or a codebook class. Using polygrams the probability of the symbol sequence WI . . . Wm is calculated with the help of m P(WI) . II P(wn I WIW2 ... Wn-2Wn-l) (2) n=2 ' v ' history Because the younger history Wm-n+1 ... Wm-I of the symbol sequence WI " . Wm is more im-portant for modeling and to restrict the num-ber of free parameters inside the LM, we only use the last n - 1 symbols instead of the whole history. P(Wm I WI .. , Wm-I) ~ P(wm I Wm - n +1 ... Wm-I) (3) , v ~ (n-I) With this shorter history we can estimate P(wm I wm- n+1'" wm-d from a given train-ing corpus using the interpolation scheme A (I ) #(WI ..• wm ) P Wm WI··· Wm-I = ( )' # WI" ,Wm-I (4) where # is a function which counts how often a symbol sequence is seen in the training data. To handle symbol sequences that were not seen in the training data we need an interpolation formalism. Linear interpolation The first interpolation method we use is the linear interpolation [7J (L = lexicon size): n - 1 P(wm I WI" .wm-d = Po' L +PI . p(wm } +P2 . p(wm I Wm-I) + L Pi . p(Wm I Wm-i+l ... Wm-I). (5) i=3 The interpolation coefficients Pi can be es-timated using the Expectation Maximization (EM) algorithm [6J on a given validation set. U sing this method an unseen symbol sequence is modeled by its subsequences weighted with the interpolation coefficients. Rational interpolation The rational interpolation method is the sec-ond interpolation we apply [7]: P(wm I WI .. , Wm-l) = utterance I ...-____ ----icqdebook class sequence h 1 I I I ,------ - --, 12 1 45 37 23 62 4 37 23 62 4 43 17 23 4 43 17 23 46 57 54 46 57 54 3 4 21 25 LM training sequence , I I Figure 2: SI?litting a ees w into subsequences wlth step S = 3 and window W = 7 Lf=lPi' (l/L)n-i. #dWI" ,Wm-IWm) E?=IPi' (l/L)n-i. #i(WI ... wm-d ' (6) where #i counts the i last predecessors (Wm-i ... Wm-l) in a given sequence w. In this interpolation formalism it is also possi-ble to estimate the interpolation coefficients using the EM-algorithm on a given validation set. This interpolation gives more weight to the symbol sequences which have often been present in the training data and are in the nearest neighborhood of the observed symbol. 3. Corpora 3.1. IDS-Corpus We performed experiments for both ap-proaches with the data from the IDS-corpus (Institut fUr Deutsche Sprache). This corpus contains data from German TV shows for the three topics speech, politics, and culture. It is a small part of the IDS-media-corpus with more than about 200 hours of spoken data. It was not easy to say, which TV show cor-responds to which topic. So we assigned, for example, one TV show to the topic "politics", if it was announced that it is a political dis-cussion about the "gulf war". All utterances of this TV show were then assigned to the topic "politics" . 316 utterances from 11 different TV shows were divided into speaker disjunc-tive training (250 utterances) and test sets (66 utterances). The length of an utterance is 39 seconds in the average. 3.2. VERBMOBIL-Corpus VERBMOBIL is a speech-to- speech transla-tion project [1 J in the domain of appointment scheduling, i.e. two persons try to fix a meet-ing date, time, and place. One of the tasks of the dialog module within VERBMOBIL is to LM training sequences multigram model 1" ...... -:===::. polygram count ..-----... [ i~~~:;~f. }--I '-__ L_M_l _-,I l i~~~~~~~. }-- LI __ L_M_N_--,I Figure 3: Training of LMs for the topics Ti keep track of the state of the dialog in terms of dialog acts. Dialog acts are e.g., "greet-ing", "confirmation of a date", l'suggestion of a place" . In VERBMOBIL a turn of one user can consist of more than one dialog act. Cur-rently, the processing is done in two steps: first, the utterance is segmented into dialog act units. Second, these units are classified into dialog acts. In VERBMOBIL there are 19 dialog acts. In our experiments we recognize 18 of these dialog acts as topics l • We used 3284 dialog acts for training and 1064 dialog acts for testing. For the experiments described below we used a hand segmentation of turns into dialog acts. 4. Topic spotting 4.1. Preparing the Sequences When we transform an utterance into a sym-bol sequence W = WI ... Wm we can train and test our topic spotter with the whole sym-bol sequence or we can observe the seluence through a sliding window with window ength Wand step S (see figure 2) to get subse-quences w. Thus we are able to get a result at each step of the recognition task and ob-serve the provisional results. 4.2. Using Language Models Using the sequence W or w we trained a poly-gram language model for each topic (see fig-ure 3). For the classification task, we combine all the polygram models together to one multi-gram model. This multigram model contains the polygram models for all topics. Running the multigram model means running all poly-gram models in parallel to calculate the a pos-teriori probabilities P(Ti I w) for each topic Ti E T of interest We decide for the topic with the maximum a posteriori probability. For the whole sequence W (if we obsere w) we accumulate the hits for each subsequence wand decide for the topic which receives the most hits. lOne dialog act was only present a few times in the training set and could thus not be trained. 5. Experiments and Results Using the data from the IDS-corpus we cal-culated cess and used different length W for our sliding window. The step was in all ex-periments set to S = 1. So we were able to get a result for every 80ms of the ut-terance with the actual context of the win-dow W. Table 1 shows the best results we achieved for different sizes of Wand interpola-tion methods using polygrams of length three. We performed the same experiments with the code book class sequences wdh trtgram - LM 74 77 70 81 70 74 78 85 68 68 56 75 56 63 50 87 II 67 I 65 II 64 I 71 II 65 65 71 80 Table 1: Recognition results in percent for cess (L =linear interpolation, R =rational interpolation) phoneme sequences. Here the window size W (W phonemes) is only approximately constant over time. We again reached the best results when we used the maximum context of three. The results for this experiment are given in ta-ble 2. We compared the results for our auto-phoneme sequences w~th trtgram - LM 81 81 85 89 79 74 81 81 100 81 94 81 75 81 II 79 I 73 II 82 I 77 82 76 79 77 Table 2: Recognition results in percent for phoneme sequences matically segmentized symbol sequences with LMs trained on the transliterated word se-quences (simulating 100% word recognition) which reached a recognition result of 89% for a sliding window of W = 10 and 85% for the whole utterance. Thus we can see that is pos-sible to classify the three topics of the IDS-corpus using subword units and polygram lan-guage models. Using the VERBMOBIL-corpus as database we only did experiments with the phoneme se-quences. At this point of our research we only used the spoken phoneme sequences of the 18 dialog acts for training and testing. We calculated the a posteriori probability for the whole symbol sequences of an utterance. Thus we reached a recognition rate of 64% for the 18 dialog acts when we used the phoneme se-quences, a rational interpolation method and maximum poly_gram size of four for the lan-guage models. We also trained a topic spotter on the transliterated word sequence as we did for the IDS-corpus. Here we reached a recog-nition rate of 63%. This shows that (for the given training data) we can model the topics better with phoneme sequences than with the word sequences. One reason is that, e.g. the words appointment and appointments are dif-ferent for the word based LMs, but the stem (appointment) is equal for the phoneme based LMs. A more detailed discussion of the exper-iments is given in [8]. 6. Conclusion We presented two approaches for topic spot-ting with the use of subword units. Because we use vector quantization and a domain in-dependent phoneme segmentizer, we only need the utterances of the training set labeled with their topics. The main difference between both approaches is the fact, that we use cess of equidistant length in our vector quantizer and a symbol sequence of variable length in our phoneme based approach. In both ap-proaches the topics are modeled with the help of stochastic language models. During the recognition task for all topics the a posteri-ori probability is calculated in parallel and the topic with the maximum probability is cho-sen. We showed that depending on the defini-tion of topic and the amount of training data our approach performed almost as good or bet-ter than spotter which uses a "perfect" word recognition module. 7. References 1. T. Bub and J. Schwinn. Verbmobil: The Evolution of a Complex Large Speech-to-Speech Translation System. In Int . Conf. on Spoken Language Proces&ing, volume 4, pages 1026-1029 , Philadelphia, 1996. 2. S. J ekat , A. Klein, E. Maier, 1. Maleck, M. Mast, and J. Quantz . Dialogue Acts in Verbmobil. Verbmobil Re-port 65 , 1995. 3. T. Kuhn. Die Erkennungsphase in einem Dialogsllstem, vol-ume 80 of Dissertationen zur Kunstlichen Intelligenz. Infix, St. Augusti n, 1995. 4. Y . Linde, A. Buzo, and R .M. Gray. An Algorithm for Vec-tor Quantizer Design. IEEE Trans. on Communications, 28(1): 84- 95, 1980. 5. E . Parris and M. Carrey. Topic spotting with task indepen-dent Models. In Proc. European Conf. on Speech Commu-nication and Technology, pages 2133-2136, Madrid, Spain, September 1995. 6. E . G. Schukat-Talamazzini . Automatische Spracherkennung - Grundlagen, statistische Modelle und ejJiziente Algorith-men. Vieweg, Braunschweig, 1995. 7. E.G . Schukat-Talamazzini . Stochastic Language Models. In Electrotechnical and Computer Science Conference, Por-toroi, Slovenia, 1995. 8. V. Warnke. Topik- und Topikgrenzen-Spotting. Diplomar-beit , Lehrstuhl fur Mustererkennung (Informatik 5), Univer-sita t Erlangen ·Nurnberg, 1996. 9. M. Wint raub . Keyword-spotting using sri's decipher large vocabulary speech recognition system. In Proceedings Inter-national Conference on Automatic Speech and Signal Pro-cessing, pages 463-466, 1993. 

In this paper we present a new approach for topic spotting based on subword units and feature vectors instead of words. In our first approach, we only use vector quantized feature vectors and polygram language models for topic representation. In the second approach, we use phonemes instead of the vector quantized feature vectors and model the topics again using polygram language models. We trained and tested the two methods on two different corpora. The first is a part of a media corpus which contains data from TV shows for three different topics. The second is the  Verbmobil-corpus where we used 18 dialog acts as topics. Each corpus was splitted into disjunctive test and training sets. We achieved recognition rates up to 82% for the three topics of the media corpus and up to 64% using 18 dialog acts of the Verbmobil-corpus as topics

V. Warnke

S. Harbeck

E. Nöth

H. Niemann

CiteSeerX

Topic Spotting Using Subword Units

https://publikationen.sulb.uni-saarland.de/bitstream/20.500.11880/25383/1/Vm_205_.pdf

Topic spotting using subword units

Abstract

Similar works

Full text

Available Versions

Scientific documents from the Saarland University,

Universaar

Acronym

CiteSeerX