ABSTRACT Using sentence templates and a stochastic context-free grammar a large corpus (10,000 sentences) has been created, where prosodic phrase boundaries are labeled in the sentences automatically during sentence generation. With perception experiments on a subset of 500 utterances we verified that 92 % of the automatically marked boundaries were perceived as prosodically marked. In initial automatic classification experiments for three levels of boundaries recognition rates up to 81 % could be achieved. 1.1 Introduction and Material A successful automatic detection of phrase boundaries can be of great help for parsing a word hypotheses graph in an automatic speech understanding (ASU) system. Our recognition paradigm lies within the statistical approach; we therefore need a large training database, i.e. a corpus with reference labels for prosodically marked phrase boundaries. In this paper we wil

A. Batliner

A. Kießling

E. Nöth

H. Niemann

R. Kompe

U. Kilian

CiteSeerX

In “SpeechRecognition and Coding — New Advancesand Trends”,Rubio–Ayusoand Lopez–
Soler (eds.), Springer, 1995, pp. 89–92.
1
The Prosodic Marking of Phrase
Boundaries: Expectations and Results
A. Batliner1
R. Kompe, A. Kießling, E. N¨ oth, H. Niemann2
U. Kilian3
ABSTRACT Using sentence templates and a stochastic context-free grammar a large corpus
(10,000 sentences) has been created, where prosodic phrase boundaries are labeled in the sen-
tences automatically during sentence generation. With perception experiments on a subset of
500 utterances we veriﬁed that 92% of the automatically marked boundaries were perceived as
prosodicallymarked.In initial automaticclassiﬁcationexperiments for three levels of boundaries
recognition rates up to 81% could be achieved.
1.1 Introduction and Material
A successful automatic detection of phrase boundaries can be of great help for parsing a
wordhypothesesgraph inan automatic speech understanding(ASU)system. Ourrecognition
paradigmlieswithinthestatisticalapproach;we thereforeneeda largetrainingdatabase, i.e. a
corpus withreference labels for prosodicallymarked phrase boundaries. In this paper we will
presenta methodforautomaticgenerationofthesereference labelsthatenables ustogenerate
“arbitrarily large” corpora. To verify the validity of our approach we conducted perception
experiments where naive listeners had to label prosodic phrase boundaries.
The material we investigated is part of the German domain dependent speech database
ERBA,“ErlangerBahnAnfragen”(Erlangentraininquiries),a largespeech trainingdatabase
for word recognition. To maximize the variabilityof the phonetic context we wanted to have
as many different training sentences as possible. A stochastic sentence generator was used
based on a context free grammar and 38 sentence templates, that can create an “arbitrarily
large” text corpus where each utterance is unique. Optional parts are deﬁned that are to be
used in a certain percentage of the created sentences. The a priori probability of alternative
word groups can be set. The utterances consist of one sentence with or withouta subordinate
clause and a shortellipticsentence. At the Univ. of Erlangen, the Univ. of Bielefeld,Daimler-
Benz (Ulm), and Philips (Aachen) 10,000 of these sentences were recorded (100 untrained
speakers with 100 utterances each) resulting in a speech database of 14 hours. The size of
the vocabulary was 949 including 571 train stops. The recordings were conducted in quiet
ofﬁce environments using headphones and desklab recording devices (Gradient). The signals
were digitizedwith16kHz, 14 bits. The subset of the database ERBAused for the perceptual
1L.M.-Univ. M¨ unchen, Institut f¨ ur Deutsche Philologie, 80799 M¨ unchen, F.R. of Germany
2Univ.Erlangen-N¨ urnberg,Lehrstuhlf¨ urMustererkennung(Inf.5),91058Erlangen,F.R.ofGermany
3Daimler-Benz Forschungsinstitut Ulm, 89081 Ulm, F.R. of Germany
4This work was supported by the German Ministry for Research and Technology (BMFT) in the
joint research project ASL/VERBMOBIL. Only the authors are responsiblefor the contents.2 A. Batliner, R. Kompe, A. Kießling, E. N¨ oth, H. Niemann, U. Kilian
evaluation consists of utterances of 10 speakers (5 male, 5 female) with roughly the same
regional variety of standard German (Franconian). Out of the 100 utterances produced by
each speaker, 50 utterances were chosen thatwere longenoughtocontain a sufﬁcient number
of different phrase boundaries each and did not contain semantic anomalies. The latter can
occur duringsentence generation(e.g. “I wanttoleave between 10and 10o’clock”) and were
not discarded manually before the recording of the database, because we considered their
inﬂuence on the word recognitionto be negligible.
1.2 Boundary Marking Based on Linguistic Knowledge
Syntactic boundaries were marked in the grammar and included in the sentence generation
process with some context-sensitive post-processing. The text read by the speakers did not
containthese markers. We distinguishfourtypes of boundaries(examples are translated word
by word):
￿ B3 boundary: boundaries between elliptic clause and clause e.g. Guten Morgen B3 Ich
m¨ ochte gerne
￿
￿
￿
￿
￿ (Good morning B3 I would like to
￿
￿
￿
￿
￿ ), between main and subordinate
clause e.g.
￿
￿
￿
￿
￿ einen Zug B3der sehr fr¨ uhf¨ ahrt(
￿
￿
￿
￿
￿ a trainB3 thatvery early leaves), orat
coordinatingparticles between clauses e.g. ich m¨ ochte um acht Uhr nach M¨ unchen fahren
B3 und m¨ oglichst fr¨ uh ankommen (I would like at eight o’clock to Munich to go B3 and
as early as possible arrive).
￿ B2 boundary: boundaries between constituents as e.g. in der Nacht B2 mit dem IC B2
nach Ulm (during the night B2 with the IC B2 to Ulm), and boundaries at coordinating
particles between constituents as e.g. zwischen Ulm B2 und Stuttgart (between Ulm B2
and Stuttgart).
￿ B1 boundary: boundaries that syntactically belong to the normal constituent boundaries
B2 but that are most certainly not marked prosodically because they are close to a B3
boundaryorthebeginning/endoftheutterancease.g. ichm¨ ochteB1amn¨ achstenDienstag
B2zwischendrei B2undsechs UhrB2vonHamburgB2nachUlmB1fahren(Iwouldlike
B1 next Tuesday B2 between three B2 andsix o’clock B2 from Hamburg B2 to Ulm B1 to
go). At a B1 boundarywe, so to speak, hypothesizea prosodicallyclitic, weak constituent
thatintegrateswiththesucceedingorprecedingstrongerconstituentintoa greaterprosodic
phrase.
￿ B0 boundary: every word boundary that does not belong to B1, B2, B3.
1.3 Perception Experiments
In order to verify our expectations concerning the prosodic marking of syntactic phrase
boundaries, perception experiments were run with ten “naive” listeners (students) each. The
subjects were given the utterances inorthographicform withoutany punctuationmarks. They
wereaskedtomarkthespacebetweentwowordsiftheyfeltitseparatedtwodifferent“chunks”
ofspeech. Thelistenerswere instructednottorelyupontheirknowledgeofcanonicalformsor
sentence structure,althoughinﬂuenceofthesefactorscancertainlynotberuledoutaltogether.1. The Prosodic Marking of Phrase Boundaries: Expectationsand Results 3
The perception data were compared
withthe labeled places of phrase bound-
aries.Eachpossiblephraseboundarypo-
sitioncouldgeta scorefrom0(nomark)
up to 10 (all 10 subjects in the test per-
ceivedaphraseboundaryasprosodically
marked.)
In ﬁgure 1.1, the results of the percep-
tion experiments are given for the four
different boundary types. The distribu-
tion of the B0, B1, and B3 boundaries
meet our expectations and cluster at the
left end (very few scores for B0 and B1
number of subjects
p
e
r
c
e
n
t
s
c
o
r
e
s
0 1 2 3 4 5 6 7 8 9 10
0
20
40
60
80
100
B0 (n=3830) B1 (n=1216)
B2 (n=1173) B3 (n=350)
FIGURE 1.1: Perception results for boundary types
boundaries)orattherightend(many scoresforB3boundaries).Mostprobably,clause bound-
aries e.g. can thus be successfully handled in an ASU system. The B2 boundaries behave
differently: only 63% were marked by more than 4 subjects. It might be at the discretion
of the speaker if he/she wants to mark these boundaries. In 92% of the cases where at least
5 listeners perceived a boundary there was an automatically generated reference boundary
(B2,B3). Also in 92% of the cases where less than 5 listeners perceived a boundary there
was no automatically generated reference boundary (B0,B1). This and the fact that three of
the four boundary classes in ﬁgure 1.1 are clear-cut and meet our expectations leads us to
the conclusionthat the automatically generated reference boundaries are adequate and can be
used to train and test classiﬁers.
Using a Gaussian classiﬁer and the features described below we so far got a recognition
rate of66%forthethreeclasses. Withthiswe determinedthe differencesbetween the number
of listeners who perceived a boundary and the probability computed by the classiﬁer times
ten. In 54% of the cases the absolute difference is less than or equal to two.
1.4 Automatic Classiﬁcation of Phrase Boundaries
Initially, the experiments were based on the spoken word chain, which also contains pause
information.A time alignment of the word chain was achieved automatically using an HMM
word recognition module. A F0-contour was computed using the algorithm described in [2]
resulting in one value per frame (10 msec) measured in semi-tones. Note that the F0-contour
mightbe erroneousandwas notcorrected manually.For each wordboundarya set ofprosodic
features was computed:
￿ length of the pause
￿ thenormalized(sameasin[5])andunnormalizeddurationofthesyllableandofthesyllable
nucleus prior to the boundary; the mean and the standard deviation of the duration for the
phoneme class of the syllable nucleus
￿ for the frame withthe maximum energy withinthe twosyllables to the left and to the right
of the boundary, the energy itself and the position of the frame relative to the boundary;
the average energy of the two syllables to the left and to the right of the boundary
￿ the linear regression coefﬁcients of the F0-contour computed over 2 and 4 syllables to the
left and to the right of the boundary4 A. Batliner, R. Kompe, A. Kießling, E. N¨ oth, H. Niemann, U. Kilian
￿ onset, minimum, maximum and offset F0 and their positions on the time axis relative to
the boundary computed over the two syllables to the left and to the right of the boundary.
We trained a quadratic polynomial classiﬁer [1] using these 31 features in order to dis-
criminate between the three classes B0+B1, B2, B3. The trainingdatabase consisted of6900
ERBA utterances from 69 speakers. The test set consisted of 1000 ERBA utterances from the
10 speakers who were used for the perception experiments. Using the a prioriprobabilitiesof
the classes a recognition rate of 81% was achieved (mean recognition rate 51%); assuming
equal distribution of the classes a recognition rate of 71% was achieved (mean recognition
rate 69%).
1.5 Concluding Remarks
In the future we will improve the classiﬁer as well as the feature set and combine it with lan-
guage models based on classiﬁcation trees similar to [4] or with stochastic language models.
In this context ergodic Hidden Markov Models will be considered as in the work reported
in [5] for English. We will also build an intonation model integrating phrase boundaries as
well as phrase accents. For this we plan to develop a method which enables us to gener-
ate automatically phrase accent reference labels based on a text corpus like ERBA, where
prosodic phrase boundaries are already marked4. In ongoing work (see [3] for details) we
integrateinformationaboutaccents intothe wordrecognitionmodule ofourASU system. We
have already achieved encouraging recognition improvements just using the lexical accent
information for the modeling of the subword units: the recognitionerror on the word as well
as on the sentence level was reduced by around 5%. We hope to get further improvements by
lookingat phrase accents and adding new suprasegmental features to the feature vector.
1.6 REFERENCES
[1] J. Franke. On the functional classiﬁer. In Proc. of 1st Int. Conf. on Document Analysis
and Recognition, pages 481–489, St. Malo, 1991.
[2] A. Kießling,R. Kompe, H. Niemann, E. N¨ oth, and A. Batliner. DP-Based Determination
of F0 contours from speech signals. In Proc. Int. Conf. on Acoustics, Speech and Signal
Processing, volume 2, pages II–17–II–20,San Francisco, 1992.
[3] H.Niemann,E.N¨ oth,E.G. Schukat-Talamazzini,A.Kießling,R.Kompe,T.Kuhn,K.Ott,
and S. Rieck. Statistical Modeling of Segmental and Suprasegmental Information. In
this volume.
[4] M.Q. Wang and J. Hirschberg. Automatic Classiﬁcation of Intonational Phrase Bound-
aries. Computer Speech & Language, 6(2):175–196,1992.
[5] C. Wightman and M. Ostendorf. Automatic Recognition of Intonational Features. In
Proc. Int. Conf. on Acoustics, Speech and Signal Processing, volume 1, pages I–221–I–
224, San Francisco, 1992.
4Due to lack of space, we cannot report the results of a parallel perception experiment with the
same material where subjects had to mark each syllable they perceived as accented. There was a good
agreement between the perception data and the labeled places of accents. These results as well as the
relationship between phraseaccents and phrase boundarieswill be discussedin a forthcoming paper.

1 The Prosodic Marking of Phrase Boundaries: Expectations and Results

http://www5.informatik.uni-erlangen.de/Forschung/Publikationen/1995/Batliner95-TPM.pdf

1 The Prosodic Marking of Phrase Boundaries: Expectations and Results

Abstract

Similar works

Full text

Available Versions

CiteSeerX