Huckvale, M

English

UCL Discovery

Proc. 13th Int. Congress.Phonetic Sciences, Stockholm, August 1995.PHONETIC CHARACTERISATION AND LEXICAL ACCESS INNON-SEGMENTAL SPEECH RECOGNITIONMark HuckvaleUniversity College London, London, U.K.ABSTRACTAn isolated-word speech recognitionsystem, built without the use of linearsegments for acoustic modelling or lexi-cal access, is justified, described anddemonstrated.  The system comprisesphonetic feature analysis operating onfour independent tiers, parallel phonotac-tic parsing, and lexical access based on aneural-network inspired lexicon struc-ture.  Performance is however still infe-rior to a baseline segmental system.INTRODUCTIONThis paper describes an attempt tobring together into a single operationalsystem a selection of  alternatives to thelinear segmental approach to phoneticmodelling and lexical access found incontemporary automatic speech recogni-tion systems.The most important departure fromcurrent architectures is the explicit sepa-ration of phonetics and phonology in thesystem.  In the new system the role of thefirst is to characterise speech-specificelements of the sound signal, while therole of the second is to establish thefunctions of these elements in linguisticencoding.  In contrast, current systemsbased on phones-in-context use linearphonological units to organise theiracoustic models as well as for lexicalaccess.  Such systems have particularweaknesses, including (i) poor modellingof variation of acoustic realisation ofphonological units in context, (ii) failureto model post-lexical phonetic variety be-cause of the need for complex and arbi-trary context-sensitive realisation rules,(iii) failure to exploit contextual variationas discriminative information, (iv) failureto use temporally extended informationrelevant to phonological identity, (v)failure to exploit prosodic structure in thesignal.  These weaknesses lead to sys-tems which lack discriminative power,are unable to exploit known pronuncia-tion variety in context or in accent, fail toextract the most from impoverished sig-nals, and ignore the information andconstraints available in the rhythm, stressand intonation of the speech.On the other hand, linear phonologi-cal-unit based acoustic models provide asimple and computationally effectivebasis for recognition.  There is a synergybetween a linear phonological accountand syntactic pattern recognition algo-rithms such as Hidden Markov Modelling(particularly the Viterbi decodingscheme).  It has been said that in speechrecognition good knowledge is of no usewithout good algorithms for applying it.Hidden Markov Modelling has been suc-cessful because it forms a coherent viewof the acoustic to phonological mapping,rather than an accurate one.Thus the challenge is to find effectiveprocedures for the exploitation of moresophisticated models of speech.DESIGNIn this section we justify the non-seg-mental recognition system described inthe following section.  More details maybe found in [1].Phonetic componentThe role of the phonetic component ina non-segmental system is to model therange of variety of acoustic realisation ofelemental phonetic characteristics.  Foreach given characteristic at each timeframe, the phonetic component suppliesthe probability that the element has beenrealised (by a given speaker in a givenacoustic environment).  By relaxing theProc. 13th Int. Congress.Phonetic Sciences, Stockholm, August 1995.requirement that these characteristicsneed to be themselves phonological wecan make this component more sensitiveto sub-phonemic changes, to syllabic andprosodic structure.  Although we can nolonger exploit phonological sequenceconstraints we can still exploit phoneticconstructional constraints that arise dueto the fact that the signal was spoken.  Inthe simplest model, the phoneticcomponent operates on a number of tierswhere the phonetic properties inside atier are mutually exclusive, while prop-erties across tiers are mutually independ-ent. As we shall see this allows the use ofa syntactic pattern recognition scheme tooperate within a tier.Lexical AccessFrom the phonetic characterisation ofthe signal it is necessary to explain thephonetic evidence as realisations of asequence of words, subject to a numberof constraints: (i) words occur strictlysequentially, (i.e. only one word is activeat any one time), (ii) citation form pho-netic structures of words are subject to alimited range of contextual modifications,(iii) word selection is guided by the task(vocabulary, syntax, etc.).Since at this stage we do not have aphonological representation, all we cando is activate word hypotheses on thebasis of the likelihood that they mighthave given rise to the phonetic evidence.Following the TRACE model of lexicalaccess [2] we can see that each phoneticcharacteristic can feed 'activation' intothe lexicon, (but in this case without aninterposing phonemic layer).  Given atiered phonetic analysis, any single tieractivates a number of possible wordhypotheses.  The initial activations ofwords need not be zero, since there maybe prior evidence (from the task) for thelikelihood of words.Phonological categorisationFrom the word activations (overtime), it is necessary to determine themost likely word sequence.  Unfortu-nately, what we have at the moment isessentially a whole-word template rec-ognition system, and it is easy to showthat such systems cannot be extended tolarge vocabularies without the exploita-tion of phonological knowledge.  Eachword has been activated on the basis ofphonetic similarity with the input, but it islikely that some components of the wordmatch better than other components.Thus the vowel of [pi] may match the in-put quite well, but the consonant maymatch badly.  If each word has independ-ent pronunciation models of phoneticrealisation, it is possible that the vowel of[ti] might not match as well as the vowelof [pi].  Thus an input '"T" may be rec-ognised as "P" because the vowelmatches overcome the consonantalmatches.  The solution to this is to indi-cate that the vowel in [pi], i.e. /i/, is thesame as the vowel /i/ in [ti].  With thisconstraint, the difference in the vowelscores is irrelevant and the consonantalmatch controls the outcome.  This isphonological knowledge that must bespecified in addition to the phonetic reali-sation of words.One way of imposing these pho-nological constraints is to establish a setof phonological units above the words,which share activations between wordswhich have similar phonological pre-scriptions.  Thus an /i/ unit short-circuitsactivation between [pi] and [ti] to coun-teract exactly any difference due to in-dependent models of the vowel.ARCHITECTUREThe specific implementation of thenon-segmental recognition architecturefor an isolated word recognition task maybe separated into: (i) multiple Phoneticfeature components that deliver phoneticfeature analyses of 30ms of speechsignal, (ii) Phonotactic decodingcomponents that deliver element se-quence likelihoods for each tier, (iii) aLexical access component that takes theProc. 13th Int. Congress.Phonetic Sciences, Stockholm, August 1995.element sequence scores and delivers aword hypothesis using lexical and pho-nological information.  More details maybe found in [3].Phonetic feature componentThe phonetic feature component op-erates on four independent tiers, corre-sponding to multiple broad-class analysesof the signal.In the Excitation tier, phonetic ele-ments represent Silence (SIL), Voicing(VOI), Frication (FRC) and Mixed exci-tation (MIX).  In the Degree tier, ele-ments represent Oral closure (STP), Na-sal (NAS), Fricative (FRC), Approximant(APP), Close vowel (CLS), Mid Vowel(MID) and Open vowel (OPN).  In thePosition tier, elements represent Labial(LAB), Dental (DEN), Alveolar exclud-ing /s/ (ALV), /s/ frication (FRS),Front/Palatal (FRN), Central (CEN),Back (BAK), Velar (VEL) and Silence(SIL).  In the Strength tier, the elementsrepresent Burst (BUR), Aspiration(ASP), Other frication (FRC), Vocalic(VOW), Voiced plosive (VGP) and Si-lence (SIL).These tiers together are sufficient todifferentiate English words apart fromshort and long vowels at a single place(e.g. bit vs. beat) and dental and labio-dental fricatives (e.g. thin vs. fin).  Per-formance on elements for these contrastsis currently unsatisfactory.For each tier, a Multi-Layer Percep-tron (MLP) classifier was trained be-tween a spectral representation of thesignal and the target element classes.Each tier had its own MLP with 3x10msframes representing 19 filterbank ener-gies + overall energy (i.e. 60 parameters)as input and 1 output per element class.Each MLP had a single hidden layer of asize equal to three times the output layersize.  The training data was 666 differentmonosyllabic words spoken by onespeaker.  There were approximately83,000 training vectors.Each training word was annotated andthe element labels generated by rule usinga mapping that took into accountboundaries and the nature of adjoiningsegments.  Training was performed usingan adaptive back-propagation methodfirstly on the automatically generatedelement labels, and then, after realign-ment with the partially trained network,against realigned element labels.Phonotactic decoding componentTo generate an element sequence, aViterbi decoding was performed on theMLP outputs for a tier over the wholeduration of a word.  See Figure 1.  Thisprocess delivered a score for each phono-tactically possible sequence in the testvocabulary for each tier.  Over the 4 tiersthere were 450 possible elementsequences, but only the best scoring 50%in each tier were used for lexical accessLexical access componentTo identify the lexical item a networklexicon was used based on [1].  Here thephonetic input was provided by the ele-ment sequence scores;  these then fedactivations to the word units accordingto 'dictionary' pronunciations of thewords.  Thus words were only connectedto element sequences expected in the ci-tation pronunciation.  To smooth activa-tions across words, a level of phonologi-cal units were constructed above theword units, which channelled activationbetween words sharing similar pho-nological descriptions - in this experimentshared syllabic components.  Thus wordactivations arose primarily from thephonetic input, but subsequently therewas interaction and competition betweenwords mediated by a set of phonologicalunits. The most strongly activated wordunit was chosen to be the recognisedword.RESULTSFor testing the architecture, 359monosyllabic words, different to thetraining words, but spoken by the sameProc. 13th Int. Congress.Phonetic Sciences, Stockholm, August 1995.speaker were used.  The raw recognitionperformance of the Phonetic featureanalysis component was:Tier Frames correctExcitation 91.6 %Degree 82.8 %Position 74.7 %Strength 87.9 %The raw recognition scores for theelement sequences was:Tier Top 1 Top 5Excitation 76.6 % 98.3 %Degree 46.0 % 80.2 %Position 23.4 % 52.9 %Strength 44.0 % 88.6 %For the feature-to-word activationsalone, without the use of the phonologi-cal units for smoothing, the word rec-ognition performance was 51%.  Smallamounts of phonological unit activationfed back to the word units improvedrecognition performance only slightly, to53%.  Performance is so weak primarilydue to the poor performance of the Posi-tion tier.Baseline recognition performance us-ing a monophone HMM trained on thesame material (and having approximatelythe same number of free parameters asthe set of MLPs) was over 90%.FURTHER INFORMATIONThe author welcomes comments onM.Huckvale@ucl.ac.ukREFERENCES[1] Huckvale, M.A. (1990),"Exploiting Speech Knowledge in NeuralNets for Recognition", Speech Com-munication, p1.[2] McClelland, J.E. & Elman, J.L.(1986), "Interactive Processes in SpeechPerception: The TRACE Model", inParallel Distributed Processing, ed Ru-melhart & McClelland, MIT Press.[3] Huckvale, M.A. (1994), "WordRecognition from Tiered PhonologicalModels", Proc. IOA Conf. Speech andHearing, Windermere.Figure 1.  Tiered analysis of the test word 'times'.

Phonetic characterisation and lexical access in non-segmental speech recognition

Abstract

Similar works

Full text

Available Versions

UCL Discovery