We study the effect that phonetic onset has on acoustic and articulatory reaction times. An acoustic study by Rastle et al. (2005) shows that the place and manner of the first consonant in a target affects acoustic RT. An articulatory study by Kawamoto et al. (2008) shows that the same effect is not present in articulatory reaction time of the lips. We hypothesise, therefore, that in a replication with articulatory instrumentation for the tongue, we should find the same acoustic effect, but no effect in the articulatory reaction time. As a proof of concept of articulatory
measurement from ultrasound images, we report results from a pilot experiment which also extends the dataset to include onset-less syllables. The hypothesis is essentially confirmed with statistical analysis and we explore and discuss the effect of different vowels and onset types (including null onsets) on articulatory
and acoustic RT and speech production.https://www.internationalphoneticassociation.org/icphs/icphs2015caslpub3946pub84

Palo, Pertti

Schaeffler, Sonja

Scobbie, James M.

Queen Margaret University eResearch

EFFECT OF PHONETIC ONSET ON ACOUSTIC AND ARTICULATORYSPEECH REACTION TIMES STUDIED WITH TONGUE ULTRASOUNDPertti Palo∗, Sonja Schaeffler, and James M. ScobbieCASL, Queen Margaret University, Edinburgh∗ ppalo@qmu.ac.ukABSTRACTWe study the effect that phonetic onset has on acous-tic and articulatory reaction times. An acousticstudy by Rastle et al. (2005) shows that the placeand manner of the first consonant in a target affectsacoustic RT. An articulatory study by Kawamoto etal. (2008) shows that the same effect is not presentin articulatory reaction time of the lips. We hypothe-sise, therefore, that in a replication with articulatoryinstrumentation for the tongue, we should find thesame acoustic effect, but no effect in the articulatoryreaction time. As a proof of concept of articulatorymeasurement from ultrasound images, we report re-sults from a pilot experiment which also extends thedataset to include onset-less syllables. The hypoth-esis is essentially confirmed with statistical analy-sis and we explore and discuss the effect of differentvowels and onset types (including null onsets) on ar-ticulatory and acoustic RT and speech production.Keywords: Speech reaction time, articulation, ul-trasound.1. INTRODUCTIONVoice keys are devices which are used to measurespeech reaction time (RT) [6, 8]. Voice keys areknown to have a phonetic bias [10, 12, 16]. Whilepart of this bias is most likely to be a signal detectionproblem – hence the variable bias across differentvoice key devices – there is also a known productionbias due to the first segments of the target utterance[12].The bias caused by the production of differentconsonantal onsets with three different vowels fol-lowing the onset consonant was studied by Rastle etal. [13]. The crucial part of their experiment wasto alter the standard delayed naming instruction toallow the participants mentally prepare and even re-hearse speaking the target utterance as much as theparticipants wanted. After this the participants wereasked to produce the by now known utterance ina speeded trial after a randomly delayed go-signal.The participants were asked to keep their vocal tractSS sf sSff T szThh zTjm zvlnm jD Dn jrn l rvrwwkt lpvt kbb kpgdg tpddbg24027030050 100 150OD (ms)EAI(ms)Vowel a i @Figure 1: Influence of consonantal onset dura-tion on acoustic naming latency in delayed nam-ing [13]. OD = onset consonant duration, EAI =execution to acoustic interval i.e. acoustic RT.at rest until receiving the go-signal. The results showsystematic effects of consonant quality on the acous-tic onset time (Figure 1). However, Rastle et al. didnot record articulation at all, nor measure the bias ofonsetless – i.e. vowel initial – utterances.Articulatory measurements of speech RTs havebeen reported as well [9, 14]. Kawamoto et al. [9]studied lip articulation in RT tasks using the Rastleinstructions and standard delayed naming instruc-tions. They provide systematic measurements ofArticulatory onset to Acoustic onset interval (AAI)across several task conditions, but only for lip ar-ticulations. In contrast, Schaeffler et al. [14] mea-sured the difference between lingual and labial RTsin a picture naming task. They found that on averagethere was no difference, and also that there was noclear link between an initial labial consonant and lipmovement and lingual and tongue movement. How-ever, they did not control for the effect of the vowelof the onset syllable.Another aspect of speech preparation are audi-ble prespeech sounds such as breath intake, tongueclicks and lip smacks [15]. These sounds – and in-deed silent articulations which just fail to produce asound – are not purely associated with speech pro-duction at a lexical or segmental level but result fromgeneral processes of preparation, such as breathingand swallowing. However, they might be more evi-dent in normal speech than in single-word RT con-texts.Taking into account all of the results summarisedabove, it is clear that there is a need for a compre-hensive approach tackling multiple articulators. Inthis study we undertake a pilot delayed naming ex-periment to answer the following research question:Does articulatory RT (ArtRT) measured with tongueultrasound repeat the general trend seen in Figure 1or is the trend produced by differences in the ar-ticulatory preparation that is needed before speechsounds can be produced? Based on the results inprevious studies [9, 13, 14] it seems likely that thatthe differences in AcRT are produced by differencesin articulatory preparation (i.e. AAI) rather than ar-ticulatory onset times.2. MATERIALS AND METHODS2.1. ParticipantOne native Scottish English male speaker with cor-rected to normal vision and no known hearing prob-lems participated in the experiment (age 25 years).2.2. StimuliWe carried out a partial replication of the Rastle et al.delayed naming experiment [13] with the followingmajor changes: Instead of using phonetically tran-scribed syllables as stimuli, we used lexical mono-syllabic words. The use of lexical words makes itpossible to have phonetically naive participants inthe experiment. In addition, we wanted to test ifwords with a vowel onset pattern in a systematic waywith those with a consonant onset. Thus, the wordswere of /CVC/ and /VC/ type.The words were chosen from a pronunciationlexicon of Standard Scottish English generated au-tomatically with Unisyn [7]. The dictionary wassearched for word triads, which had the same onsetconsonant, one each of the vowels /a,i,O/, and thefinal consonant was a stop – i.e. one of /k,p,t/. Tomake it possible to get an adequate number of rep-etitions for each word, it was necessary to limit thenumber of onset consonants. The set used in thisstudy was selected by maximising the lexical fre-quency of the target words.The target words used in this study were: at, eat,ought, back, beat, bought, dat, deep, dot, fat, feet,fought, gap, geek, got, hat, heat, hot, cat, keep,caught, lack, leap, lot, map, meet, mock, nat, neat,not, pack, Pete, pop, rat, reap, rock, sat, seat, sought,shack, sheet, shop, tap, teak, talk, whack, wheat, andwhat.2.3. Ultrasound and audio recordingsThe experiment was run with synchronised ultra-sound and sound recording controlled with Articu-late Assistant Advanced (AAA) software [2] whichwas also used for the manual segmentation of ultra-sound videos. The participant was fitted with a head-set to ensure stabilisation of the ultrasound probe[1]. Ultrasound recordings were obtained at a framerate of∼83 frames per second with a high speed Ul-trasonix system. Sound was recorded with a smallAudio Technica AT803b microphone, which was at-tached to the ultrasound headset. The audio data wassampled at 22,050 Hz.2.4. ProcedureEach trial consisted of the following sequence: (1)The participant read the next target word from alarge font print out. (2) When the participant feltthat he was ready to speak the word, he indicatedso by pressing a button on a keyboard. (3) Thekey press activated the sound and ultrasound record-ing. The experimental software automatically initi-ated the ultrasound recording about 0.5 s after thesound recording began providing an adequate win-dow to examine the stability of the participant’s ar-ticulation before the go-signal was given. (4) After arandom delay, which was uniformly distributed be-tween 1200 ms and 1800 ms, the computer produceda go-signal – a 50 ms long 1000 Hz pure tone.It was emphasised to the participant that it wasimportant to keep his mouth (lips, tongue, etc.) atrest during phases (1) and (2), and when he heardthe go-signal to “read the word out loud as fast andas accurately as possible keeping in mind that this isa speeded trial.” The experiment consisted of foursessions on two separate days. The 48 stimuli wererepeated seven times throughout the whole experi-ment in seven internally randomised blocks.2.5. Acoustic and ultrasound segmentationThe acoustic recordings were segmented with Praat[4]. The relevant phonetic segmentation boundarieswere acoustic onset (AcRT) and the acoustic offsetof then initial consonant to provide onset consonantduration (OD). In addition to these the beginning ofthe go-signal was also annotated with Praat to pro-#bdfghklmnprsStw#bdfghklmnprsStw#bdfghklmnprsStw2002252502753000 40 80 120OD (ms)AcousticRT(ms)Vowel a i OFigure 2: Acoustic RT as a function of onset du-ration (OD). Tokens without onset consonant (i.e./VC/ words) are marked with #.vide time of the onset of stimulus for both acousticand articulatory RT measures.The ultrasound recordings were segmented withAAA [2]. The only articulatory boundary was artic-ulatory onset (ArtRT). The ultrasound videos werealso inspected for movement before the go-signal toexclude tokens where the articulatory onset occurredbefore or during the go-signal (see next Section).3. RESULTSThe recording of three tokens had been stopped be-fore the participant had had time to pronounce thetarget word. These tokens as well as 12 tokens wherethe articulatory onset happened in less than 60 msfrom or even before the go-signal onset, were ex-cluded. As were four clear outliers: two tokens withac_RT exceeding 450 ms, one /bOt/ with ac_OD >50 ms (on listening, a mispronunciation) and one/sat/ with ac_OD > 200 ms. Summary statistics arereported in Table 1.Table 1: Means and standard deviations for themeasured variables. Over all sample size n= 315.Variable Mean (ms) Stdev (ms)AAI 123.0 40.0AcRT 261.6 45.5ArtRT 138.6 33.0OD 53.9 38.3As a first step towards answering the research#bdfghklmnprsStw#bdfghklmnprsStw#bfghklmnprsStw901201500 40 80 120OD (ms)AAI(ms)Vowel a i OFigure 3: Articulatory onset to Acoustic onset In-terval (AAI) as a function of onset duration (OD).Tokens without onset consonant (i.e. /VC/ words)are marked with #.#bdfghklmnprsStw#bdfghklmnprsStw#bdfghklmnprsStw1301501700 40 80 120OD (ms)ArticulatoryRT(ms)Vowel a i OFigure 4: Articulatory RT as a function of onsetduration (OD). Tokens without onset consonant(i.e. /VC/ words) are marked with #.questions we made scatter plots of the averagesfor each onset consonant / vowel combination (Fig-ures 2-4). As can be seen in Figure 2, the presentdata does indeed pattern in the same way as the orig-inal data [13]. It can also be seen that the /VC/ utter-ances fit the pattern.Looking at Figures 3 and 4, we see that the for-mer reproduces the general pattern, while the latterdoes not. This informal analysis seems to confirmthe hypothesis made in the Introduction. To see ifthis result can be considered statistically significant,the data was analysed in R [11] by iteratively fit-ting linear models (step-up process [3]) to explainthe variation in AcRT.The iteration process identified two statisticallysignificant models that successfully predict AcRT.Given in R formula notation they are:(1) log(AcRT )∼ ArtRT +OD+ trialand(2) log(AcRT )∼ ArtRT +C1+ trial,where trial is the running trial number, C1 is the on-set consonant and AcRT has been logarithmised tocorrect for the skewness of its distribution. Includ-ing OD and C1 in Models 1 and 2, respectively, areboth highly statistically significant (P-values ≈ 0).The R2 values of Models 1 and 2 respectively are:R21 = 0.4276 and R22 = 0.6294, meaning that Model 2has greater explanatory power. Finally, it should benoted that some of the variation in ArtRT can be ex-plained with C1 (procedure as for the other models,P-value = 7.91e-06, R2 = 0.1541).4. DISCUSSIONIt is quite convincing to see that the data (Figure 2from a single speaker aligns so well with previousresults from a larger study [13, see also Figure 1].The average OD range is smaller in the present data:Rastle et al. had a range of roughly 0-150 ms,present data has 0-120 ms, as well as there beinga less pronounced difference between the AcousticRTs.It is furthermore, very interesting to see that AAIas a function of OD inherits practically all of the in-verse relationship evident in AcRT as a function OD.There are only two obvious groups which break theinversely proportional relationship of AAI to OD: Agroup of bilabial consonants in the lower left cor-ner and all /kVC/ words on the opposite side of themain trend. Bilabial consonants are likely to havean early start with lip articulation. Considering thiswould probably increase the AAI for /bak/, /bit/and /mit/ and thus move them into better alignmentwith the general AAI/OD trend.On the other hand the /kVC/ words are proba-bly right where they should be for this speaker. TheAAI of the /kVC/ words clearly aligns with thoseof /gVC/ words. At the same time the long aspira-tion produced by the speaker lengthens OD consid-erably and moves the /kVC/ words away from thegeneral trend. This does, however, leave open thequestion of why /pVC/ and /tVC/ words do alignwith the trend. Analysing data from more partici-pants is likely to shed light on this unvoiced plosivepattern. More importantly, it should be noted that,the event of interest (vowel onset) is overshadowedby the aspiration and the event itself really belongsto the articulatory domain and thus should be mea-sured based on articulation rather than acoustics.So what about the /VC/ words? We have inter-preted them as having a consonantal onset length of0 ms. This claim is backed up by the patterns inFigures 2 and 3. Further, more rigorous, evidenceis provided by extremely good fit of the statisticalmodels (Models 1 and 2). It might seem at firstcounter intuitive that acoustic RTs would be long for/VC/ words in comparison to /CVC/ words. How-ever, comparing our results with results from rhyth-mical speech alignment studies [5], there is a certainlogic to the situation. When a listener is asked toalign words to a regular rhythm they tend do so ata point which lies within the initial consonant – notat utterance onset. Perhaps we are seeing the samemechanism at work here.ACKNOWLEDGEMENTSWe wish to thank Steve Cowen for assistancewith the ultrasound recordings and Professor AlanWrench for advice on post-processing of the data.5. REFERENCES[1] Articulate Instruments 2008. Ultrasound Stabilisa-tion Headset Users Manual: Revision 1.4. Edin-burgh, UK: Articulate Instruments Ltd.[2] Articulate Instruments 2012. Articulate AssistantAdvanced User Guide: Version 2.14. Edinburgh,UK: Articulate Instruments Ltd.[3] Baayen, R. H. 2008. Analyzing Linguistic Data, APractical Introduction to Statistics using R. Cam-bridge: Cambridge University Press.[4] Boersma, P., Weenink, D. Praat: doing phoneticsby computer [computer program]. Version 5.1.44,retrieved 4 October 2010 from http://www.praat.org/.[5] Browman, C. P., Goldstein, L. 1988. Some notes onsyllable structure in articulatory phonology. Pho-netica 45, 140 – 155.[6] Cattell, J. M. 1886. The time taken up by cerebraloperations. Mind 11, 220 – 242.[7] Fitt, S. Unisyn lexicon release. version 1.3, re-trieved 26 November 2014 from http://www.cstr.ed.ac.uk/projects/unisyn/.[8] Kapusinski, D. A., Rosenquist, H. S. 1973. A briefhistory of the voice key. Proceedings of the AnnualConvention of the American Psychological Associ-ation Volume 8, 945 – 946.[9] Kawamoto, A. H., Liu, Q., Mura, K., Sanchez, A.2008. Articulatory preparation in the delayed nam-ing task. Journal of Memory and Language 58(2),347 – 365.[10] Kessler, B., Treiman, R., Mullenix, J. 2002. Pho-netic biases in voice key response time measure-ments. Journal of Memory and Language 47, 145– 171.[11] R Core Team, 2013. R: A Language and Environ-ment for Statistical Computing. R Foundation forStatistical Computing Vienna, Austria.[12] Rastle, K., Davis, M. H. 2002. On the complexi-ties of measuring naming. Journal of ExperimentalPsychology: Human Perception and Performance28(2), 307 – 314.[13] Rastle, K., Harrington, J. M., Croot, K. P., Colt-heart, M. 2005. Characterizing the motor executionstage of speech production: Consonantal effects ondelayed naming latency and onset duration. Jour-nal of Experimental Psychology: Human Percep-tion and Performance 31(5), 1083 – 1095.[14] Schaeffler, S., Scobbie, J., Schaeffler, F. 2014.Measuring reaction times: Vocalisation vs. articu-lation. Proceedings of 10th ISSP 383 – 386.[15] Scobbie, J. M., Schaeffler, S., Mennen, I. 2011. Au-dible aspects of speech preparation. Proceedings of17th ICPhS Hong Kong. 1782 – 1785.[16] Tyler, M. D., Tyler, L., Burnham, D. K. 2005. Thedelayed trigger voice key: An improved analoguevoice key for psycholinguistic research. BehaviorResearch Methods 37(1), 139 – 147.

Effect of phonetic onset on acoustic and articulatory
speech reaction times studied with tongue ultrasound

Queen Margaret University eTheses

https://eresearch.qmu.ac.uk/bitstream/20.500.12289/3946/1/ICPHS0840%2520Palo%2520et%2520al.pdf

Effect of phonetic onset on acoustic and articulatory speech reaction times studied with tongue ultrasound

Abstract

Similar works

Full text

Available Versions

Queen Margaret University eResearch

Queen Margaret University eTheses