Directions for the future of technology in pronunciation research and teaching

This paper reports on the role of technology in state-of-the-art pronunciation research and instruction, and makes concrete suggestions for future developments. The point of departure for this contribution is that the goal of second language (L2) pronunciation research and teaching should be enhanced comprehensibility and intelligibility as opposed to native-likeness. Three main areas are covered here. We begin with a presentation of advanced uses of pronunciation technology in research with a special focus on the expertise required to carry out even small-scale investigations. Next, we discuss the nature of data in pronunciation research, pointing to ways in which future work can build on advances in corpus research and crowdsourcing. Finally, we consider how these insights pave the way for researchers and developers working to create research-informed, computer-assisted pronunciation teaching resources. We conclude with predictions for future developments

Towards Automatic Speech-Language Assessment for Aphasia Rehabilitation

Speech-based technology has the potential to reinforce traditional aphasia therapy through the development of automatic speech-language assessment systems. Such systems can provide clinicians with supplementary information to assist with progress monitoring and treatment planning, and can provide support for on-demand auxiliary treatment. However, current technology cannot support this type of application due to the difficulties associated with aphasic speech processing. The focus of this dissertation is on the development of computational methods that can accurately assess aphasic speech across a range of clinically-relevant dimensions. The first part of the dissertation focuses on novel techniques for assessing aphasic speech intelligibility in constrained contexts. The second part investigates acoustic modeling methods that lead to significant improvement in aphasic speech recognition and allow the system to work with unconstrained speech samples. The final part demonstrates the efficacy of speech recognition-based analysis in automatic paraphasia detection, extraction of clinically-motivated quantitative measures, and estimation of aphasia severity. The methods and results presented in this work will enable robust technologies for accurately recognizing and assessing aphasic speech, and will provide insights into the link between computational methods and clinical understanding of aphasia.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/140840/1/ducle_1.pd

Dealing with linguistic mismatches for automatic speech recognition

Recent breakthroughs in automatic speech recognition (ASR) have resulted in a word error rate (WER) on par with human transcribers on the English Switchboard benchmark. However, dealing with linguistic mismatches between the training and testing data is still a significant challenge that remains unsolved. Under the monolingual environment, it is well-known that the performance of ASR systems degrades significantly when presented with the speech from speakers with different accents, dialects, and speaking styles than those encountered during system training. Under the multi-lingual environment, ASR systems trained on a source language achieve even worse performance when tested on another target language because of mismatches in terms of the number of phonemes, lexical ambiguity, and power of phonotactic constraints provided by phone-level n-grams. In order to address the issues of linguistic mismatches for current ASR systems, my dissertation investigates both knowledge-gnostic and knowledge-agnostic solutions. In the first part, classic theories relevant to acoustics and articulatory phonetics that present capability of being transferred across a dialect continuum from local dialects to another standardized language are re-visited. Experiments demonstrate the potentials that acoustic correlates in the vicinity of landmarks could help to build a bridge for dealing with mismatches across difference local or global varieties in a dialect continuum. In the second part, we design an end-to-end acoustic modeling approach based on connectionist temporal classification loss and propose to link the training of acoustics and accent altogether in a manner similar to the learning process in human speech perception. This joint model not only performed well on ASR with multiple accents but also boosted accuracies of accent identification task in comparison to separately-trained models

Hizketa-ezagutzan oinarritutako estrategiak, euskarazko online OBHI (Ordenagailu Bidezko Hizkuntza Ikaskuntza) sistemetarako

211 p. (eng) 217 p. (eusk.)Tesi honetan, euskarazko hizketa-ezagutze automatikoaren bi inplementazio aztertzen dira, Ordenagailu Bidezko Hizkuntza Ikaskuntza (OBHI) sistemetarako: Ordenagailu Bidezko Ebakera Lanketa (OBEL) eta Ahozko Gramatika Praktika (AGP). OBEL sistema klasikoan, erabiltzaileari esaldi bat irakurrarazten zaio, eta fonema bakoitzerako puntuazio bat jasotzen du bueltan. AGPn, Hitzez Hitzeko Esaldi Egiaztapena (HHEE) teknika proposatu dugu, ariketak ebatzi ahala egiaztatzen dituen sistema. Bi sistemon oinarrian, esakuntza egiaztatzeko teknikak daude, Goodness of Pronunciation (GOP) puntuazioa, adibididez.Sistema horiek inplementatzeko, eredu akustikoak entrenatu behar dira, eta, horretarako, Basque Speecon-like datu-basea erabili dugu, euskararako publikoki erabilgarri dagoen datu-base bakarra. Eredu akustiko onak lortzearren, datu-basean egokitzapenak egin behar izan dira hiztegi alternatibadun bat sortuz, eta fasekako entrenamendua ere probatu da. % 12.21eko PER (fonemen errore-tasa) lortu da hala.Lehendabiziko sistema laborategiko baldintzetan testatu da, eta emaitza lehiakorrak lortu dira.Hala ere, tesi honetako OBEL eta AGP sistemen helburua da bezero/zerbitzari motako arkitektura batean ezartzea, ikasleek edonondik atzi dezaten. Hori ahalbidetzeko, HTML5eko zehaztapenak erabili dira audioa zerbitzarira grabatu ahala bidaltzeko, eta, gainera, onlineko batezbesteko- eta bariantza-normalizazio cepstraleko (CMVN, Cepstral Mean and Variance Normalisation) teknika berri bat proposatu da erabiltzaileek grabatutako audio-seinaleen kanal desberdintasunen eragina txikiagotzeko. Teknika hori tesi honetan aurkeztutako metodo batean oinarriturik dago: normalizazio anitzeko puntuatzea (MNS, Multi Normalization Scoring), eta onlineko ahots-aktibitatearen detektagailu (VAD, Voice Activity Detector) berri bat ere proposatu da metodo horretan oinarriturik. Azkenik, parametro desberdinak ebaluatu dira neurona-sareak erabiliz, eta ondorioztatu da GOP puntuazioa dela eraginkorrena

Hizketa-ezagutzan oinarritutako estrategiak, euskarazko online OBHI (Ordenagailu Bidezko Hizkuntza Ikaskuntza) sistemetarako

211 p. (eng) 217 p. (eusk.)Tesi honetan, euskarazko hizketa-ezagutze automatikoaren bi inplementazio aztertzen dira, Ordenagailu Bidezko Hizkuntza Ikaskuntza (OBHI) sistemetarako: Ordenagailu Bidezko Ebakera Lanketa (OBEL) eta Ahozko Gramatika Praktika (AGP). OBEL sistema klasikoan, erabiltzaileari esaldi bat irakurrarazten zaio, eta fonema bakoitzerako puntuazio bat jasotzen du bueltan. AGPn, Hitzez Hitzeko Esaldi Egiaztapena (HHEE) teknika proposatu dugu, ariketak ebatzi ahala egiaztatzen dituen sistema. Bi sistemon oinarrian, esakuntza egiaztatzeko teknikak daude, Goodness of Pronunciation (GOP) puntuazioa, adibididez.Sistema horiek inplementatzeko, eredu akustikoak entrenatu behar dira, eta, horretarako, Basque Speecon-like datu-basea erabili dugu, euskararako publikoki erabilgarri dagoen datu-base bakarra. Eredu akustiko onak lortzearren, datu-basean egokitzapenak egin behar izan dira hiztegi alternatibadun bat sortuz, eta fasekako entrenamendua ere probatu da. % 12.21eko PER (fonemen errore-tasa) lortu da hala.Lehendabiziko sistema laborategiko baldintzetan testatu da, eta emaitza lehiakorrak lortu dira.Hala ere, tesi honetako OBEL eta AGP sistemen helburua da bezero/zerbitzari motako arkitektura batean ezartzea, ikasleek edonondik atzi dezaten. Hori ahalbidetzeko, HTML5eko zehaztapenak erabili dira audioa zerbitzarira grabatu ahala bidaltzeko, eta, gainera, onlineko batezbesteko- eta bariantza-normalizazio cepstraleko (CMVN, Cepstral Mean and Variance Normalisation) teknika berri bat proposatu da erabiltzaileek grabatutako audio-seinaleen kanal desberdintasunen eragina txikiagotzeko. Teknika hori tesi honetan aurkeztutako metodo batean oinarriturik dago: normalizazio anitzeko puntuatzea (MNS, Multi Normalization Scoring), eta onlineko ahots-aktibitatearen detektagailu (VAD, Voice Activity Detector) berri bat ere proposatu da metodo horretan oinarriturik. Azkenik, parametro desberdinak ebaluatu dira neurona-sareak erabiliz, eta ondorioztatu da GOP puntuazioa dela eraginkorrena