Developing Speech Recognition and Synthesis Technologies to Support Computer-Aided Pronunciation Training for Chinese Learners of English

PACLIC 23 / City University of Hong Kong / 3-5 December 200

Lexical exposure and word-from encoding in 1.5-year-olds

In this study, 1.5-year-olds were taught a novel word. Some children were familiarized with the word's phonological form before learning the word's meaning. Fidelity of phonological encoding was tested in a picture-fixation task using correctly pronounced and mispronounced stimuli. Only children with additional exposure in familiarization showed reduced recognition performance given slight mispronunciations relative to correct pronunciations; children with fewer exposures did not. Mathematical modeling of vocabulary exposure indicated that children may hear thousands of words frequently enough for accurate encoding. The results provide evidence compatible with partial failure of phonological encoding at 19 months of age, demonstrate that this limitation in learning does not always hinder word recognition, and show the value of infants' word-form encoding in early lexical development

The Phonetic Specificity of British English-Learning Infants’ Word Form Recognition in Their First Year of Life

Consonants and vowels have been proposed to have distinct functions in speech perception: a consonant bias for lexical processing and a vowel bias for syntactic/prosodic processing (Nespor et al., 2003). Research in adults has consistently demonstrated that consonants have a privileged role in various lexical-level experiments across most languages. However, cross-linguistic differences have been found in the developmental trajectory of the consonant bias. For example, whilst French-learning infants display a consonant advantage in lexical processing tasks by their first birthday (e.g., Poltrock & Nazzi, 2015), British English-learning infants show an equal sensitivity to consonants and vowels until the age of 30 months (e.g., Floccia et al., 2014). Although the lexical and/or the acoustic-phonetic properties of an infant’s native language have been hypothesised to explain such variations, additional cross-linguistic tests of the consonant bias and its potential links to these factors are required. The present thesis explored this by using two experimental paradigms to further examine the phonetic specificity of British English-learning infants’ word form recognition at the onset of lexical acquisition. Experiments 1 to 3 established an equal preference for consonant and vowel mispronunciations of familiar word forms, presented either in isolation or in list form, in 5-, 11-, and 12-month-old infants using the head-turn preference procedure. Experiments 4 and 5 used an eye-tracking methodology to measure whether the congruent presentation of audio and visual speech signals led to a consonant bias in 12-month-olds’ word form recognition. An audiovisual benefit was found, with infants discriminating between phonetic mispronunciations, but only when they viewed a speaker articulate alterations of a single familiar word form. Additionally, neither acoustic factors (Experiment 1) nor lexical factors (Experiments 2 to 5) were found to influence infants’ preferences. Together, the results of this thesis provide further evidence that initial lexical processes vary cross linguistically

Ecological approaches to speech perception

A literature review demonstrates that very general scientific presuppositions which Whitehead regarded as instances of the fallacy of misplaced concreteness and Bohm labelled 'fragmentation' characterise current research in speech perception. It is then argued that the following two hypotheses allow these presuppositions to be tested 1 For every exclusively auditory experiment in speech perception, an attempted replication to the audio-visual case can be conducted which will result in a failure to replicate. 2 If an effect that is obtained through dubbing can also be produced with at least contrinsically related optical and acoustic signals, an experiment can be conducted which will result in a failure to replicate from dubbing to the more naturalistic case. A series of twelve experiments provides strong evidence to support both of the hypotheses. This is taken to establish that future speech research must orientate itself relative to naturalistic speech perception and not the dimensions of Physics. Some implications of this reorientation are discussed

Enhancing the pronunciation of problematic English consonants for Spanish learners through intralingual dubbing activities

En esta tesis doctoral se proporciona un estudio sobre el potencial de las actividades de doblaje intralingüístico en la mejora de la pronunciación de fonemas consonánticos problemáticos del inglés para estudiantes españoles, junto con otras consideraciones adicionales, como el grado en que esos fonemas resultan problemáticos para los participantes de la investigación (n=71) y un análisis pormenorizado de sus puntos de vista y opiniones sobre la actividad de doblaje.Para ello, un Grupo Experimental (GE; n=37) y un Grupo Control (GC; n=34) se grabaron en diferentes fases del estudio (GE: fase pre-test, doblajes, y fase post-test; GC: pre-test y post-test) con el fin de obtener datos relevantes y útiles sobre su pronunciación. Todos los datos recopilados han sido analizados con el Statistical Package for Social Sciences, (SPSS; v.25), aplicando el test de Wilcoxon para comparaciones intragrupales, y el U-test de Mann-Whitney para las comparaciones entre grupos. Además, los participantes de la investigación completaron dos cuestionarios para obtener información adicional al respecto.Como conclusión, la pronunciación general del GE mejoró significativamente en la mayoría de los fonemas consonánticos problemáticos durante y después de realizar las actividades de doblaje, mientras que el GC no mostró ninguna mejora significativa en su pronunciación. Además, la mayoría de los participantes del GE mostraron opiniones muy positivas hacia la actividad de doblaje, destacando su valor motivador e innovador en el aprendizaje de lenguas, así como su utilidad para mejorar las habilidades orales.<br /

Phonetic detail in the developing lexicon

Although infants show remarkable sensitivity to linguistically relevant phonetic variation in speech, young children sometimes appear not to make use of this sensitivity. Here, children's knowledge of the sound-forms of familiar words was assessed using a visual fixation task. Dutch 19-month-olds were shown pairs of pictures and heard correct pronunciations and mispronunciations of familiar words naming one of the pictures. Mispronunciations were word-initial in Experiment 1 and word-medial in Experiment 2, and in both experiments involved substituting one segment with [d] (a common sound in Dutch) or [g] (a rare sound). In both experiments, word recognition performance was better for correct pronunciations than for mispronunciations involving either substituted consonant. These effects did not depend upon children's knowledge of lexical or nonlexical phonological neighbors of the tested words. The results indicate the encoding of phonetic detail in words at 19 months

Budding bilinguals: Investigating bilingual infants’ language acquisition

Almost one in six children in Canada grow up hearing two languages. Bilingual children frequently encounter switches between their languages, and must learn words in both. My thesis tackles three central issues of early bilingual language development. Manuscript 1 investigated whether infants can detect language switches at the level of individual words. This ability could help infants cope with rapid language switching in their language input, and prepare them, once they start speaking, to have control over which language they use. I tested bilingual and monolingual 8- to 12-month-olds’ interest in single-language trials (“milk…dog”) and switched-language trials (“milk…chien”). Neither group showed evidence of differentiating between contexts, suggesting that detection of single-word language switching is more difficult than previously assumed based on prior research investigating multi-sentence language switching. Manuscript 2 examined whether infants can associate a person with the language that person is speaking. Some theories of early bilingualism propose that person–language associations help infants navigate their bilingual language input. I tested 5- to 18-month-olds’ surprisal when a speaker switches to a different language. Results showed no evidence that infants spontaneously associate a person with a language. This contrasts with common but outdated advice to caregivers to choose a single language when speaking to their child, and is consistent with research showing that bilingual infants learn languages from a variety of family language strategies. Manuscript 3 examined how bilingual infants mentally represent the sounds in familiar words. Bilingual and monolingual infants were tested to examine whether bilingual infants are more or less sensitive to mispronunciations than monolinguals. I tested 24- to 26-month-olds in a looking-while-listening task. I found a robust mispronunciation effect in bilingual and monolingual toddlers, indicating that bilinguals’ encoding of sounds in familiar words is phonetically detailed in a similar way as monolinguals’, despite bilinguals having to navigate a more complex phonetic environment. In these three manuscripts, I found that bilingual language acquisition is similar to monolingual development in many aspects, and rigorous testing of assumptions about bilingual language acquisition is needed to learn about the mechanisms bilingual infants use to acquire language

Analysis of the GOP metric for assessing non-native Spanish pronunciation in the SAMPLE corpus

Este trabajo consiste en el análisis de los resultados obtenidos en la evaluación de pronunciación a nivel fonema utilizando el algoritmo Forced GOP que ha sido implementado para ello. Se ha hecho uso de locuciones de diferentes oraciones realizadas por distintos hablantes, las cuales han sido grabadas y anotadas dentro del corpus SAMPLE. Este corpus fue desarrollado dentro de nuestro grupo de investigación en colaboración con personas del ámbito lingüista. Se ha trabajado con los datos obtenidos para identificar posibles mejoras, se hacen varias observaciones en el comportamiento de la métrica y se discute la dependencia a nivel fonema y hablante que sugiere el establecimiento de posibles umbrales para mejorar su rendimiento. Además se agregan propuestas en base a los datos de loglikelihood que arroja la FGOP y se aplican una serie de reglas para establecer un nuevo parámetro que permita dar una calificación por cada fonema. Estas calificaciones permiten generar una calificación global de pronunciación a nivel hablante. Las puntuaciones globales se han contrastado con los resultados de la FGOP y las evaluaciones realizadas por jueces humanos.Departamento de Informática (Arquitectura y Tecnología de Computadores, Ciencias de la Computación e Inteligencia Artificial, Lenguajes y Sistemas Informáticos)Máster en Investigación en Tecnologías de la Información y las Comunicacione