Language technologies in speech-enabled second language learning games : from reading to dialogue

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 237-244).Second language learning has become an important societal need over the past decades. Given that the number of language teachers is far below demand, computer-aided language learning software is becoming a promising supplement to traditional classroom learning, as well as potentially enabling new opportunities for self-learning. The use of speech technologies is especially attractive to offer students unlimited chances for speaking exercises. To create helpful and intelligent speaking exercises on a computer, it is necessary for the computer to not only recognize the acoustics, but also to understand the meaning and give appropriate responses. Nevertheless, most existing speech-enabled language learning software focuses only on speech recognition and pronunciation training. Very few have emphasized exercising the student's composition and comprehension abilities and adopting language technologies to enable free-form conversation emulating a real human tutor. This thesis investigates the critical functionalities of a computer-aided language learning system, and presents a generic framework as well as various language- and domain-independent modules to enable building complex speech-based language learning systems. Four games have been designed and implemented using the framework and the modules to demonstrate their usability and flexibility, where dynamic content creation, automatic assessment, and automatic assistance are emphasized. The four games, reading, translation, question-answering and dialogue, offer different activities with gradually increasing difficulty, and involve a wide range of language processing techniques, such as language understanding, language generation, question generation, context resolution, dialogue management and user simulation. User studies with real subjects show that the systems were well received and judged to be helpful.by Yushi Xu.Ph.D

The role of formulaic language in the spoken performances of native and nonnative speakers of English.

Lin Ming Sum.Thesis (M.Phil.)--Chinese University of Hong Kong, 2006.Includes bibliographical references (leaves i-xi).Abstracts in English and Chinese.AcknowledgementsAbstract摘要Table of ContentList of tablesList of figuresUnits of measurementChapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- What are formulaic sequences? --- p.2Chapter 1.2 --- Importance of researching on formulaic sequences --- p.4Chapter 1.2.1 --- Wray's works --- p.6Chapter 1.2.2 --- Distinction between native and normative speakers --- p.9Chapter 1.2.3 --- Building blocks of language --- p.10Chapter 1.2.4 --- Processing Unit --- p.11Chapter 1.2.5 --- A pragmatic phenomenon in social interactions --- p.13Chapter 1.3 --- Rationale --- p.17Chapter 1.4 --- Objectives --- p.19Chapter Chapter 2 --- Literature Review --- p.20Chapter 2.1 --- Defining formulaic sequences --- p.20Chapter 2.2 --- The development of research on formulaic sequences --- p.26Chapter 2.2.1 --- Context of formulaic sequences research --- p.26Chapter 2.2.2 --- Milestone works on formulaic language --- p.29Chapter 2.2.2.1 --- The first phase --- p.30Chapter 2.2.2.2 --- The second phase --- p.33Chapter 2.2.2.3 --- The third phase --- p.35Chapter 2.2.3 --- Questions and problems surrounding the field --- p.38Chapter 2.3 --- Relationship between fluency and the use of formulaic sequences --- p.44Chapter 2.4 --- Methods of identification of formulaic sequences --- p.50Chapter 2.4.1 --- The criteria listed by other researchers --- p.50Chapter 2.4.2 --- Intuition --- p.58Chapter 2.4.3 --- Case studies --- p.60Chapter 2.4.4 --- Corpus tools --- p.62Chapter 2.4.5 --- Phonological features --- p.65Chapter 2.4.6 --- Other methodologies --- p.67Chapter Chapter 3 --- Methodology --- p.72Chapter 3.1 --- Research questions --- p.72Chapter 3.1 --- The tasks --- p.73Chapter 3.2 --- The subjects --- p.74Chapter 3.3 --- Data collection --- p.75Chapter Chapter 4 --- Data analysis --- p.77Chapter Chapter 5 --- Findings --- p.80Chapter 5.1 --- Use of formulaic sequences by native and normative speakers --- p.80Chapter 5.2 --- Interaction between NS/NNS factor and task type --- p.81Chapter 5.3 --- Analysis of individual performance --- p.82Chapter 5.4 --- Fluency and the use of formulaic language --- p.85Chapter 5.5 --- Time on task and the use of formulaic language --- p.87Chapter 5.6 --- Use of formulaic language and mean length of each turn --- p.89Chapter 5.7 --- Articulation rate of formulaic sequences --- p.90Chapter 5.8 --- Summary of findings --- p.91Chapter Chapter 6 --- Discussion --- p.93Chapter 6.1 --- Use of formulaic language by native and normative speakers --- p.93Chapter 6.2 --- Interaction between NS/NNS factor and task type --- p.97Chapter 6.3 --- The use of formulaic language and fluency --- p.97Chapter 6.4 --- Time on task and the use of formulaic language --- p.99Chapter 6.5 --- Length of each turn and the use of formulaic language --- p.103Chapter 6.6 --- Rate analysis --- p.105Chapter 6.7 --- Summary of the discussion --- p.106Chapter Chapter 7 --- Limitations --- p.107Chapter Chapter 8 --- Implications of this study --- p.110Chapter 8.1 --- Implications for research --- p.110Chapter 8.2 --- Implications for pedagogy --- p.111Chapter Chapter 9 --- Directions for future research --- p.116Chapter Chapter 10 --- Conclusion --- p.119Bibliography --- p.iAppendices --- p.xiiInstructions for the Picture Story Telling --- p.xiiPicture series 1: Picnic --- p.xiiiPicture series 2: Football --- p.xivInstructions for the Problem-solving task --- p.xvProblem-solving task: The Problem Page --- p.xv

Strategies for Handling Out-of-Vocabulary Words in Automatic Speech Recognition

Nowadays, most ASR (automatic speech recognition) systems deployed in industry are closed-vocabulary systems, meaning we have a limited vocabulary of words the system can recognize, and where pronunciations are provided to the system. Words out of this vocabulary are called out-of-vocabulary (OOV) words, for which either pronunciations or both spellings and pronunciations are not known to the system. The basic motivations of developing strategies to handle OOV words are: First, in the training phase, missing or wrong pronunciations of words in training data results in poor acoustic models. Second, in the test phase, words out of the vocabulary cannot be recognized at all, and mis-recognition of OOV words may affect recognition performance of its in-vocabulary neighbors as well. Therefore, this dissertation is dedicated to exploring strategies of handling OOV words in closed-vocabulary ASR. First, we investigate dealing with OOV words in ASR training data, by introducing an acoustic-data driven pronunciation learning framework using a likelihood-reduction based criterion for selecting pronunciation candidates from multiple sources, i.e. standard grapheme-to-phoneme algorithms (G2P) and phonetic decoding, in a greedy fashion. This framework effectively expands a small hand-crafted pronunciation lexicon to cover OOV words, for which the learned pronunciations have higher quality than approaches using G2P alone or using other baseline pruning criteria. Furthermore, applying the proposed framework to generate alternative pronunciations for in-vocabulary (IV) words improves both recognition performance on relevant words and overall acoustic model performance. Second, we investigate dealing with OOV words in ASR test data, i.e. OOV detection and recovery. We first conduct a comparative study of a hybrid lexical model (HLM) approach for OOV detection, and several baseline approaches, with the conclusion that the HLM approach outperforms others in both OOV detection and first pass OOV recovery performance. Next, we introduce a grammar-decoding framework for efficient second pass OOV recovery, showing that with properly designed schemes of estimating OOV unigram probabilities, the framework significantly improves OOV recovery and overall decoding performance compared to first pass decoding. Finally we propose an open-vocabulary word-level recurrent neural network language model (RNNLM) re-scoring framework, making it possible to re-score lattices containing recovered OOVs using a single word-level RNNLM, that was ignorant of OOVs when it was trained. Above all, the whole OOV recovery pipeline shows the potential of a highly efficient open-vocabulary word-level ASR decoding framework, tightly integrated into a standard WFST decoding pipeline

Lexical Simplification System to Improve Web Accessibility

People with intellectual, language and learning disabilities face accessibility barriers when reading texts with complex words. Following accessibility guidelines, complex words can be identified, and easy synonyms and definitions can be provided for them as reading aids. To offer support to these reading aids, a lexical simplification system for Spanish has been developed and is presented in this article. The system covers the complex word identification (CWI) task and offers replacement candidates with the substitute generation and selection (SG/SS) task. These tasks have followed machine learning techniques and contextual embeddings using Easy Reading and Plain Language resources, such as dictionaries and corpora. Additionally, due to the polysemy present in the language, the system provides definitions for complex words, which are disambiguated by a rule-based method supported by a state-of-the-art embedding resource. This system is integrated into a web system that provides an easy way to improve the readability and comprehension of Spanish texts. The results obtained are satisfactory; in the CWI task, better results were obtained than with other systems that used the same dataset. The SG/SS task results are comparable to similar works in the English language and provide a solid starting point to improve this task for the Spanish language. Finally, the results of the disambiguation process evaluation were good when evaluated by a linguistic expert. These findings represent an additional advancement in the lexical simplification of texts in Spanish and in a generic domain using easy-to-read resources, among others, to provide systematic support to compliance with accessibility guidelinesThis work was supported in part by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors under Grant EPUC3M17, in part by the context of the V PRICIT (Regional Programme of Research and Technological Innovation), and in part by the Accessible Technologies Award-INDRA Technologies and the Fundación Universia (www.tecnologiasaccesibles.com

Precursors to language development in typically and atypically developing infants and toddlers: the importance of embracing complexity

In order to understand how language abilities emerge in typically and atypically developing infants and toddlers, it is important to embrace complexity in development. In this paper, we describe evidence that early language development is an experience-dependent process, shaped by diverse, interconnected, interdependent developmental mechanisms, processes, and abilities (e.g. statistical learning, sampling, functional specialization, visual attention, social interaction, motor ability). We also present evidence from our studies on neurodevelopmental disorders (e.g. Down syndrome, fragile X syndrome, Williams syndrome) that variations in these factors significantly contribute to language delay. Finally, we discuss how embracing complexity, which involves integrating data from different domains and levels of description across developmental time, may lead to a better understanding of language development and, critically, lead to more effective interventions for cases when language develops atypically

The Seven Point Circle and the Twelve Principles: An evidence-based approach to Italian Lyric Diction Instruction

Despite the ubiquitousness of Lyric Diction Instructors (LDIrs) in both the academic and professional opera world, there remains a dearth of research examining the approaches and methods used for Lyric Diction Instruction (LDIn) as well the nonexistence of university programmes through which LDIrs gain profession-specific qualifications and/or certifications. Owing to this paucity of LDIn educational background accreditation and accountability, LDIrs in both educational institutions and opera houses are typically comprised of opera coaches, present or former opera singers, or "native speakers" of the target language. Using the qualitative framework of action research, the study empirically tested my five session, Italian Lyric Diction Course for Opera Singers by examining the validity and efficaciousness of its design, materials, course content, and pedagogical approach of explicit articulatory instruction. Rather than focusing on the empirical testing itself, this article focuses on the underlying pedagogical framework, i.e., The Seven Point Circle (7PC) and the ethical code of conduct, i.e., The Twelve Point Circle (12PC) derived from my M.A. thesis study. Data collection instruments included: semi-structured participant interviews, audio recording, transcribing of the classes, and an invited panel of eight observer-feedback experts from the fields of foreign language pedagogy, pronunciation instruction, and Italian language instruction

A Sound Approach to Language Matters: In Honor of Ocke-Schwen Bohn

The contributions in this Festschrift were written by Ocke’s current and former PhD-students, colleagues and research collaborators. The Festschrift is divided into six sections, moving from the smallest building blocks of language, through gradually expanding objects of linguistic inquiry to the highest levels of description - all of which have formed a part of Ocke’s career, in connection with his teaching and/or his academic productions: “Segments”, “Perception of Accent”, “Between Sounds and Graphemes”, “Prosody”, “Morphology and Syntax” and “Second Language Acquisition”. Each one of these illustrates a sound approach to language matters

Future non-native English as a second language teachers : the impact of language proficiency on their use of English in the classroom

Cette recherche avait pour but de mettre en lumière la perception des enseignants d’anglais langue seconde non anglophones quant à leurs compétences langagières et les impacts que ceux-ci ont sur l’utilisation de l’anglais en classe avec leurs élèves. Toutefois, les impacts découverts n’étaient pas nécessairement liés aux compétences langagières des futurs enseignants d’anglais qui ont participé à cette étude, mais plutôt liés à l’utilisation de l’anglais comme langue d’enseignement dans le contexte de l’enseignement et de l’apprentissage de l’anglais. Cette étude a eu lieu dans la province de Québec dans une université francophone de taille moyenne située en région. Plusieurs futurs enseignants d’anglais langue seconde ont été interrogés à l'aide d'un questionnaire administré en ligne qu'ils ont pu remplir au moment qui leur convenait. De ces futurs enseignants, certaines personnes ont accepté d'être interrogés à la suite du questionnaire. Même si les deux langues nationales du Canada sont l'anglais et le français, la plupart des enseignants d'anglais langue seconde dans la province de Québec ne sont pas anglophones. Pour la plupart, ils vivent dans des communautés francophones et enseignent dans des écoles françaises, ce qui signifie que communiquer en français est nécessaire en dehors des cours d’anglais. Les données de la recherche ont été analysées et interprétées à l'aide d'une méthode descriptive pour le questionnaire et pour ce qui est des entretiens semi-dirigés, une approche d’analyse thématique a été privilégiée. Cette étude a révélé certains défis que les futurs enseignants d'anglais langue seconde non anglophones vivent au Québec ainsi que la perception qu'ils ont quant à leurs habiletés langagières. Bien que tous les participants aient indiqué qu'ils avaient une grande aisance à s’exprimer en anglais, certains d'entre eux ont indiqué qu'ils devaient parfois utiliser le français pour enseigner l'anglais, ce qui réduisait le temps consacré à l’utilisation de l'anglais en classe. Enfin, les résultats de cette recherche ont également exposé divers facteurs pouvant expliquer l'utilisation de la langue maternelle des élèves en classe.Abstract : The main purpose of this study was to better understand the perceptions of non-native future ESL teachers in relation to their language proficiency in English and the impact their proficiency had on their use of English with their pupils while on practicum. However, the impacts that were uncovered were not necessarily related to the language proficiency of the student teachers who participated in the study, but rather linked to the use of English as the language of instruction in the context of language teaching. This study took place in a medium-size French regional university in Québec where several pre-service teachers were surveyed using an online questionnaire that they took at their convenience. As a follow up to the questionnaire, a few of these future teachers agreed to be interviewed. Even though English is one of the two national languages of Canada, most ESL teachers in the province of Québec are non-native English speakers. For the most part, they live in French communities and teach in French schools, which means communicating in French is necessary outside their classrooms. The research data were analysed and interpreted using a descriptive approach for the questionnaire and a thematic analysis approach for the semi-structured interviews. The study revealed some of the challenges of being a future non-native ESL teacher in Québec and the perceptions they have towards their English proficiency. Although all the participants indicated that they were very fluent in English, some of them reported that they needed to use French at times to teach English, which reduced the amount of time spent using English in the classroom. Finally, the findings also exposed various factors that can account for the use of the mother tongue of the pupils in the classroom