Automatic Speech Recognition for Low-resource Languages and Accents Using Multilingual and Crosslingual Information

This thesis explores methods to rapidly bootstrap automatic speech recognition systems for languages, which lack resources for speech and language processing. We focus on finding approaches which allow using data from multiple languages to improve the performance for those languages on different levels, such as feature extraction, acoustic modeling and language modeling. Under application aspects, this thesis also includes research work on non-native and Code-Switching speech

Investigation into the underlying linguistic cues of Chinese synaesthesia

Synaesthesia is a neurological condition in which a sensory or cognitive stimulus consistently co-activates another sensory/cognitive quality, in addition to its usual qualities. For example, synaesthetes might see colours when they read words. This additional quality can be from a different modality (e.g., tactile stimuli triggering colour in addition to touch sensations) or from different aspects within the same modality (e.g. visually perceived shape stimuli triggering colour in addition to shape sensations). Coloured language is one of the most common, and most studied types of synaesthesia. The processes that govern such systematic associations of colours and language have been linked to the mechanisms underlying the processing of language. This thesis provides the first psycholinguistic exploration of synaesthesia in Chinese, in particular about how synaesthetic colouring is triggered from Chinese characters and their phonetic spellings in relation to psycholinguistic processes of character recognition. This thesis presents six empirical studies to provide evidence for the following facts: (a) that synaesthetic colouring of Chinese characters is a genuine phenomenon in the Chinese population and may affect as many as 1 in 100 Chinese people, with a (non-significant) female-to-male ratio of about 2:1; (b) that synaesthetic colours are influenced by the characters' constituent radicals (i.e., morphemic units), and (c) also by their associated phonetic spellings (in the spelling systems known as Pinyin and Bopomo); and (d) that even non-synaesthete Chinese speakers colour characters in predictable ways. These findings are discussed in relation to native (L1) versus non-native (L2) Chinese synaesthetes, and to the Chinese versus English systems. Hence, a further issue of this thesis considers how synaesthetic colouring in one's first language may affect their colouring in later-acquired languages. Synaesthetic transfer is discussed in relation to how, and how fast, the transfer can be established to a new language. Taken together, this thesis provides the most detailed information so far available about mechanisms that trigger synaesthetic colours in the Chinese language

The analysis of breathing and rhythm in speech

Speech rhythm can be described as the temporal patterning by which speech events, such as vocalic onsets, occur. Despite efforts to quantify and model speech rhythm across languages, it remains a scientifically enigmatic aspect of prosody. For instance, one challenge lies in determining how to best quantify and analyse speech rhythm. Techniques range from manual phonetic annotation to the automatic extraction of acoustic features. It is currently unclear how closely these differing approaches correspond to one another. Moreover, the primary means of speech rhythm research has been the analysis of the acoustic signal only. Investigations of speech rhythm may instead benefit from a range of complementary measures, including physiological recordings, such as of respiratory effort. This thesis therefore combines acoustic recording with inductive plethysmography (breath belts) to capture temporal characteristics of speech and speech breathing rhythms. The first part examines the performance of existing phonetic and algorithmic techniques for acoustic prosodic analysis in a new corpus of rhythmically diverse English and Mandarin speech. The second part addresses the need for an automatic speech breathing annotation technique by developing a novel function that is robust to the noisy plethysmography typical of spontaneous, naturalistic speech production. These methods are then applied in the following section to the analysis of English speech and speech breathing in a second, larger corpus. Finally, behavioural experiments were conducted to investigate listeners' perception of speech breathing using a novel gap detection task. The thesis establishes the feasibility, as well as limits, of automatic methods in comparison to manual annotation. In the speech breathing corpus analysis, they help show that speakers maintain a normative, yet contextually adaptive breathing style during speech. The perception experiments in turn demonstrate that listeners are sensitive to the violation of these speech breathing norms, even if unconsciously so. The thesis concludes by underscoring breathing as a necessary, yet often overlooked, component in speech rhythm planning and production

Corpus-Based Research on Chinese Language and Linguistics

This volume collects papers presenting corpus-based research on Chinese language and linguistics, from both a synchronic and a diachronic perspective. The contributions cover different fields of linguistics, including syntax and pragmatics, semantics, morphology and the lexicon, sociolinguistics, and corpus building. There is now considerable emphasis on the reliability of linguistic data: the studies presented here are all grounded in the tenet that corpora, intended as collections of naturally occurring texts produced by a variety of speakers/writers, provide a more robust, statistically significant foundation for linguistic analysis. The volume explores not only the potential of using corpora as tools allowing access to authentic language material, but also the challenges involved in corpus interrogation, analysis, and building

Relationships among Constructs of L2 Chinese Reading and Language Background.

Ph.D. Thesis. University of Hawaiʻi at Mānoa 2017

Exploring Cross-linguistic Effects and Phonetic Interactions in the Context of Bilingualism

This Special Issue includes fifteen original state-of-the-art research articles from leading scholars that examine cross-linguistic influence in bilingual speech. These experimental studies contribute to the growing number of studies on multilingual phonetics and phonology by introducing novel empirical data collection techniques, sophisticated methodologies, and acoustic analyses, while also presenting findings that provide robust theoretical implications to a variety of subfields, such as L2 acquisition, L3 acquisition, laboratory phonology, acoustic phonetics, psycholinguistics, sociophonetics, blingualism, and language contact. These studies in this book further elucidate the nature of phonetic interactions in the context of bilingualism and multilingualism and outline future directions in multilingual phonetics and phonology research