The phonetics of second language learning and bilingualism

This chapter provides an overview of major theories and findings in the field of second language (L2) phonetics and phonology. Four main conceptual frameworks are discussed and compared: the Perceptual Assimilation Model-L2, the Native Language Magnet Theory, the Automatic Selection Perception Model, and the Speech Learning Model. These frameworks differ in terms of their empirical focus, including the type of learner (e.g., beginner vs. advanced) and target modality (e.g., perception vs. production), and in terms of their theoretical assumptions, such as the basic unit or window of analysis that is relevant (e.g., articulatory gestures, position-specific allophones). Despite the divergences among these theories, three recurring themes emerge from the literature reviewed. First, the learning of a target L2 structure (segment, prosodic pattern, etc.) is influenced by phonetic and/or phonological similarity to structures in the native language (L1). In particular, L1-L2 similarity exists at multiple levels and does not necessarily benefit L2 outcomes. Second, the role played by certain factors, such as acoustic phonetic similarity between close L1 and L2 sounds, changes over the course of learning, such that advanced learners may differ from novice learners with respect to the effect of a specific variable on observed L2 behavior. Third, the connection between L2 perception and production (insofar as the two are hypothesized to be linked) differs significantly from the perception-production links observed in L1 acquisition. In service of elucidating the predictive differences among these theories, this contribution discusses studies that have investigated L2 perception and/or production primarily at a segmental level. In addition to summarizing the areas in which there is broad consensus, the chapter points out a number of questions which remain a source of debate in the field today.https://drive.google.com/open?id=1uHX9K99Bl31vMZNRWL-YmU7O2p1tG2wHhttps://drive.google.com/open?id=1uHX9K99Bl31vMZNRWL-YmU7O2p1tG2wHhttps://drive.google.com/open?id=1uHX9K99Bl31vMZNRWL-YmU7O2p1tG2wHAccepted manuscriptAccepted manuscrip

Acoustic cue weighting strategy and the impact of training for cochlear implant users and normal hearing listeners with acoustic simulation

This project investigates the impact and plasticity of perceptual cue weighting strategy for normal hearing (NH) listeners with cochlear implant (CI) acoustic simulation and CI users. It is hypothesised that how listeners allocate perceptual attention on different speech cues is related to how accurately and effectively they can restore the phonemic structures from the acoustic inputs. Therefore, it can be beneficial to use auditory training to guide listeners’ attention to the more reliable and informative cues for their own specific language, in order to improve speech recognition and ease listening effort. The first part of this project investigated the impact of perceptual weighting strategy for both groups of listeners. Listeners’ sentence recognition and pupillary response (taken as a measure of listening effort) were measured. They were then taken into the same model with listeners’ acoustic cue weighting ratio and auditory sensitivity to explore their relation. The second aim of this project was to examine the possibility of using auditory training to change listeners’ acoustic cue weighting strategy towards an optimal one. A distributional training method was used here, with the sampling of spectral and temporal cues in the training word stimuli manipulated in a way that only the spectral cue followed a bimodal distribution that resembled the natural speech. This was to increase the saliency of the spectral cue, in order to direct listeners’ attention to the spectral cue. Sentence recognition performance in quiet, acoustic cue weighting strategy and pupillary responses were measured before and after the training to examine the effectiveness of the training. Findings from this project will extend the current understanding on CI users’ perceptual cue weighting strategy and provide inspiration for a more comprehensive rehabilitation scheme for CI users

Perceptual strategies underlying second language acquisition

The literature suggests that listeners do not pay equal attention to all available acoustic information. Instead, when perceiving speech, they place more importance on some acoustic cues than others (Francis & Nusbaum, 2002). The patterns of weights assigned to different cues appear to change with increased linguistic experience, not only in the first language (L1; Mayo et al., 2003) but also in the second language (L2; Chandrasekaran et al., 2010). However, the role of attention and salience in cue weighting is still under discussion. This thesis presents a series of experiments designed to test this hypothesis in the context of native English speakers and Mandarin Chinese learners of English. First, we compared how prior experience (language background, musical training, and their interaction) shapes cue weighting strategies and tested whether the cue weighting of different cues reflects the direction of attention towards them or their salience. Compared to English speakers, Mandarin speakers showed enhanced attention to and preferential use of pitch across tasks but no increased pitch salience. Effects of musical training were contingent upon participants’ L1. We also demonstrated that perceptual strategies are not consistent across tasks, suggesting they are not driven by domain-general abilities. Second, since acoustic cues play different roles across languages, learning a new language might require listeners to make greater use of L1-irrelevant dimensions. We designed a targeted training focused on redirecting listeners’ attention towards an L2-relevant acoustic cue. Although the observed training effects were not long-lasting, we showed that perceptual strategies in categorizing L2 prosody could be adjusted with as little as three hours of training. This finding has the potential to inform the development of L2 learning paradigms targeting specific auditory challenges experienced by learners. Overall, this thesis provides novel insights into the long-debated role of dimension-selective attention and dimensional salience in cue weighting

Binding and unbinding the auditory and visual streams in the McGurk effect

International audienceSubjects presented with coherent auditory and visual streams generally fuse them into a single per- cept. This results in enhanced intelligibility in noise, or in visual modification of the auditory per- cept in the McGurk effect. It is classically considered that processing is done independently in the auditory and visual systems before interaction occurs at a certain representational stage, resulting in an integrated percept. However, some behavioral and neurophysiological data suggest the existence of a two-stage process. A first stage would involve binding together the appropriate pieces of audio and video information before fusion per se in a second stage. Then it should be possible to design experiments leading to unbinding . It is shown here that if a given McGurk stimulus is preceded by an incoherent audiovisual context, the amount of McGurk effect is largely reduced. Various kinds of incoherent contexts (acoustic syllables dubbed on video sentences or phonetic or temporal modi- fications of the acoustic content of a regular sequence of audiovisual syllables) can significantly reduce the McGurk effect even when they are short (less than 4s). The data are interpreted in the framework of a two-stage "binding and fusion" model for audiovisual speech perception

THE USE OF SEGMENTAL AND SUPRASEGMENTAL INFORMATION IN LEXICAL ACCESS: A FIRST- AND SECOND-LANGUAGE CHINESE INVESTIGATION

The present study investigated first language (L1) and second language (L2) Chinese categorization of tones and segments and use of tones and segments in lexical access. Previous research has shown that English listeners rely more on pitch height than pitch direction when perceiving lexical tones; however, it remains unclear if this superior use of pitch height aids English-speaking learners of Chinese in identifying the tones of Chinese that differ in initial pitch height. The present study aimed to investigate this issue to determine whether this pitch height advantage aids English-speaking Chinese learners in identifying the tones of Chinese by looking at the time course of categorization of Chinese tones that differed in initial pitch as well as segments. A norming study was first conducted to investigate the duration of acoustic input needed to hear tone and segment (rime) distinctions. In a gated AX discrimination task, native Chinese listeners and naïve English listeners heard increasingly large fragments of tonal pairs and segmental pairs that varied in the expected disambiguation point. The results of this norming study were used to select tonal and segmental stimulus pairs were controlled (as best as is possible) for the disambiguation timing in the next two experiments. Experiment 1 investigated the time course of categorization of tones and segments using a forced-choice gating task designed to tap into listeners’ identification of fragment categories taken from syllables that differ only in tones or only in segments. Native Chinese listeners and L1-English L2-Chinese listeners heard a single fragment of a Chinese word and identified either the tone or the rime of the heard fragment from two presented options. The results showed that the segmental contrasts had higher accuracy than tonal contrasts for both groups. The L2-Chinese listeners performed comparably to the native listeners on both tonal and segmental contrasts, and L2 Chinese listeners showed no advantage over native listeners. The second goal of this study was to investigate the time course of the use of tones and segments in lexical access. Previous work has shown that native Chinese listeners use tones and segments simultaneously in lexical access. Previous work on how second language learners of Chinese use tones in lexical access compared to segments showed that tones and segments are used at the same time; however, work in the segmental domain suggest that this should not be the case, and learners should struggle to use the new tones in online lexical access. As such, this work aimed to reinvestigate the timing of use of tones and segments in second language Chinese, as well as to compare learners’ use of tones and segments to native listeners with a highly time-sensitive measure: visual-world eye-tracking. Experiment 2 investigated the time course of use of tones and segments in online spoken word recognition for L1 and L2 groups. The same segmental and tonal pairs used in Experiment 1 were used in a visual-world eye-tracking experiment. Native Chinese listeners and L1-English L2-Chinese listeners saw two pairs of words displayed as corresponding images: one tonal pair and one segmental pair. Eye movements were recorded as participants heard a single target word in isolation and clicked on the corresponding picture. The eye movement data revealed that native Chinese listeners use tones and segments to begin constraining the lexical search at approximately the same time, and tonal information constrained the search more rapidly than did segments. The L2-Chinese learners showed segmental use comparable to that of native listeners; however, their tonal use was delayed by approximately 100 ms. In terms of speed, learners also showed more rapid use of tones in constraining the lexical search, although tones and segments were used to constrain the lexical search more slowly than they did for native listeners. These results are discussed in relation to recent L1 studies on lexical access of tones and segments and computational modeling of suprasegmental information. The results of this research is in line with previous work that showed tones and segments are used to constrain lexical access simultaneously; however, the current work does not support the conclusion that tones and segments are used in the same way, with tones constraining the lexical search faster than segments. It is suggested that the cause of this tone speed advantage is the number of competitors removed from competition when the processor is certain of a tone as opposed to certain of a segment or even rime. The present results also speak to the literature on the use of segmental and suprasegmental information in a second language and suggest that the timing of use of different cues to lexical identity is dependent on if that cue is used in the L1, since segments were processed at the same time as native speakers while tones were delayed. Speed of use seems to be independent of whether or not it is used in the L1, with both tones and segments being processed slower overall compared to native listeners

Learning and Retention of Novel Words in Musicians and Non-Musicians: The Impact of Enriched Auditory Experience on Behavioral Performance and Electrophysiologic Measures

abstract: Music training is associated with measurable physiologic changes in the auditory pathway. Benefits of music training have also been demonstrated in the areas of working memory, auditory attention, and speech perception in noise. The purpose of this study was to determine whether long-term auditory experience secondary to music training enhances the ability to detect, learn, and recall new words. Participants consisted of 20 young adult musicians and 20 age-matched non-musicians. In addition to completing word recognition and non-word detection tasks, each participant learned 10 nonsense words in a rapid word-learning task. All tasks were completed in quiet and in multi-talker babble. Next-day retention of the learned words was examined in isolation and in context. Cortical auditory evoked responses to vowel stimuli were recorded to obtain latencies and amplitudes for the N1, P2, and P3a components. Performance was compared across groups and listening conditions. Correlations between the behavioral tasks and the cortical auditory evoked responses were also examined. No differences were found between groups (musicians vs. non-musicians) on any of the behavioral tasks. Nor did the groups differ in cortical auditory evoked response latencies or amplitudes, with the exception of P2 latencies, which were significantly longer in musicians than in non-musicians. Performance was significantly poorer in babble than in quiet on word recognition and non-word detection, but not on word learning, learned-word retention, or learned-word detection. CAEP latencies collapsed across group were significantly longer and amplitudes were significantly smaller in babble than in quiet. P2 latencies in quiet were positively correlated with word recognition in quiet, while P3a latencies in babble were positively correlated with word recognition and learned-word detection in babble. No other significant correlations were observed between CAEPs and performance on behavioral tasks. These results indicated that, for young normal-hearing adults, auditory experience resulting from long-term music training did not provide an advantage for learning new information in either favorable (quiet) or unfavorable (babble) listening conditions. Results of the present study suggest that the relationship between music training and the strength of cortical auditory evoked responses may be more complex or too weak to be observed in this population.Dissertation/ThesisDoctoral Dissertation Speech and Hearing Science 201

Investigating the neural mechanisms underlying auditory and audio-visual perception in younger and older adults

This thesis aimed to address questions in two distinct areas of research in ageing and cognitive neuroscience. Firstly, given that the pre-stimulus state of cortical oscillations had been shown to predict behavioural and neural responses, we addressed the question of whether pre-stimulus oscillatory mechanisms change or remain consistent in the ageing brain. Secondly, previous research had shown that Audio-visual (AV) speech influences the amplitude and latency of evoked activity. Our research addressed the questions of whether/how AV enhancement and visual predictability of AV speech is represented in evoked activity in noisy listening conditions, and whether such Electroencephalographic (EEG) signatures remain stable with age. In Chapter 3 we investigated the consistency of how pre-stimulus activity influences auditory frequency discrimination performance in young and older participants. In both groups the power of pre-stimulus activity influenced the encoding of sensory evidence reflected by early evoked components, while the phase influenced choice formation in later-activated EEG components. Importantly, for the early EEG components we did not find evidence for a systematic difference in the time scales of the perceptually relevant pre-stimulus activity. In the later-activated EEG component we found a trend for perceptually relevant rhythmic activity to arise from slower frequencies in the ageing brain. At the same time our data replicate previous findings of a significant age-related slowing of Auditory Evoked Potential (AEP) latency, modulations of AEP amplitudes, and a flattening of the spectral profile of EEG activity. In Chapter 4, we investigated the consistency of behaviour and evoked activity underlying AV speech integration in a speech-in-noise discrimination task in younger and older adults. Behaviourally, younger and older adults performed comparably. Performance was greater for Audio-visually informative (AVinf) speech compared to Auditory-only informative (AOinf) speech across groups and noise levels, and was poorer at low noise levels. AV enhancement was greater in high noise levels, across all participants, and older adults derived greater AV enhancement compared to younger adults (an effect that was consistent across noise levels). In terms of visual predictability, we found that word discrimination performance was greater for target words with non-labial initial phonemes (assumed least visually predictive), compared to labial initial phonemes (assumed most visually predictive). Furthermore, we found that AV enhancement was greater for labial initial phonemes, compared to non-labial initial phonemes, and this was consistent across age groups.Neurally, we found that AV enhancement is represented by a centro-parietal P3-like activity in older adults and an N4-like fronto-central activity in younger adults, but found that this activity did not correlate with behavioural AV enhancement. Our results point to distinct patterns of late evoked activity underlying AV enhancement between younger and older adults, possibly representing distinct cognitive (memory) strategies in predicting upcoming target stimuli. At the same time our data replicate previous findings of a significant age-related slowing of AEP latency, modulations of AEP amplitudes, and a flattening of the spectral profile of EEG activity. In Chapter 5 we investigated the consistency of evoked activity underlying the visual predictability of AV speech. We found that visual predictability was reflected by late fronto-central negativity in older adults, but not in younger adults. However, we did not find evidence of an interaction between visual predictability and AV enhancement in terms of evoked activity, raising further questions about how visual predictability of speech is represented the brain’s electrophysiology. Our results point to distinct patterns of late evoked activity underlying visual predictability of visual speech, again possibly reflecting differential strategies in predictive coding. In summary, the results of this thesis demonstrate that pre-stimulus mechanisms in auditory pitch perception remain consistent in the younger and older adult brain, while spectral dynamics change with age. Our results also replicate previous work demonstrating age-related delays in peak latency, and changes in peak amplitude, of early auditory evoked activity. And lastly, we demonstrate that differences in the EEG signatures of AV enhancement between younger and older adults emerge in late evoked activity, and that visual predictability of speech is represented in late evoked activity only in older adults

Individual differences in phonetic imitation and their role in sound change

This paper explores the possibility that the spread of sound change within a community correlates with individual differences in imitation capacities. The devoicing of labiodental fricatives in Dutch serves as a case study of an ongoing sound change showing regional and individual variation. The imitation capacities of Dutch speakers born and raised in five regions of the Dutch language area were investigated in a forced imitation task (Study 2) and a spontaneous imitation task (Study 3), and compared to baseline productions (Study 1) of the variable undergoing sound change. Results showed that the leaders of sound change in each region were significantly less accurate in imitating model talkers-when they were instructed to-than conservative speakers, but they were more inclined to spontaneously imitate talkers. These insights are discussed in view of the literature on different types and measures of imitation capacities, on the actors of sound change and the two apparently paradoxical features of the language system: Its stability and its potential for sound change

Negative vaccine voices in Swedish social media

Vaccinations are one of the most significant interventions to public health, but vaccine hesitancy creates concerns for a portion of the population in many countries, including Sweden. Since discussions on vaccine hesitancy are often taken on social networking sites, data from Swedish social media are used to study and quantify the sentiment among the discussants on the vaccination-or-not topic during phases of the COVID-19 pandemic. Out of all the posts analyzed a majority showed a stronger negative sentiment, prevailing throughout the whole of the examined period, with some spikes or jumps due to the occurrence of certain vaccine-related events distinguishable in the results. Sentiment analysis can be a valuable tool to track public opinions regarding the use, efficacy, safety, and importance of vaccination