The listening talker: A review of human and algorithmic context-induced modifications of speech

International audienceSpeech output technology is finding widespread application, including in scenarios where intelligibility might be compromised - at least for some listeners - by adverse conditions. Unlike most current algorithms, talkers continually adapt their speech patterns as a response to the immediate context of spoken communication, where the type of interlocutor and the environment are the dominant situational factors influencing speech production. Observations of talker behaviour can motivate the design of more robust speech output algorithms. Starting with a listener-oriented categorisation of possible goals for speech modification, this review article summarises the extensive set of behavioural findings related to human speech modification, identifies which factors appear to be beneficial, and goes on to examine previous computational attempts to improve intelligibility in noise. The review concludes by tabulating 46 speech modifications, many of which have yet to be perceptually or algorithmically evaluated. Consequently, the review provides a roadmap for future work in improving the robustness of speech output

Native Speaker Perceptions of Accented Speech: The English Pronunciation of Macedonian EFL Learners

The paper reports on the results of a study that aimed to describe the vocalic and consonantal features of the English pronunciation of Macedonian EFL learners as perceived by native speakers of English and to find out whether native speakers who speak different standard variants of English perceive the same segments as non-native. A specially designed computer web application was employed to gather two types of data: a) quantitative (frequency of segment variables and global foreign accent ratings on a 5-point scale), and b) qualitative (open-ended questions). The result analysis points out to three most frequent markers of foreign accent in the English speech of Macedonian EFL learners: final obstruent devoicing, vowel shortening and substitution of English dental fricatives with Macedonian dental plosives. It also reflects additional phonetic aspects poorly explained in the available reference literature such as allophonic distributional differences between the two languages and intonational mismatch

Perceptual adjustment to time-compressed Speech: a cross-linguistic study

revious research has shown that, when hearers listen to artificially speeded speech, their performance improves over the course of 10-15 sentences, as if their perceptual system was "adapting" to these fast rates of speech. In this paper, we further investigate the mechanisms that are responsible for such effects. In Experiment 1, we report that, for bilingual speakers of Catalan and Spanish, exposure to compressed sentences in either language improves performance on sentences in the other language. Experiment 2 reports that Catalan/Spanish transfer of performance occurs even in monolingual speakers of Spanish who do not understand Catalan. In Experiment 3, we study another pair of languages--namely, English and French--and report no transfer of adaptation between these two languages for English-French bilinguals. Experiment 4, with monolingual English speakers, assesses transfer of adaptation from French, Dutch, and English toward English. Here we find that there is no adaptation from French and intermediate adaptation from Dutch. We discuss the locus of the adaptation to compressed speech and relate our findings to other cross-linguistic studies in speech perception

THE RELATIONSHIP BETWEEN ACOUSTIC FEATURES OF SECOND LANGUAGE SPEECH AND LISTENER EVALUATION OF SPEECH QUALITY

Second language (L2) speech is typically less fluent than native speech, and differs from it phonetically. While the speech of some L2 English speakers seems to be easily understood by native listeners despite the presence of a foreign accent, other L2 speech seems to be more demanding, such that listeners must expend considerable effort in order to understand it. One reason for this increased difficulty may simply be the speaker’s pronunciation accuracy or phonetic intelligibility. If a L2 speaker’s pronunciations of English sounds differ sufficiently from the sounds that native listeners expect, these differences may force native listeners to work much harder to understand the divergent speech patterns. However, L2 speakers also tend to differ from native ones in terms of fluency – the degree to which a speaker is able to produce appropriately structured phrases without unnecessary pauses, self-corrections or restarts. Previous studies have shown that measures of fluency are strongly predictive of listeners’ subjective ratings of the acceptability of L2 speech: Less fluent speech is consistently considered less acceptable (Ginther, Dimova, & Yang, 2010). However, since less fluent speakers tend also to have less accurate pronunciations, it is unclear whether or how these factors might interact to influence the amount of effort listeners exert to understand L2 speech, nor is it clear how listening effort might relate to perceived quality or acceptability of speech. In this dissertation, two experiments were designed to investigate these questions

The role of experience in processing foreign-accented speech

The present study examines the perceptual accommodation of the bilabial stop-consonant voicing contrast (i.e., /b/ vs. /p/), in several English- and Spanish-accented contexts, by native Spanish listeners with different degrees of experience with accented speech. In a series of four experiments, we confronted three potential mechanisms for the perceptual accommodation of foreign-accented sounds. According to the first mechanism (phonetic relaxation), listeners accommodate foreign-accented sounds by relaxing the phonetic boundary between native speech sound categories. According to the second mechanism (phonetic calibration), listeners accommodate foreign-accented sounds by adjusting the location of native perceptual boundaries according to the phonetic realization of native categories in the foreign-accented speech context. Finally, according to the third mechanism (phonetic switching), foreign-accented speech sounds are accommodated by switching to a non-native system of phonetic representations that was previously developed through long-term experience with the speech norm of the foreign accent. Experimental results indicate that Spanish listeners did not relax the phonetic boundary between /b/ and /p/ in an English-accented Spanish context (Experiments 1 and 3). However, they accommodated English-accented Spanish voicing differently, depending on their degree of experience with the English-accented speech norm. When Spanish listeners had little or no experience with the English norm, they calibrated the location of the perceptual boundary between /b/ and /p/ according to the Spanish or English phonetic realization of these sounds in the speech context (Experiment 4). Alternatively, when they had a high degree of experience with English-accented speech, they accommodated English-accented Spanish /b/ and /p/ by using an English-like system of phonetic representations that was not predictable from the phonetic realization of /b/ and /p/ in the speech context (Experiments 1 and 2). These experimental results contribute to a better understanding of the role played by non-native experience in the perceptual accommodation of foreign-accents. In particular, they indicate that native listeners may rely on previous long-term experience with the native language of the foreign-accented speaker to efficiently accommodate foreign-accented speech variability in a different way to which they accommodate speech variability from different native-accented speakers

Production and perception of speaker-specific phonetic detail at word boundaries

Experiments show that learning about familiar voices affects speech processing in many tasks. However, most studies focus on isolated phonemes or words and do not explore which phonetic properties are learned about or retained in memory. This work investigated inter-speaker phonetic variation involving word boundaries, and its perceptual consequences. A production experiment found significant variation in the extent to which speakers used a number of acoustic properties to distinguish junctural minimal pairs e.g. 'So he diced them'—'So he'd iced them'. A perception experiment then tested intelligibility in noise of the junctural minimal pairs before and after familiarisation with a particular voice. Subjects who heard the same voice during testing as during the familiarisation period showed significantly more improvement in identification of words and syllable constituents around word boundaries than those who heard different voices. These data support the view that perceptual learning about the particular pronunciations associated with individual speakers helps listeners to identify syllabic structure and the location of word boundaries

An Experimental Evaluation of Stop-Plosive and Fricative Consonant Intelligibility by Tracheoesophageal Speakers in Quiet and Noise

Despite functional levels of postlaryngectomy communication, individuals who undergo total laryngectomy and tracheoesophageal (TE) puncture voice restoration continue to experience significant communication difficulties in noisy environments. In an effort to identify and further characterize TE speakers’ intelligibility in noise, the current auditory perceptual study investigated stop-plosive and fricative intelligibility of TE speech in quiet and in the presence of multi-talker noise. Eighteen listeners evaluated monosyllabic consonant-vowel-consonant words produced by 14 TE speakers using an open-response paradigm. Our findings indicate that overall intelligibility was significantly lower in noise. Further examination showed a differential effect of noise on intelligibility according to manner and phoneme position. While overall error patterns remained consistent across conditions, voicing distinction was affected differentially according to manner and position. The present investigation provides valuable insight into the difficulties faced by TE speakers in noisy speaking environments, as well as a basis for optimization of counseling and postlaryngectomy voice rehabilitation

Perceptual learning of context-sensitive phonetic detail

[Abstract abbreviated due to inability of DSpace@Cambridge to display phonetic symbols. Please see the full abstract in the attached pdf file.] Although familiarity with a talker or accent is known to facilitate perception, it is not clear what underlies this phenomenon. Previous research has focused primarily on whether listeners can learn to associate novel phonetic characteristics with low-level units such as features or phonemes. However, this neglects the potential role of phonetic information at many other levels of representation. To address this shortcoming, this thesis investigated perceptual learning of systematic phonetic detail relating to higher levels of linguistic structure, including prosodic, grammatical and morphological contexts. Furthermore, in contrast to many previous studies, this research used relatively natural stimuli and tasks, thus maximising its relevance to perceptual learning in ordinary listening situations. This research shows that listeners can update their phonetic representations in response to incoming information and its relation to linguistic-structural context. In addition, certain patterns of systematic phonetic detail were more learnable than others. These findings are used to inform an account of how new information is integrated with prior experience in speech processing, within a framework that emphasises the importance of phonetic detail at multiple levels of representation.This work was funded by an AHRC grant