The Perception of Prosody in English-Speaking Children with Cochlear Implants: A Systematic Review

Objective: The goal of this paper was to systematically review literature in order to investigate the perception of prosody in English-speaking children with cochlear implants. Methods: A comprehensive search utilizing various peer-reviewed databases accessible through the City University of New York (CUNY) Graduate Center Library was conducted to identify relevant studies. Inclusion criteria included studies that examined prosody perception in pre-and post-lingually deafened children with cochlear implants. Children who utilized unilateral, bilateral, and bimodal configurations of cochlear implants were therefore included in this search. Results: 9 studies met the inclusion criteria for this systematic review. The findings demonstrated both negative and positive outcomes for pediatric users of cochlear implants. Of the 9 studies included in this systematic review, 6 (66%) included an outcome measure that assessed emotion perception, and 3 (33%) included an outcome measure that examined specific domains of speech prosody perception. Additionally, 2 of the 9 (22%) included studies specifically investigated the connection between music and the perception of emotional speech prosody. Discussion: Results support the use and continued development of intensive (re)habilitation emphasizing suprasegmental and paralinguistic aspects of speech through prosody perception measures sensitive to both emotional and linguistic components. Positive effects of music training were also found in audio-only conditions for the perception of emotional prosody. Future research needs to be based on larger sample sizes, and should offer more alternative choices in the identification of emotional or prosodic cues, heightening prosody classification difficulty for prosody perception tasks. Incorporating differing levels of background noise and reverberation during prosody perception tasks is also recommended to simulate situations which are more representative of complex listening situations encountered by pediatric cochlear implant users. Conclusions: Performance on emotion recognition and other aspects of prosody perception including music perception is generally poorer in children with cochlear implants than in participants in comparison groups, such as normal-hearing children. Specifically, the findings of this systematic review support the use and validation of intensive (re)habilitation measures emphasizing suprasegmental and paralinguistic aspects of speech, as well as emotion and music prosody perception

The development of emotion recognition from facial expressions and non-linguistic vocalizations during childhood

Sensitivity to facial and vocal emotion is fundamental to children's social competence. Previous research has focused on children's facial emotion recognition, and few studies have investigated non-linguistic vocal emotion processing in childhood. We compared facial and vocal emotion recognition and processing biases in 4- to 11-year-olds and adults. Eighty-eight 4- to 11-year-olds and 21 adults participated. Participants viewed/listened to faces and voices (angry, happy, and sad) at three intensity levels (50%, 75%, and 100%). Non-linguistic tones were used. For each modality, participants completed an emotion identification task. Accuracy and bias for each emotion and modality were compared across 4- to 5-, 6- to 9- and 10- to 11-year-olds and adults. The results showed that children's emotion recognition improved with age; preschoolers were less accurate than other groups. Facial emotion recognition reached adult levels by 11 years, whereas vocal emotion recognition continued to develop in late childhood. Response bias decreased with age. For both modalities, sadness recognition was delayed across development relative to anger and happiness. The results demonstrate that developmental trajectories of emotion processing differ as a function of emotion type and stimulus modality. In addition, vocal emotion processing showed a more protracted developmental trajectory, compared to facial emotion processing. The results have important implications for programmes aiming to improve children's socio-emotional competence

An exploration of sarcasm detection in children with Attention Deficit Hyperactivity Disorder

This document is the Accepted Manuscript version of the following article: Amanda K. Ludlow, Eleanor Chadwick, Alice Morey, Rebecca Edwards, and Roberto Gutierrez, ‘An exploration of sarcasm detection in children with Attention Deficit Hyperactivity Disorder’, Journal of Communication Disorders, Vol. 70: 25-34, November 2017. Under embargo. Embargo end date: 31 October 2019. The Version of Record is available at doi: https://doi.org/10.1016/j.jcomdis.2017.10.003.The present research explored the ability of children with ADHD to distinguish between sarcasm and sincerity. Twenty-two children with a clinical diagnosis of ADHD were compared with 22 age and verbal IQ matched typically developing children using the Social Inference–Minimal Test from The Awareness of Social Inference Test (TASIT, McDonald, Flanagan, & Rollins, 2002). This test assesses an individual’s ability to interpret naturalistic social interactions containing sincerity, simple sarcasm and paradoxical sarcasm. Children with ADHD demonstrated specific deficits in comprehending paradoxical sarcasm and they performed significantly less accurately than the typically developing children. While there were no significant differences between the children with ADHD and the typically developing children in their ability to comprehend sarcasm based on the speaker’s intentions and beliefs, the children with ADHD were found to be significantly less accurate when basing their decision on the feelings of the speaker, but also on what the speaker had said. Results are discussed in light of difficulties in their understanding of complex cues of social interactions, and non-literal language being symptomatic of children with a clinical diagnosis of ADHD. The importance of pragmatic language skills in their ability to detect social and emotional information is highlighted.Peer reviewe

Generating expressive speech for storytelling applications

Work on expressive speech synthesis has long focused on the expression of basic emotions. In recent years, however, interest in other expressive styles has been increasing. The research presented in this paper aims at the generation of a storytelling speaking style, which is suitable for storytelling applications and more in general, for applications aimed at children. Based on an analysis of human storytellers' speech, we designed and implemented a set of prosodic rules for converting "neutral" speech, as produced by a text-to-speech system, into storytelling speech. An evaluation of our storytelling speech generation system showed encouraging results

How to improve TTS systems for emotional expressivity

Several experiments have been carried out that revealed weaknesses of the current Text-To-Speech (TTS) systems in their emotional expressivity. Although some TTS systems allow XML-based representations of prosodic and/or phonetic variables, few publications considered, as a pre-processing stage, the use of intelligent text processing to detect affective information that can be used to tailor the parameters needed for emotional expressivity. This paper describes a technique for an automatic prosodic parameterization based on affective clues. This technique recognizes the affective information conveyed in a text and, accordingly to its emotional connotation, assigns appropriate pitch accents and other prosodic parameters by XML-tagging. This pre-processing assists the TTS system to generate synthesized speech that contains emotional clues. The experimental results are encouraging and suggest the possibility of suitable emotional expressivity in speech synthesis

Emotional Prosody Measurement (EPM): A voice-based evaluation method for psychological therapy effectiveness

The voice embodies three sources of information: speech, the identity, and the emotional state of the speaker (i.e., emotional prosody). The latter feature is resembled by the variability of the F0 (also named fundamental frequency of pitch) (SD F0). To extract this feature, Emotional Prosody Measurement (EPM) was developed, which consists of 1) speech recording, 2) removal of speckle noise, 3) a Fourier Transform to extract the F0-signal, and 4) the determination of SD F0. After a pilot study in which six participants mimicked emotions by their voice, the core experiment was conducted to see whether EPM is successful. Twenty-five patients suffering from a panic disorder with agoraphobia participated. Two methods (storytelling and reliving) were used to trigger anxiety and were compared with comparable but more relaxed conditions. This resulted in a unique database of speech samples that was used to compare the EPM with the Subjective Unit of Distress to validate it as measure for anxiety/stress. The experimental manipulation of anxiety proved to be successful and EPM proved to be a successful evaluation method for psychological therapy effectiveness

A neural marker for social bias toward in-group accents

Accents provide information about the speaker's geographical, socio-economic, and ethnic background. Research in applied psychology and sociolinguistics suggests that we generally prefer our own accent to other varieties of our native language and attribute more positive traits to it. Despite the widespread influence of accents on social interactions, educational and work settings the neural underpinnings of this social bias toward our own accent and, what may drive this bias, are unexplored. We measured brain activity while participants from two different geographical backgrounds listened passively to 3 English accent types embedded in an adaptation design. Cerebral activity in several regions, including bilateral amygdalae, revealed a significant interaction between the participants' own accent and the accent they listened to: while repetition of own accents elicited an enhanced neural response, repetition of the other group's accent resulted in reduced responses classically associated with adaptation. Our findings suggest that increased social relevance of, or greater emotional sensitivity to in-group accents, may underlie the own-accent bias. Our results provide a neural marker for the bias associated with accents, and show, for the first time, that the neural response to speech is partly shaped by the geographical background of the listener

Cracking the social code of speech prosody using reverse correlation

Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals

The Perception of Korean Boundary Tones by First and Second Language Speakers

This paper reports an experiment which investigated the perception of prosody in Korean or non-word utterances by native Korean speakers and English learners of Korean. Listeners rated the degrees of positivity and excitement of resynthesized utterances with different pitch ranges and durations. The results revealed no significant differences between the two groups of listeners. The variations in pitch range and duration had systematic effects on the ratings. However, the interactions between various factors suggest that the mapping between prosodic shapes and their paralinguistic meaning is not straightforward