A cross-linguistic study of affective prosody production by monolingual and bilingual children

The main objective of the research reported in the dissertation was to investigate the production of affective speech by monolingual and simultaneous bilingual children in Scottish English and French. The study was designed to address several important issues with respect to affective speech. First, possibility of identifying and compar- ing acoustic correlates of affective speech in productions of monolingual children was explored in a cross-linguistic perspective. Second, affective speech of bilingual chil- dren was examined in their two languages and compared to that of their monolingual peers. Third, vocal emotions encoded by monolingual and bilingual children were tested through the identification by French and Scottish monolingual adults. Five bilingual and twelve monolingual children were recorded for a cross-linguistically comparable corpus of affective speech. Children played four emotions (anger, fear, sadness and happiness) on one token utterance with the help of visual materials, which served as the reference of the expressed emotions, and as an affect inducing material. A large number of child speakers brings better understanding of cross- language and within-language variability in vocal affective expressions. The corpus was acoustically analysed and used in a cross-linguistic perception test with Scottish and French monolingual adults. The results of the perception test support the existing view in the cross-cultural research on emotions: even if people from different cultural groups could identify each others' emotions, an in-group advantage was generally present. Another im- portant finding was that some affective states were more successfully identified in one of the languages by the two groups of listeners. Specifically, French anger, as expressed by bilingual and monolingual children, was identified more successfully by both French and Scottish listeners than anger, encoded by bilinguals and mono- linguals in Scottish English, thus suggesting that children showed some emotions more in one of the languages. The joint analysis of production and perception data confirmed the association of the studied acoustic correlates with affective states, but x also showed the variability of different strategies in their usage. While some speak- ers used all the measured acoustic correlates to a significantly large extent, other speakers used only some of them. Apparently, the usage of all the possible acoustic correlates is not obligatory for successful identification. Moreover, one of the studied affective states (fear) was characterised by more variable usage of acoustic correlates than others. Cross-linguistic differences were attested in the usage of some acoustic correlates and in the preferred strategies for the realisation of affective states. Simultaneous bilingual children could encode affective states in their two lan- guages; moreover, on average, their affective states are identified even better than those of monolingual children. This ability to successfully encode vocal emotions can be interpreted as a signal of high social competence in bilingual children. Produc- tion results show that all bilingual children realise some cross-linguistic differences in their affective speech. Nevertheless, interaction between the languages in the affec- tive speech was discovered both in the production and perception data for bilinguals. This finding comes in support of other studies which call language interaction as a characteristic feature of bilingual phonetic acquisition. The specific pattern of the affective speech realisation is individual for each bilingual child, depending on the affective state and on the used language. In this context, the theory of integrated continuum, developed by Cook (2003), is discussed for its possibility to describe the paralinguistic organisation in the bilingual mind. This thesis thus contributes to a better understanding of phonetic learning by monolingual and bilingual children in the context of affective speech. It also gives a detailed analysis of cross-language and within-language variability present in affec- tive speech. This new data will be of interest to the researchers working in speech sciences, psycholinguistics, developmental and cross-cultural psychology.sub_shsunpub80_ethesesunpu

Beyond happiness: Building a science of discrete positive emotions.

While trait positive emotionality and state positive-valence affect have long been the subject of intense study, the importance of differentiating among several "discrete" positive emotions has only recently begun to receive serious attention. In this article, we synthesize existing literature on positive emotion differentiation, proposing that the positive emotions are best described as branches of a "family tree" emerging from a common ancestor mediating adaptive management of fitness-critical resources (e.g., food). Examples are presented of research indicating the importance of differentiating several positive emotion constructs. We then offer a new theoretical framework, built upon a foundation of phylogenetic, neuroscience, and behavioral evidence, that accounts for core features as well as mechanisms for differentiation. We propose several directions for future research suggested by this framework and develop implications for the application of positive emotion research to translational issues in clinical psychology and the science of behavior change. (PsycINFO Database Recor

How do you say ‘hello’? Personality impressions from brief novel voices

On hearing a novel voice, listeners readily form personality impressions of that speaker. Accurate or not, these impressions are known to affect subsequent interactions; yet the underlying psychological and acoustical bases remain poorly understood. Furthermore, hitherto studies have focussed on extended speech as opposed to analysing the instantaneous impressions we obtain from first experience. In this paper, through a mass online rating experiment, 320 participants rated 64 sub-second vocal utterances of the word ‘hello’ on one of 10 personality traits. We show that: (1) personality judgements of brief utterances from unfamiliar speakers are consistent across listeners; (2) a two-dimensional ‘social voice space’ with axes mapping Valence (Trust, Likeability) and Dominance, each driven by differing combinations of vocal acoustics, adequately summarises ratings in both male and female voices; and (3) a positive combination of Valence and Dominance results in increased perceived male vocal Attractiveness, whereas perceived female vocal Attractiveness is largely controlled by increasing Valence. Results are discussed in relation to the rapid evaluation of personality and, in turn, the intent of others, as being driven by survival mechanisms via approach or avoidance behaviours. These findings provide empirical bases for predicting personality impressions from acoustical analyses of short utterances and for generating desired personality impressions in artificial voices

Spoken affect classification : algorithms and experimental implementation : a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Computer Science at Massey University, Palmerston North, New Zealand

Machine-based emotional intelligence is a requirement for natural interaction between humans and computer interfaces and a basic level of accurate emotion perception is needed for computer systems to respond adequately to human emotion. Humans convey emotional information both intentionally and unintentionally via speech patterns. These vocal patterns are perceived and understood by listeners during conversation. This research aims to improve the automatic perception of vocal emotion in two ways. First, we compare two emotional speech data sources: natural, spontaneous emotional speech and acted or portrayed emotional speech. This comparison demonstrates the advantages and disadvantages of both acquisition methods and how these methods affect the end application of vocal emotion recognition. Second, we look at two classification methods which have gone unexplored in this field: stacked generalisation and unweighted vote. We show how these techniques can yield an improvement over traditional classification methods

Spring School on Language, Music, and Cognition: Organizing Events in Time

The interdisciplinary spring school “Language, music, and cognition: Organizing events in time” was held from February 26 to March 2, 2018 at the Institute of Musicology of the University of Cologne. Language, speech, and music as events in time were explored from different perspectives including evolutionary biology, social cognition, developmental psychology, cognitive neuroscience of speech, language, and communication, as well as computational and biological approaches to language and music. There were 10 lectures, 4 workshops, and 1 student poster session. Overall, the spring school investigated language and music as neurocognitive systems and focused on a mechanistic approach exploring the neural substrates underlying musical, linguistic, social, and emotional processes and behaviors. In particular, researchers approached questions concerning cognitive processes, computational procedures, and neural mechanisms underlying the temporal organization of language and music, mainly from two perspectives: one was concerned with syntax or structural representations of language and music as neurocognitive systems (i.e., an intrapersonal perspective), while the other emphasized social interaction and emotions in their communicative function (i.e., an interpersonal perspective). The spring school not only acted as a platform for knowledge transfer and exchange but also generated a number of important research questions as challenges for future investigations

Speech-based recognition of self-reported and observed emotion in a dimensional space

The differences between self-reported and observed emotion have only marginally been investigated in the context of speech-based automatic emotion recognition. We address this issue by comparing self-reported emotion ratings to observed emotion ratings and look at how differences between these two types of ratings affect the development and performance of automatic emotion recognizers developed with these ratings. A dimensional approach to emotion modeling is adopted: the ratings are based on continuous arousal and valence scales. We describe the TNO-Gaming Corpus that contains spontaneous vocal and facial expressions elicited via a multiplayer videogame and that includes emotion annotations obtained via self-report and observation by outside observers. Comparisons show that there are discrepancies between self-reported and observed emotion ratings which are also reflected in the performance of the emotion recognizers developed. Using Support Vector Regression in combination with acoustic and textual features, recognizers of arousal and valence are developed that can predict points in a 2-dimensional arousal-valence space. The results of these recognizers show that the self-reported emotion is much harder to recognize than the observed emotion, and that averaging ratings from multiple observers improves performance

Are affective speakers effective speakers? – Exploring the link between the vocal expression of positive emotions and communicative effectiveness

This thesis explores the effect of vocal affect expression on communicative effectiveness. Two studies examined whether positive speaker affect facilitates the encoding and decoding of the message, combining methods from Phonetics and Psychology.This research has been funded through a Faculty Studentship by the University of Stirling and a Fellowship by the German Academic Exchange Service (DAAD)

A neural marker for social bias toward in-group accents

Accents provide information about the speaker's geographical, socio-economic, and ethnic background. Research in applied psychology and sociolinguistics suggests that we generally prefer our own accent to other varieties of our native language and attribute more positive traits to it. Despite the widespread influence of accents on social interactions, educational and work settings the neural underpinnings of this social bias toward our own accent and, what may drive this bias, are unexplored. We measured brain activity while participants from two different geographical backgrounds listened passively to 3 English accent types embedded in an adaptation design. Cerebral activity in several regions, including bilateral amygdalae, revealed a significant interaction between the participants' own accent and the accent they listened to: while repetition of own accents elicited an enhanced neural response, repetition of the other group's accent resulted in reduced responses classically associated with adaptation. Our findings suggest that increased social relevance of, or greater emotional sensitivity to in-group accents, may underlie the own-accent bias. Our results provide a neural marker for the bias associated with accents, and show, for the first time, that the neural response to speech is partly shaped by the geographical background of the listener

Predicting continuous conflict perception with Bayesian Gaussian processes

Conflict is one of the most important phenomena of social life, but it is still largely neglected by the computing community. This work proposes an approach that detects common conversational social signals (loudness, overlapping speech, etc.) and predicts the conflict level perceived by human observers in continuous, non-categorical terms. The proposed regression approach is fully Bayesian and it adopts Automatic Relevance Determination to identify the social signals that influence most the outcome of the prediction. The experiments are performed over the SSPNet Conflict Corpus, a publicly available collection of 1430 clips extracted from televised political debates (roughly 12 hours of material for 138 subjects in total). The results show that it is possible to achieve a correlation close to 0.8 between actual and predicted conflict perception