On the acoustics of overlapping laughter in conversational speech

The social nature of laughter invites people to laugh together. This joint vocal action often results in overlapping laughter. In this paper, we show that the acoustics of overlapping laughs are different from non-overlapping laughs. We found that overlapping laughs are stronger prosodically marked than non-overlapping ones, in terms of higher values for duration, mean F0, mean and maximum intensity, and the amount of voicing. This effect is intensified by the number of people joining in the laughter event, which suggests that entrainment is at work. We also found that group size affects the number of overlapping laughs which illustrates the contagious nature of laughter. Finally, people appear to join laughter simultaneously at a delay of approximately 500 ms; a delay that must be considered when developing spoken dialogue systems that are able to respond to users’ laughs

Investigating Fine Temporal Dynamics of Prosodic and Lexical Accommodation

Conversational interaction is a dynamic activity in which participants engage in the construction of meaning and in establishing and maintaining social relationships. Lexical and prosodic accommodation have been observed in many studies as contributing importantly to these dimensions of social interaction. However, while previous works have considered accommodation mechanisms at global levels (for whole conversations, halves and thirds of conversations), this work investigates their evolution through repeated analysis at time intervals of increasing granularity to analyze the dynamics of alignment in a spoken language corpus. Results show that the levels of both prosodic and lexical accommodation fluctuate several times over the course of a conversation

Timescales of Massive Human Entrainment

The past two decades have seen an upsurge of interest in the collective behaviors of complex systems composed of many agents entrained to each other and to external events. In this paper, we extend concepts of entrainment to the dynamics of human collective attention. We conducted a detailed investigation of the unfolding of human entrainment - as expressed by the content and patterns of hundreds of thousands of messages on Twitter - during the 2012 US presidential debates. By time locking these data sources, we quantify the impact of the unfolding debate on human attention. We show that collective social behavior covaries second-by-second to the interactional dynamics of the debates: A candidate speaking induces rapid increases in mentions of his name on social media and decreases in mentions of the other candidate. Moreover, interruptions by an interlocutor increase the attention received. We also highlight a distinct time scale for the impact of salient moments in the debate: Mentions in social media start within 5-10 seconds after the moment; peak at approximately one minute; and slowly decay in a consistent fashion across well-known events during the debates. Finally, we show that public attention after an initial burst slowly decays through the course of the debates. Thus we demonstrate that large-scale human entrainment may hold across a number of distinct scales, in an exquisitely time-locked fashion. The methods and results pave the way for careful study of the dynamics and mechanisms of large-scale human entrainment.Comment: 20 pages, 7 figures, 6 tables, 4 supplementary figures. 2nd version revised according to peer reviewers' comments: more detailed explanation of the methods, and grounding of the hypothese

Joint Modeling of Content and Discourse Relations in Dialogues

We present a joint modeling approach to identify salient discussion points in spoken meetings as well as to label the discourse relations between speaker turns. A variation of our model is also discussed when discourse relations are treated as latent variables. Experimental results on two popular meeting corpora show that our joint model can outperform state-of-the-art approaches for both phrase-based content selection and discourse relation prediction tasks. We also evaluate our model on predicting the consistency among team members' understanding of their group decisions. Classifiers trained with features constructed from our model achieve significant better predictive performance than the state-of-the-art.Comment: Accepted by ACL 2017. 11 page

Infants segment words from songs - an EEG study

Children’s songs are omnipresent and highly attractive stimuli in infants’ input. Previous work suggests that infants process linguistic–phonetic information from simplified sung melodies. The present study investigated whether infants learn words from ecologically valid children’s songs. Testing 40 Dutch-learning 10-month-olds in a familiarization-then-test electroencephalography (EEG) paradigm, this study asked whether infants can segment repeated target words embedded in songs during familiarization and subsequently recognize those words in continuous speech in the test phase. To replicate previous speech work and compare segmentation across modalities, infants participated in both song and speech sessions. Results showed a positive event-related potential (ERP) familiarity effect to the final compared to the first target occurrences during both song and speech familiarization. No evidence was found for word recognition in the test phase following either song or speech. Comparisons across the stimuli of the present and a comparable previous study suggested that acoustic prominence and speech rate may have contributed to the polarity of the ERP familiarity effect and its absence in the test phase. Overall, the present study provides evidence that 10-month-old infants can segment words embedded in songs, and it raises questions about the acoustic and other factors that enable or hinder infant word segmentation from songs and speech

Conversational Alignment: A Study of Neural Coherence and Speech Entrainment

Conversational alignment refers to the tendency for communication partners to adjust their verbal and non-verbal behaviors to become more like one another during the course of human interaction. This alignment phenomenon has been observed in neural patterns, specifically in the prefrontal areas of the brain (Holper et al., 2013; Cui et al., 2012; Dommer et al., 2012; Holper et al., 2012; Funane et al., 2011; Jiang et al., 2012); verbal behaviors such acoustic speech features (e.g., Borrie & Liss, 2014; Borrie et al., 2015; Lubold & Pon-Barry, 2014), phonological features (e.g., Babel, 2012; Pardo, 2006), lexical selection (e.g., Brennan & Clark, 1996; Garrod & Anderson, 1989), syntactic structure (e.g., Branigan, Pickering, & Cleland, 2000; Reitter, Moore, & Keller, 2006); and motor behaviors including body posture, facial expressions and breathing rate (e.g., Furuyama, Hayashi, & Mishima, 2005; Louwerse, Dale, Bard, & Jeuniaux, 2012; Richardson, March, & Schmit, 2005; Shockley, Santana, & Fowler, 2003; McFarland, 2001). While conversational alignment in itself, is a largely physical phenomenon, it has been linked to significant functional value, both in the cognitive and social domains. Cognitively, conversational alignment facilitates spoken message comprehension, enabling listeners to share mental models (Garrod & Pickering, 2004) and generate temporal predictions about upcoming aspects of speech. From a social perspective, behavioral alignment has been linked with establishing turn-taking behaviors, and with increased feelings of rapport, empathy, and intimacy between conversational pairs (e.g., Lee et al. 2010; Nind, & Macrae, 2009; Smith, 2008; Bailenson & Yee, 2005; Chartrand & Barg, 1999; Miles, Putman & Street, 1984; Street & Giles, 1982). Benus (2014), for example, observed that individuals who align their speech features are perceived as more socially attractive and likeable, and have interactions that are more successful. These cognitive and social benefits, associated with conversational alignment, have been observed in both linguistic and neural data (e.g., Holper et al., 2012; 2013, Cui et al. 2012; Jiang et al., 2012; Egetemeir et al., 2011; Stephens et al. 2010). The purpose of the current study was to examine conversational alignment as a multi-level communication phenomenon, by examining the relationship between neural and speech behaviors. To assess neural alignment, we used Near-Infrared Spectroscopy (NIRS), a non-invasive neuroimaging technology that detects cortical increases and decreases in the concentration of oxygenated and deoxygenated hemoglobin at multiple measurement sites to determine the rate that oxygen is being released and absorbed (Ferrari & Quaresima, 2012). While still considered a relatively new neural imaging technique, NIRS has been well established as an efficacious and effective data collection approach, particularly appropriate for social interaction research (e.g., Holper et al., 2013; Jiang et al., 2012; Holper et al., 2012; Suda et al., 2010). We utilized hyperscanning, a technique that allows for the quantitation of two simultaneous signals, allowing us to document neural alignment between two individuals (Babiloni & Astolfi, 2012). Recent studies have revealed neural alignment between two persons in cooperative states, including alignment in the right superior frontal cortices and medial prefrontal regions (Cui et al., 2012; Dommer et al., 2012; Funane et al., 2011). This increased prefrontal interbrain alignment has also been observed in other social interactions, including joint attention tasks (Dommer et al., 2012), imitation tasks (Holper et al., 2012), competitive games (Cheng et al., 2015, Duan et al., 2013), teaching-learning interactions (Holper et al., 2013), face- to-face communication (Jiang et al., 2012), mother-child interactions (Hirata et al., 2014), and during cooperative singing tasks (Osaka et al., 2015). Interestingly, Jiang et al. (2012) showed that increased neural alignment only occurred between conversational participants when they were speaking face-to-face, but not when participants had their backs facing one another. The authors speculated that the multi-sensory information, for example motor behaviors such as gestures, was required for neural alignment to occur