9 research outputs found
Proceedings of Gesture and Speech in Interaction (GESPIN)
Wagner P, Bryhadyr N. Mutual Visibility and Information Structure Enhance Synchrony between Speech and Co-Speech Movements. Journal of Multimodal Communication Studies. 2017;4(1-2):69-74
Does information-structural acoustic prosody change under different visibility conditions?
Wagner P, Bryhadyr N, Schröer M, Ludusan B. Does information-structural acoustic prosody change under different visibility conditions? In: Proceedings of ICPhS. 2019.It is well-known that the effort invested in prosodic expression can be adjusted to the information structure in a message, but also to the characteristics of the transmission channel. To investigate wether visibly accessible cues to information structure or facial prosodic expression have a differentiated impact on acoustic prosody, we modified the visibility conditions in a spontaneous dyadic interaction task, i.e. a verbalized version of TicTacToe. The main hypothesis was that visibly accessible cues should lead to a decrease in prosodic effort. While we found that - as expected - information structure is expressed throughout a number of acoustic-prosodic cues, visible accessibility to context information makes accents shorter, while accessability to an interlocutor's facial expression slightly increases the mean F0 of an accent
Pitch Accent Trajectories across Different Conditions of Visibility and Information Structure - Evidence from Spontaneous Dyadic Interaction
Wagner P, Bryhadyr N, Schröer M. Pitch Accent Trajectories across Different Conditions of Visibility and Information Structure - Evidence from Spontaneous Dyadic Interaction. In: Proceedings of Interspeech. 2019.Previous research identified a differential contribution of information
structure and the visibility of facial and contextual information
to the acoustic-prosodic expression of pitch accents.
However, it is unclear whether pitch accent shapes are affected
by these conditions as well. To investigate whether varying
context cues have a differentiated impact on pitch accent trajectories
produced in conversational interaction, we modified
the visibility conditions in a spontaneous dyadic interaction
task, i.e. a verbalized version of TicTacToe. Besides varying
visibility, the game task allows for measuring the impact of
information-structure on pitch accent trajectories, differentiating
important and unpredictable game moves. Using GAMMs
on four speaker groups (identified by a cluster analysis), we
could isolate varying strategies of prosodic adaptation to contextual
change. While few speaker groups showed a reaction to
the availability of visible context cues (facial prosody or executed
game moves), all groups differentiated the verbalization
of unpredictable and predictable game moves with a groupspecific
trajectory adaptation. The importance of game moves
resulted in differentiated adaptations in two out of four speaker
groups. The detected strategic trajectory adaptations were characterized
by different characteristics of boundary tones, adaptations
of the global f0-level, or the shape of the corresponding
pitch accent
image70
https://dc.cod.edu/ilccoer2018images/1066/thumbnail.jp
Degrees of agreement using Indian head nods and bobble gestures
Jabeen F, Bryhadyr N, Wagner P. Degrees of agreement using Indian head nods and bobble gestures. Presented at the International Society of Gesture Studies Conference, Chicago, IL, USA.Head bobble is stereotypically associated with the people from the Indian Subcontinent. The
interpretation of this gesture as conveying degrees of agreement has led to amusing anecdotes of
cultural misunderstanding. However, there is no existing analysis of this gesture and how it differs
from nods used to indicate agreement. Moreover, it is not clear if bobbles and nods are used only by
listeners or speakers. This research investigates the use of bobbles and nods by Hindi speakers and
listeners from India to indicate explicit agreement marked by “yes” or “Ok” and implicit agreement.
We analyzed YouTube videos (90 minutes) of street interviews recorded in Delhi, India. The videos
were annotated to mark the role of the interlocutors (Speakers: turn holders; Listeners: provided
feedback with backchannel and/or stayed silent when another person talked) and the
communicative contexts for each target gesture. Overall, speakers used nods more often (66%) then
listeners, whereas bobbles were used both by listeners and speakers with equal frequency. Both
listeners and speakers used nods to mark explicit agreement more frequently (Listeners: 63%,
Speakers: 86%) than implicit agreement. As for bobbles, listeners used them slightly more frequently
to convey implicit (54%) as compared with explicit agreement. Contrarily, speakers used bobbles
overwhelmingly to mark explicit agreement (92%). This leads us to claim that (1) Hindi-speaking
Indian listeners and speakers prefer to use nods to convey explicit agreement and (2) bobbles are
used for implicit agreement by speakers but implicit agreement by listeners. This shows that there is
a functional split in the use of nods and bobbles that depends on the role of interlocutors. Moreover,
there is no one-to-one relation between target gestures, interlocutors’ roles, and the communicative
contexts. This supports the claim that bobbles are not as clearly associated with strong agreement
as nods are
Yes or no: An analysis of Indian head gestures
Jabeen F, Bryhadyr N, Wagner P. Yes or no: An analysis of Indian head gestures. Presented at the International Society of Gesture Studies Conference, Chicago, IL, USA.Head bobble is stereotypically associated with people from the Indian Subcontinent. The apparent
substitution of bobbles with nods and shakes is anecdotally reported to be problematic for non-
Indian interlocutors. However, there is no existing analysis of bobbles and how they differ from nods
and shakes generally believed to indicate agreement and disagreement respectively. Moreover, it is
not clear if these gestures are used exclusively by listeners or speakers. This study investigates the
use of nods, shakes, and bobbles by Hindi speakers and listeners from India and reports the
communicative contexts used for these gestures. We analyzed YouTube videos (90 minutes) of
street interviews recorded in Delhi, India. The videos were annotated to mark interlocutors’ roles
(Speakers: turn holders; Listeners: provided feedback with backchannel and/or stayed silent when
another person talked) and communicative contexts for each gesture (N = 755). The analysis
showed that speakers used bobbles (63%) and shakes (64%) more frequently than listeners, whereas
nods were used by speakers and listeners with similar frequency (49%, 51% respectively). As for the
contexts, listeners overwhelmingly used nods and bobbles for agreement/backchanneling (Nods:
97%, Bobbles: 91%). Furthermore, listeners used shakes for agreement/backchanneling (71%) as well
as in the context of negation (14.5%), and while being presented with new information (14.5%).
However, speakers predominantly used shakes to convey negation (63%) as well as when providing
information (13.5%) or elaborating on it (13.5%). Nods and bobbles were not preferred in a particular
context and speakers used them for agreement/backchanneling (Nods: 38%, Bobbles: 38.5%),
elaboration (Nods: 28%, 23%), providing information (Nods: 29%, Bobbles: 36%), and negation (Nods:
5%, Bobbles: 2.5%). This data indicates that shakes are clearly associated with negation as compared
with nods and bobbles. The interchangeable use of nods and bobbles explains the confusion faced
by non-Indians when communicating with the Indian interlocutors
A crosslinguistic study on the interplay of fillers and silences
Betz S, Bryhadyr N, Kosmala L, Schettino L. A crosslinguistic study on the interplay of fillers and silences. In: Proceedings of DiSS 2021. 2021.We present a crosslinguistic study on the interplay of hesitation silences and fillers in conversation. The research questions have been addressed for English in a previous DiSS workshop paper (Betz & Kosmala, 2019) and this study extends the analysis to German, Italian and French. The research questions are: 1) Does the type of the filler influence following silence duration 2) Does the duration of the filler correlate with silence duration 3) Does silence duration vary depending on its distance from filler. The analysis shows cross-linguistic similarities and differences, thus highlighting the role and the language- and culture-specific nature of disfluencies
Cognitive Load Increases Spoken and Gestural Hesitation Frequency
Betz S, Bryhadyr N, TĂĽrk O, Wagner P. Cognitive Load Increases Spoken and Gestural Hesitation Frequency. Languages. 2023;8(1): 71.This study investigates the interplay of spoken and gestural hesitations under varying amounts of cognitive load. We argue that not only fillers and silences, as the most common hesitations, are directly related to speech pausing behavior, but that hesitation lengthening is as well. We designed a resource-management card game as a method to elicit ecologically valid pausing behavior while being able to finely control cognitive load via card complexity. The method very successfully elicits large amounts of hesitations. Hesitation frequency increases as a function of cognitive load. This is true for both spoken and gestural hesitations. We conclude that the method presented here is a versatile tool for future research and we present foundational research on the speech-gesture link related to hesitations induced by controllable cognitive load
A Crosslinguistic Study on the Interplay of Fillers and Silences
We present a crosslinguistic study on the interplay of hesitation silences and fillers in conversation. The research questions have been addressed for English in a previous DiSS workshop paper (Betz & Kosmala, 2019) and this study extends the analysis to German, Italian and French. The research questions are: 1) Does the type of the filler influence following silence duration 2) Does the duration of the filler correlate with silence duration 3) Does silence duration vary depending on its distance from filler. The analysis shows cross-linguistic similarities and differences, thus highlighting the role and the language- and culture-specific nature of disfluencies