Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications

Lücking A, Bergmann K, Hahn F, Kopp S, Rieser H. Data-based analysis of speech and gesture: the Bielefeld Speech and Gesture Alignment corpus (SaGA) and its applications. Journal on Multimodal User Interfaces. 2013;7(1-2):5-18.Communicating face-to-face, interlocutors frequently produce multimodal meaning packages consisting of speech and accompanying gestures. We discuss a systematically annotated speech and gesture corpus consisting of 25 route-and-landmark-description dialogues, the Bielefeld Speech and Gesture Alignment corpus (SaGA), collected in experimental face-to-face settings. We first describe the primary and secondary data of the corpus and its reliability assessment. Then we go into some of the projects carried out using SaGA demonstrating the wide range of its usability: on the empirical side, there is work on gesture typology, individual and contextual parameters influencing gesture production and gestures’ functions for dialogue structure. Speech-gesture interfaces have been established extending unification-based grammars. In addition, the development of a computational model of speech-gesture alignment and its implementation constitutes a research line we focus on

A semantic account for iconic gestures

Lücking A. A semantic account for iconic gestures. In: "Gesture – Evolution, Brain, and Linguistic Structures". Europa Universität Viadrina; 2010

Ikonische Gesten : Grundzüge einer linguistischen Theorie

Lücking A. Ikonische Gesten : Grundzüge einer linguistischen Theorie. Berlin [u.a.]: De Gruyter; 2012

Towards the score of communication

International audienceThe exchange of verbal and non-verbal com- munication signals in face-to-face dialogue is complexly organised in several ways: each contribution is produced and processed incre- mentally, contributions may be consecutive (e.g. question-answer pairs) or overlapping (e.g. backchannelling), and the contributions themselves may be multimodal. Contributions nonetheless exhibit pairwise utterance coher- ence, and in two respects: across tiers and across discourse co-texts. For these reasons, we propose to distribute dialogue agents across different tiers and to \textquoteleftincrementalize\textquoteright the se- quential notion of turns according to the model of music-inspired communication score