Rieser H. Speech-gesture Interfaces. An Overview. In:  Proceedings of the 35th Annual Conference of the German Linguistic Society (DGfS). 2013: 282-283

Rieser, Hannes

Publications at Bielefeld University

AG11Haus 6, Raum S24particular) can take over information structural tasks and serve to mark focusdomains (Ebert, Evert & Wilmes 2011). In this workshop we want to followup on the recent development to investigate gestures under formal semanticand pragmatic aspects.Speech-gesture Interfaces. An OverviewHannes Rieser / U. BielefeldMittwoch, 13.3., 14.00–14.30Using theory of grammar tools for description of gesture is not popular amongresearchers trading in gestural matters. There is, however, an obvious link bet-ween gesture research and formal description of language and information:the notion of meaning. Given that we want to get at the meaning and functionof gesture, we need a clear notion of meaning and this is not to be had disre-garding philosophical, logical and semantic work in the meaning realm. Ano-ther link between formal theorizing and gesture research is bound up with theidea that gesture and speech interact: gesture emphasises, specifies or com-plements the semantics of speech, i.e. gesture and speech interface. The in-terface machinery useful in this context is a data structure bound to grammarallowing for the exchange of information, viz. the respective meanings of ge-sture and speech. Along these general lines the talk provides an overview oninterface structures in recent research. We shortly introduce McNeill’s ideas(1992, cf. Röpke 2011) on this topic and proceed treating proposition-boundinterfaces based on Lambda calculus (Rieser 2009/10), Montague Grammar(Giorgolo 2010, Röpke et al. 2013) and HPSG (Lücking 2011, Lascaridesand Alahverdzhieva 2010). There have also been various attempts to embedspeech-gesture interfaces in dynamic theories of discourse and dialogue, mostprominently Lascarides und Stone (2009) using SDRT, and Poesio and Rie-ser’s (2009) approach extending the dialogue model of Poesio and Traum(1997). The following is among the questions of interest touched: Where isthe interface situated, is it, e.g. located within grammar or does it constitutean external domain of its own? What are the interface’s formal properties, i.e.how is it organized and represented? How is it tied up with the rest of thegrammar? Furthermore, which notion of meaning is used to explicate gesturemeaning? What is the effect of gesture meaning and meaning of speech: whatdo they achieve individually and collectively? Finally, a sum-up on the ans-wers to these questions is presented completing the overview.282AG11Haus 6, Raum S24References Alahverdzhieva, K. and A. Lascarides (2010). Analysing Language andCo-verbal Gesture in Constraint-based Grammars. In: Proceedings of 17th Conferenceon HPSG. Paris, pp. 5–25. • Giorgolo, G. (2010). Space and Time in Our Hands.LOT Utrecht. • Lascarides, A. und M. Stone (2009). A Formal Semantic Analysis ofGesture. In: JofS, pp. 1–57. • Lücking, A. (2013). Ikonische Gesten. Grundzüge einerlinguistischen Theorie. Berlin, De Gruyter Mouton. • Rieser, H. (2010). FactoringOut a Gesture Typology from the Bielefeld Speech-Gesture-Alignment Corpus. In:Proceedings of GW 2009. Berlin: Springer, pp. 47–60. • Rieser, H. (2011a). GesturesIndicating Dialogue Structure. Proceedings of SEMdial 2011, Los Angeles, California.• Rieser, H. (2011b). How to Disagree on a Church-Window’s Shape Using Gesture.In Festschrift für Janos Sandor Petöfi. Münster: LIT Verlag, pp. 231-241. • Rieser, H.and Poesio, M. (2009). Interactive gesture in dialogue: a PTT model. In Proceedingsof the SIGDIAL 2009 Conference, pp. 87–96. • Röpke, I. (2011). Watching the growthpoint grow. In: Proceedings of the Second GESPIN Conference 2011.Gestures at the Semantcs-Pragmatics InterfaceGianluca Giorgolo / King’s College, LondonMittwoch, 13.3., 14.30–15.00In this talk I will address the issue of how to best formally model the infor-mational contribution of gesture with respect to our current understanding ofverbal meaning. I will take for granted that gestures are used, both conscious-ly and unconsciously, to convey information and that listeners make use of theinformation conveyed by gestures when decoding the messages of speakers.The open question is how this integrated interpretation is achieved and howwe can model it given the toolkit offered by formal semantics.I will start by overviewing a number of proposals in the literature thatpresent alternative implementations of a unified semantics for verbal langua-ge and gestures. Some of these proposals feature a more semantic approach(e.g. Giorgolo [2010]) while other have a more pragmatic oriented point ofview (e.g. Lascarides and Stone [2009]). I will explore the relevance of thedistinction between lexical and compositional semantics and pragmatics forthe issue of gesture semantics. In particular I will analyse the issues we facewhen modelling the semantic contribution of gestures in light of current deve-lopments in formal semantics that clarify the relationship between semanticsand pragmatics and that point towards a multilayered picture of meaning con-tributions [Stanley, 2000, Potts, 2007]. I will reanalyse the results of Giorgoloand Needham [2012] according to this view and compare how gestures beha-ve with respect to linguistic phenomena, such as presuppositions and different283

Speech-gesture Interfaces. An Overview

http://pub.uni-bielefeld.de/download/2579684/2652123

Speech-gesture Interfaces. An Overview

Abstract

Similar works

Full text

Available Versions

Publications at Bielefeld University