Gaze and Gestures in Telepresence: multimodality, embodiment, and roles of collaboration

This paper proposes a controlled experiment to further investigate the usefulness of gaze awareness and gesture recognition in the support of collaborative work at a distance. We propose to redesign experiments conducted several years ago with more recent technology that would: a) enable to better study of the integration of communication modalities, b) allow users to freely move while collaborating at a distance and c) avoid asymmetries of communication between collaborators.Comment: Position paper, International Workshop New Frontiers in Telepresence 2010, part of CSCW2010, Savannah, GA, USA, 7th of February, 2010. http://research.microsoft.com/en-us/events/nft2010

HERMES: Journal of Language and Communication Studies 41

Multimodality is a recent academic development, fuelling a surge of related research (Kress/van Leeuwen 1996; 2001; Baldry/Thibault 2006; Royce/Bowcher 2007). In parallel to this, the turn of the millennium has seen an increase in the inclusion of typography, graphics and illustration in fiction yet, with only a few exceptions (Gibbons forthcoming a; forthcoming b), printed literature has often been neglected in multimodal study. Focusing on the ‘imagetext novel’ VAS: An Opera in Flatland, written by Steve Tomasula and designed by Stephen Farrell (2002), this paper explores multimodal printed literature through cognitive-poetic analysis. The examination of visual elements is aided by theories from visual perception and multimodal research. This cognitive and perceptual methodology is strengthened through reflection upon recent findings from neuroscientific work on embodiment. In consequence, this paper presents a fresh approach to multimodality, an approach which not only attends to all modes of meaning-making equally, as well as collaboratively, but one which considers the cognitive and embodied aspects of a multimodal literary experience

Drawing as transcription: how do graphical techniques inform interaction analysis?

Drawing as a form of analytical inscription can provide researchers with highly flexible methods for exploring embodied interaction. Graphical techniques can combine spatial layouts, trajectories of action and anatomical detail, as well as rich descriptions of movement and temporal effects. This paper introduces some of the possibilities and challenges of adapting graphical techniques from life drawing and still life for interaction research. We demonstrate how many of these techniques are used in interaction research by illustrating the postural configurations and movements of participants in a ballet class. We then discuss a prototype software tool that is being developed to support interaction analysis specifically in the context of a collaborative data analysis session

Unconventional Political Campaigns in Romania: Presidential Impeachment Referendum (2007)

Every referendum brings forth a crisis of legitimacy. Traian Băsescu experienced this political situation in 2007 when he was suspended by the Romanian Parliament. Having as theoretical background social semiotics (Kress, van Leeuwen [1996] 2006; van Leeuwen, 2005), our paper focuses on four semiotic systems (represented participants, composition, multimodality, interactive participants) applied to the images produced by the Romanian citizens, who had been “called up” to an unconventional political campaign through a personal production of a digital guerilla. Beyond its mere iconic function, number “322” has an indexical function of pinpointing the MPs who voted for Traian Băsescu’s impeachment. This number constitutes the main reason for the embodiment of these Romanian MPs in pictorial metaphors. Metaphors activate the semiotic system of multimodality because they become a sign of creativity by combining layers belonging to different iconic isotopic clusters. The humour comes from the allotopies (Greimas 1966) created through the surprising interweaving of politicians and animals, fairy-tales characters or evil/ good heroes