Framing Movements for Gesture Interface Design

Gesture interfaces are an attractive avenue for human-computer interaction, given the range of expression that people are able to engage when gesturing. Consequently, there is a long running stream of research into gesture as a means of interaction in the field of human-computer interaction. However, most of this research has focussed on the technical challenges of detecting and responding to people’s movements, or on exploring the interaction possibilities opened up by technical developments. There has been relatively little research on how to actually design gesture interfaces, or on the kinds of understandings of gesture that might be most useful to gesture interface designers. Running parallel to research in gesture interfaces, there is a body of research into human gesture, which would seem a useful source to draw knowledge that could inform gesture interface design. However, there is a gap between the ways that ‘gesture’ is conceived of in gesture interface research compared to gesture research. In this dissertation, I explore this gap and reflect on the appropriateness of existing research into human gesturing for the needs of gesture interface design. Through a participatory design process, I designed, prototyped and evaluated a gesture interface for the work of the dental examination. Against this grounding experience, I undertook an analysis of the work of the dental examination with particular focus on the roles that gestures play in the work to compare and discuss existing gesture research. I take the work of the gesture researcher McNeill as a point of focus, because he is widely cited within gesture interface research literature. I show that although McNeill’s research into human gesture can be applied to some important aspects of the gestures of dentistry, there remain range of gestures that McNeill’s work does not deal with directly, yet which play an important role in the work and could usefully be responded to with gesture interface technologies. I discuss some other strands of gesture research, which are less widely cited within gesture interface research, but offer a broader conception of gesture that would be useful for gesture interface design. Ultimately, I argue that the gap in conceptions of gesture between gesture interface research and gesture research is an outcome of the different interests that each community brings to bear on the research. What gesture interface research requires is attention to the problems of designing gesture interfaces for authentic context of use and assessment of existing theory in light of this

Head-Driven Phrase Structure Grammar

Head-Driven Phrase Structure Grammar (HPSG) is a constraint-based or declarative approach to linguistic knowledge, which analyses all descriptive levels (phonology, morphology, syntax, semantics, pragmatics) with feature value pairs, structure sharing, and relational constraints. In syntax it assumes that expressions have a single relatively simple constituent structure. This volume provides a state-of-the-art introduction to the framework. Various chapters discuss basic assumptions and formal foundations, describe the evolution of the framework, and go into the details of the main syntactic phenomena. Further chapters are devoted to non-syntactic levels of description. The book also considers related fields and research areas (gesture, sign languages, computational linguistics) and includes chapters comparing HPSG with other frameworks (Lexical Functional Grammar, Categorial Grammar, Construction Grammar, Dependency Grammar, and Minimalism)

Gestures and cooperation: considering non verbal communication in the design of interactive spaces

This dissertation explores the role of gestures in computer supported collaboration. People make extensive use of non-verbal forms of communication when they interact with each other in everyday life: of these, gestures are relatively easy to observe and quantify. However, the role of gestures in human computer interaction so far has been focused mainly on using conventional signs like visible commands, rather than on exploiting all nuances of such natural human skill. We propose a perspective on natural interaction that builds on recent advances in tangible interaction, embodiment and computer supported collaborative work. We consider the social and cognitive aspects of gestures and manipulations to support our claim of a primacy of tangible and multi-touch interfaces, and describe our experiences focused on assessing the suitability of such interface paradigms to traditional application scenarios. We describe our design and prototype of an interactive space for group-work, in which natural interfaces, such as tangible user interfaces and multi-touch screens, are deployed so as to foster and encourage collaboration. We show that these interfaces can lead to an improvement in performances and that such improvements appear related to an increase of the gestures performed by the users. We also describe the progress on the state of the art that have been necessary to implement such tools on commodity hardware and deploy them in a relatively uncontrolled environment. Finally, we discuss our findings and frame them in the broader context of embodied interaction, drawing useful implications for interactions design, with emphasis on how to enhance the activity of people in their workplace, home, school, etc. supported in their individual and collaborative tasks by natural interfaces

A gesture-based approach to teaching english as a second language.

An artificial hand-gesture code was designed by the author to present and elicit English as an L2 in the English language classroom. The gesture code was combined with a methodological approach bearing resemblances to input-based instruction and Focus on Form. The combination of the gesture code together with the methodology when applied in the classroom was named GestureWay (GW). The rationale behind this teaching focus was to provide students with an alternative input reference to text and allow learners accelerated exposure and practice of holistic language in its oral form thereby assisting in the development of communicative spoken L2 competency. An experiment was set up over an academic year comparing a group of nineteen nine-year-olds at a Seville primary school (Spain) who received a course of GW and a control group who continued with the usual programme of English instruction. Pre-tests and post-tests were carried out on both groups. Results showed that when communicative acquisition levels were assessed, the experimental group scored significantly higher marks than the control group especially in oral story-telling activities. Comparisons between non-communicative aspects of language knowledge such as controlled discrete item tests resulted in more evenly spread scores between the two groups or with a small advantage to the experimental group. Besides the quantitative measurements taken from both groups, a qualitative appraisal was made of the consequences of implementing GestureWay in the classroom with learners unused to this style of teaching. The overall observations were favourable in that learners quickly accepted and successfully continued with the dynamic to the end of the course and responded well despite some teething problems of difficulties in adapting to classroom environments of highly communicative and cognitively intensive classroom environments

Affordable avatar control system for personal robots

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2009.Includes bibliographical references (p. 76-79).Social robots (personal robots) emphasize individualized social interaction and communication with people. To maximize communication capacity of a personal robot, designers make it more anthropomorphic (or zoomorphic), and people tend to interact more naturally with such robots. However, adapting anthropomorphism (or zoomorphism) in social robots makes morphology of a robot more complex; thus, it becomes harder to control robots with existing interfaces. The Huggable is a robotic Teddy bear platform developed by the Personal Robots Group at the MIT Media Lab. It has its specific purpose in healthcare, elderly care, education, and family communication. It is important that a user can successfully convey the meaningful context in a dialogue via the robot's puppeteering interface. I investigate relevant technologies to develop a robotic puppetry system for a zoomorphic personal robot and develop three different puppeteering interfaces to control the robot: the website interface, wearable interface, and sympathetic interface. The wearable interface was examined through a performance test and the web interface was examined through a user study.by Jun Ki Lee.S.M

Head-Driven Phrase Structure Grammar

Head-Driven Phrase Structure Grammar (HPSG) is a constraint-based or declarative approach to linguistic knowledge, which analyses all descriptive levels (phonology, morphology, syntax, semantics, pragmatics) with feature value pairs, structure sharing, and relational constraints. In syntax it assumes that expressions have a single relatively simple constituent structure. This volume provides a state-of-the-art introduction to the framework. Various chapters discuss basic assumptions and formal foundations, describe the evolution of the framework, and go into the details of the main syntactic phenomena. Further chapters are devoted to non-syntactic levels of description. The book also considers related fields and research areas (gesture, sign languages, computational linguistics) and includes chapters comparing HPSG with other frameworks (Lexical Functional Grammar, Categorial Grammar, Construction Grammar, Dependency Grammar, and Minimalism)

Shifting Interfaces: art research at the intersections of live performance and technology

Merged with duplicate record 10026.1/809 on 08.20.2017 by CS (TIS)This collection of published works is an outcome of my practice-led inter-disciplinary collaborative artistic research into deepening understanding of creative process in the field of contemporary dance. It comprises thirty written works published from 1999 to 2007 in various formats and platforms. This collection is framed by a methodological discussion that provides insight into how this research has intersected over time with diverse fields of practice including contemporary dance, digital and new media arts and non-art domains such as cognitive and social science. Fields are understood in the context of this research to be largely constituted out of the expert practices of individual collaborators. This research starts from an interest in the Impact of new media technologies on dance making/ choreography. The collection of works show evidence, established in the first two publications, of an evolving engagement with two concepts related to this interest: (1) the 'algorithm' as a process-level connection or bridge between dance composition and computation; (2) the empirical study of movement embedded as a 'knowledge base' in the practices of both computer animation and dance and thus forming a special correspondence between them. This collection provides evidence of this research through a period of community-building amongst artists using new media technologies in performance, and culminates in the identification of an emerging 'community of practice' coming together around the formation of a unique body of knowledge pertaining to dance. The late 1990s New Media Art movement provided a supportive context for Important peer-to-peer encounters with creators and users of software tools and platforms in the context of inter-disciplinary art-making. A growing interest in software programming as a creative practice opened up fresh perspectives on possible connections with dance making. It became clear that software's utility alone, including artistic uses of software, was a limited conception. This was the background thinking that informed the first major shift in the research towards the design of software that might augment the creative process of expert choreographers and dancers. This shift from software use to its design, framed by a focus on the development of tools to support dance creation, also provided strong rationale to deepen the research into dance making processes. In the second major phase of the research presented here, scientific study is brought collaboratively to bear on questions related to choreographic practice. This lead to a better understanding of ways in which dancers and choreographers, as 'thinking bodies', interact with their design tools and each other in the context of creation work. In addition to this collection, outcomes of this research are traceable to other published papers and art works it has given rise to. Less easily measureable, but just as valuable, are the sustained relations between individuals and groups behind the 'community of practice' now recognised for its development of unique formats for bringing choreographic ideas and processes into contact, now and in the future, with both general audiences and other specialist practices