Understanding 3D mid-air hand gestures with interactive surfaces and displays: a systematic literature review

3D gesture based systems are becoming ubiquitous and there are many mid-air hand gestures that exist for interacting with digital surfaces and displays. There is no well defined gesture set for 3D mid-air hand gestures which makes it difficult to develop applications that have consistent gestures. To understand what gestures exist we conducted the first comprehensive systematic literature review on mid-air hand gestures following existing research methods. The results of the review identified 65 papers where the mid-air hand gestures supported tasks for selection, navigation, and manipulation. We also classified the gestures according to a gesture classification scheme and identified how these gestures have been empirically evaluated. The results of the review provide a richer understanding of what mid-air hand gestures have been designed, implemented, and evaluated in the literature which can help developers design better user experiences for digital interactive surfaces and displays

Understanding 3D mid-air hand gestures with interactive surfaces and displays: a systematic literature review

3D gesture based systems are becoming ubiquitous and there are many mid-air hand gestures that exist for interacting with digital surfaces and displays. There is no well defined gesture set for 3D mid-air hand gestures which makes it difficult to develop applications that have consistent gestures. To understand what gestures exist we conducted the first comprehensive systematic literature review on mid-air hand gestures following existing research methods. The results of the review identified 65 papers where the mid-air hand gestures supported tasks for selection, navigation, and manipulation. We also classified the gestures according to a gesture classification scheme and identified how these gestures have been empirically evaluated. The results of the review provide a richer understanding of what mid-air hand gestures have been designed, implemented, and evaluated in the literature which can help developers design better user experiences for digital interactive surfaces and displays

Gesture elicitation study on how to opt-in & Opt-out from interactions with public displays

Public interactive displays with gesture-recognizing cameras enable new forms of interactions. However, often such systems do not yet allow passers-by a choice to engage voluntarily or disengage from an interaction. To address this issue, this paper explores how people could use different kinds of gestures or voice commands to explicitly opt-in or opt-out of interactions with public installations. We report the results of a gesture elicitation study with 16 participants, generating gestures within five gesture-types for both a commercial and entertainment scenario. We present a categorization and themes of the 430 proposed gestures, and agreement scores showing higher consensus for torso gestures and for opting-out with face/head. Furthermore, patterns indicate that participants often chose non-verbal representations of opposing pairs such as ‘close and open’ when proposing gestures. Quantitative results showed overall preference for hand and arm gestures, and generally a higher acceptance for gestural interaction in the entertainment setting

Ubiquitous computing and natural interfaces for environmental information

Dissertação apresentada na Faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para obtenção do Grau de Mestre em Engenharia do Ambiente, perfil Gestão e Sistemas AmbientaisThe next computing revolution‘s objective is to embed every street, building, room and object with computational power. Ubiquitous computing (ubicomp) will allow every object to receive and transmit information, sense its surroundings and act accordingly, be located from anywhere in the world, connect every person. Everyone will have the possibility to access information, despite their age, computer knowledge, literacy or physical impairment. It will impact the world in a profound way, empowering mankind, improving the environment, but will also create new challenges that our society, economy, health and global environment will have to overcome. Negative impacts have to be identified and dealt with in advance. Despite these concerns, environmental studies have been mostly absent from discussions on the new paradigm. This thesis seeks to examine ubiquitous computing, its technological emergence, raise awareness towards future impacts and explore the design of new interfaces and rich interaction modes. Environmental information is approached as an area which may greatly benefit from ubicomp as a way to gather, treat and disseminate it, simultaneously complying with the Aarhus convention. In an educational context, new media are poised to revolutionize the way we perceive, learn and interact with environmental information. cUbiq is presented as a natural interface to access that information

Barehand Mode Switching in Touch and Mid-Air Interfaces

Raskin defines a mode as a distinct setting within an interface where the same user input will produce results different to those it would produce in other settings. Most interfaces have multiple modes in which input is mapped to different actions, and, mode-switching is simply the transition from one mode to another. In touch interfaces, the current mode can change how a single touch is interpreted: for example, it could draw a line, pan the canvas, select a shape, or enter a command. In Virtual Reality (VR), a hand gesture-based 3D modelling application may have different modes for object creation, selection, and transformation. Depending on the mode, the movement of the hand is interpreted differently. However, one of the crucial factors determining the effectiveness of an interface is user productivity. Mode-switching time of different input techniques, either in a touch interface or in a mid-air interface, affects user productivity. Moreover, when touch and mid-air interfaces like VR are combined, making informed decisions pertaining to the mode assignment gets even more complicated. This thesis provides an empirical investigation to characterize the mode switching phenomenon in barehand touch-based and mid-air interfaces. It explores the potential of using these input spaces together for a productivity application in VR. And, it concludes with a step towards defining and evaluating the multi-faceted mode concept, its characteristics and its utility, when designing user interfaces more generally

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Poetic design: a theory of everyday practice

This study aims to define design poetics as a category of design practice set apart from commercial, industrial or market-led design that generates a collection of experimental artefacts which investigate the everyday life of contemporary culture. It is argued that in creating an active interplay between users (human agents) and objects, poetic design involves a different kind of production (which is not about improving the functionality of a product) and alternative forms of "consumption" (which is not about a „using up‟ of objects), by developing new practices of living with things. As such it is suggested that design poetics depends on the production developed by consumers as a creative users (postproducers), within unconventional experiential and social scenarios of living. In changing the bilateral relationship object-user poetic design develops objects from the point of view of the user – its activities and models of operation and this aspect is related to an emotional and experiential evaluation. Thus the study proposes a re-evaluation of objects and users through experiential, narrative and performative criteria in order to understand their various roles and functions. In proposing these particular points of evaluation, poetic objects are distinguished as a particular category of objects together with the practices they engender or support; and within a network of relationships and contexts, as specific sites of interaction.1 In this light, it is shown that poetic design proposes a class of objects that respond to needs beyond the objects‟ instrumental (functional, practical) power; but to their contribution to life experience, embodying a variety of processes and manifestations. They translate immaterial interactions and make these interrelations visible

Redefining editorial experience: user experience & user interface design in digital publications

The publishing industry is facing a transition era, where the development of new digital technologies has led people to adopt new reading habits, where every day is less common to receive content in a piece of printed paper, and more and more common to do it through a screen. Digital publications have been present for a while, facing constant changes, so this theoretical-practical research aims to explore their transition, what they have done, and what are they doing to respond to their readers’ needs. Emphasized in digital magazines, the objective is to understand them from its roots, analyzing concepts related to editorial design and through theory and the work of designers who redefined the concept of publications on the screens. This theoretical part is complemented with the study of two areas of design that are related to the development of digital products: user experience and user interface. The two case studies presented, Wired and The New Yorker magazines explore what magazines that have lead the market in terms of innovation and design have done in recent years and how are they facing today’s challenges. In addition, in this critical analysis exploration, topics such as their online strategy, design of their apps, and their activity in social networks are explored. The practical project developed, seeks to apply the concepts explored in the theoretical part of the research, in order to respond to the needs of a cultural magazine of Mexico, La Tempestad. Through a survey, the reading habits of the target audience are analyzed to understand what readers are looking for and create a solution that is appropriate for them. A graphic proposal of a digital magazine in the form of an application is presented, with interactions, design, and navigational tools the app seeks to create a unique experience with their readers