Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

When conventional procedures are no longer the rule for application: design as a discipline opens up to new possibilities

This paper discusses the development of the prototype application ‘LabanAssist’. It looks at the design rationale used for the creation of what is fundamentally a system for recording dance knowledge on a score, as identifiable and replicable signs and symbols. A system made necessary because the conventions of other established disciplines, such as engineering and computer science practices, were no longer considered to be effective alone, in facilitating the production of well-designed cultural artefacts (Calvert, Fox, Ryman, & Wilke, 2005; Ebenreuter, 2005). It is important to ask how can we understand design as a discipline amongst other fields of study with longstanding conventions and traditions and if the discipline of design offers effective ways of thinking about the creation and art of making products or services for the enhancement of the human experience? Is design a discipline because it adheres to existing and established rules of interdisciplinary knowledge from which it draws, or is it a discipline in its own right that as a significant field of intellectual development utilizes interdisciplinary knowledge as a basis for creativity and invention?” While there is no simple answer to these questions, the design approach adopted for the development of the prototype application ‘LabanAssist’ offers a working example in which the central theme of grammar, or more particularly the rules of a language, depart from the conventional use for its practical application. This application is one in which a literal understanding of grammar is no longer seen as an adequate basis for the generation of dance knowledge expressed via symbolic writing systems. Instead, this research focuses on the way in which the figurative aspects of language can be represented in the design of an interface to orient user thinking and facilitate the generation of diverse movement compositions. Keywords: Labanotation; Grammar; Literal; Figurative; Tropes; Poetic Constructs; Broad Terms; Interface.</p

Design and semantics of form and movement (DeSForM 2006)

Design and Semantics of Form and Movement (DeSForM) grew from applied research exploring emerging design methods and practices to support new generation product and interface design. The products and interfaces are concerned with: the context of ubiquitous computing and ambient technologies and the need for greater empathy in the pre-programmed behaviour of the ‘machines’ that populate our lives. Such explorative research in the CfDR has been led by Young, supported by Kyffin, Visiting Professor from Philips Design and sponsored by Philips Design over a period of four years (research funding £87k). DeSForM1 was the first of a series of three conferences that enable the presentation and debate of international work within this field: • 1st European conference on Design and Semantics of Form and Movement (DeSForM1), Baltic, Gateshead, 2005, Feijs L., Kyffin S. & Young R.A. eds. • 2nd European conference on Design and Semantics of Form and Movement (DeSForM2), Evoluon, Eindhoven, 2006, Feijs L., Kyffin S. & Young R.A. eds. • 3rd European conference on Design and Semantics of Form and Movement (DeSForM3), New Design School Building, Newcastle, 2007, Feijs L., Kyffin S. & Young R.A. eds. Philips sponsorship of practice-based enquiry led to research by three teams of research students over three years and on-going sponsorship of research through the Northumbria University Design and Innovation Laboratory (nuDIL). Young has been invited on the steering panel of the UK Thinking Digital Conference concerning the latest developments in digital and media technologies. Informed by this research is the work of PhD student Yukie Nakano who examines new technologies in relation to eco-design textiles

Using Design Fiction to Inform Shape-Changing Interface Design and Use

Shape-changing interfaces are tangible, physically dynamic devices which enable user-experience beyond 2D screens. Within Human Computer Interaction, researchers are developing these from low-resolution, low-fidelity prototypes, toward a vision of a truly malleable world. The main focus is in producing and testing hardware, and basic user interactions, which leaves the question unanswered: what are shape-changing interfaces good for? In response, we propose the use of design fiction to investigate potential applications for this technology: to create and analyse artifacts relating to future use-scenarios for shape-change. Whilst research within shape-change often proposes future use-cases for prototypes during discussion, they are seldom in a form that presents them as everyday artifacts. Here, we present and discuss a printed game-play instruction manual for a truly high resolution shape-changing game entitled First Hand, which aims to draw parallels between current gaming practices and the tangible nature of shape-changing interfaces

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities

Using Design Fiction to Inform Shape-Changing Interface Design and Use

Shape-changing interfaces are tangible, physically dynamic devices which enable user-experience beyond 2D screens. Within Human Computer Interaction, researchers are developing these from low-resolution, low-fidelity prototypes, toward a vision of a truly malleable world. The main focus is in producing and testing hardware, and basic user interactions, which leaves the question unanswered: what are shape-changing interfaces good for? In response, we propose the use of design fiction to investigate potential applications for this technology: to create and analyse artifacts relating to future use-scenarios for shape-change. Whilst research within shape-change often proposes future use-cases for prototypes during discussion, they are seldom in a form that presents them as everyday artifacts. Here, we present and discuss a printed game-play instruction manual for a truly high resolution shape-changing game entitled First Hand, which aims to draw parallels between current gaming practices and the tangible nature of shape-changing interfaces

Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of domains and social contexts of Human-Computer Interaction: the embedding of computing capabilities into physical hand-held objects, and the emergence of large interactive surfaces, such as tabletops and wall boards. Both interactive surfaces and small computational devices usually allow for direct and space-multiplex input, i.e., for the spatial coincidence of physical action and digital output, in multiple points simultaneously. Such a powerful combination opens novel opportunities for the design of what are considered as hybrid interactions in this work. This thesis explores the affordances of physical interaction as resources for interface design of such hybrid interactions. The hybrid systems that are elaborated in this work are envisioned to support specific social and physical contexts, such as collaborative cooking in a domestic kitchen, or collaborative creativity in a design process. In particular, different aspects of physicality characteristic of those specific domains are explored, with the aim of promoting skill transfer across domains. irst, different approaches to the design of space-multiplex, function-specific interfaces are considered and investigated. Such design approaches build on related work on Graspable User Interfaces and extend the design space to direct touch interfaces such as touch-sensitive surfaces, in different sizes and orientations (i.e., tablets, interactive tabletops, and walls). These approaches are instantiated in the design of several experience prototypes: These are evaluated in different settings to assess the contextual implications of integrating aspects of physicality in the design of the interface. Such implications are observed both at the pragmatic level of interaction (i.e., patterns of users' behaviors on first contact with the interface), as well as on user' subjective response. The results indicate that the context of interaction affects the perception of the affordances of the system, and that some qualities of physicality such as the 3D space of manipulation and relative haptic feedback can affect the feeling of engagement and control. Building on these findings, two controlled studies are conducted to observe more systematically the implications of integrating some of the qualities of physical interaction into the design of hybrid ones. The results indicate that, despite the fact that several aspects of physical interaction are mimicked in the interface, the interaction with digital media is quite different and seems to reveal existing mental models and expectations resulting from previous experience with the WIMP paradigm on the desktop PC

Using Design Fiction to Inform Shape-Changing Interface Design and Use

Shape-changing interfaces are tangible, physically dynamic devices which enable user-experience beyond 2D screens. Within Human Computer Interaction, researchers are developing these from low-resolution, low-fidelity prototypes, toward a vision of a truly malleable world. The main focus is in producing and testing hardware, and basic user interactions, which leaves the question unanswered: what are shape-changing interfaces good for? In response, we propose the use of design fiction to investigate potential applications for this technology: to create and analyse artifacts relating to future use-scenarios for shape-change. Whilst research within shape-change often proposes future use-cases for prototypes during discussion, they are seldom in a form that presents them as everyday artifacts. Here, we present and discuss a printed game-play instruction manual for a truly high resolution shape-changing game entitled First Hand, which aims to draw parallels between current gaming practices and the tangible nature of shape-changing interfaces