Tangibles: Technologies and interaction for learning

With recent developments in computing and networking new kinds of interfaces, such as tangible interfaces, and consequently new forms of interaction with technology, have emerged. ‘Tangibles’ generally refer to interfaces where computational power is embedded in everyday artefacts or customised objects, which can be wirelessly networked or linked to various forms of digital representation. The emergence of increasingly small microchips and digital sensing technologies means that embedding technology in both artefacts and the environment is becoming more commonplace. In the field of Human Computer Interaction (HCI) this group of technologies may be described as graspable interfaces (e.g. Fitzmaurice et al., 1995), tangible interaction (e.g. Ullmer and Ishii, 2001), and tangible bits (e.g. Ishii, 1997). Shaer and Hornecker’s (2010) definition offers a useful description for the purposes of this chapter: ‘‘Interfaces that are concerned with providing tangible representations to digital information and controls, allowing users to quite literally grasp data with their hands” and thus physically manipulate associated representations (p. 4). There are three key categories of systems that sit under this umbrella term: constructive assembly kits, token and constraint systems, and interactive surfaces

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

inFORM: Dynamic Physical Affordances and Constraints through Shape and Object Actuation

Past research on shape displays has primarily focused on rendering content and user interface elements through shape output, with less emphasis on dynamically changing UIs. We propose utilizing shape displays in three different ways to mediate interaction: to facilitate by providing dynamic physical affordances through shape change, to restrict by guiding users with dynamic physical constraints, and to manipulate by actuating physical objects. We outline potential interaction techniques and introduce Dynamic Physical Affordances and Constraints with our inFORM system, built on top of a state-of-the-art shape display, which provides for variable stiffness rendering and real-time user input through direct touch and tangible interaction. A set of motivating examples demonstrates how dynamic affordances, constraints and object actuation can create novel interaction possibilities.National Science Foundation (U.S.). Graduate Research Fellowship (Grant 1122374)Swedish Research Council (Fellowship)Blanceflor Foundation (Scholarship

Toolkits for the Development of Hybrid Games: from Tangible Tabletops to Interactive Spaces

Durante los últimos años, los dispositivos tabletop han sido considerados el entorno ideal para los juegos híbridos, los cuales combinan técnicas de juego tradicional, como el uso de objetos físicos para interactuar con el juego de una forma natural, con las nuevas posibilidades que los tabletops ofrecen de aumentar el espacio de juego con imágenes digitales y audio.Sin embargo, los juegos híbridos no se restringen simplemente a tabletops, pudiéndose jugar también en entornos más amplios en los que convergen otros paradigmas de interacción. Por esta razón, el uso de juegos híbridos en Espacios Interactivos está ganando fuerza, pero el número y heterogeneidad de dispositivos y estilos de interacción que se encuentran en estos entornos hace que el diseño y prototipado de juegos sea una tarea difícil. Por lo tanto, el gran reto se encuentra en ofrecer a diseñadores y desarrolladores herramientas apropiadas para la creación de estas aplicaciones.En esta línea de trabajo, el grupo Affective Lab lanzó el proyecto JUGUEMOS (TIN2015-67149-C3-1R), un proyecto nacional centrado en el desarrollo de juegos híbridos en entornos interactivos. Esta Tesis Doctoral se enmarca en este proyecto.El primer paso de la realización de esta tesis fue establecer los dos objetivos principales (Capítulo 1):1) El primer objetivo que se estableció fue profundizar en el uso de tabletops tangibles en terapia con niños con necesidades especiales. Durante los últimos años el grupo Affective Lab había visto la potencialidad de los tabletops tangibles para el trabajo con niños pequeños, pero todavía era necesario llevar a cabo más experiencias y evaluaciones en el ámbito terapéutico, así como explorar si otros grupos de usuarios (adultos con problemas cognitivos) podían beneficiarse de las características de los tabletops.2) El segundo objetivo consistió en diseñar e implementar un toolkit para el desarrollo de juegos híbridos para espacios interactivos. Se decidió que el toolkit estuviera dirigido a desarrolladores para facilitar su trabajo a la hora de crear este tipo de aplicaciones.Una vez establecidos los objetivos, se realizó un estado del arte a su vez dividido en dos partes (Capítulo 2):1) Se realizó una categorización de juegos híbridos para entender y extraer sus principales características, así como los principales retos que surgen al desarrollar este tipo de juegos. También se estudiaron toolkits cuyo objetivo era el desarrollo de juegos híbridos.2) Se estudiaron juegos híbridos desarrollados para niños con necesidades especiales y adultos con problemas cognitivos que hacían uso de la Interacción Tangible y tabletops, así como toolkits dirigidos a terapeutas o educadores para ayudarles en la creación de actividades para sus pacientes.Para llevar a cabo las experiencias y evaluaciones relacionadas con el primer objetivo, se hizo uso del tabletop tangible NIKVision, desarrollado previamente por el grupo Affective Lab, y el toolkit KitVision, una herramienta dirigida a profesionales sin conocimientos de programación para la creación de actividades tangibles y que fue desarrollado durante el Proyecto Final de Carrera de la autora. En el Capítulo 3 de esta Tesis se comenta brevemente el tabletop NIKVision y la arquitectura de KitVision, se describen las evaluaciones que se llevaron a cabo con terapeutas con el objetivo de mejorar y probar la utilidad del toolkit, y se explica una experiencia de un año durante la cual una terapeuta ocupacional de ASAPME, una asociaciónque trabaja con adultos con problemas cognitivos, estuvo usando el tabletop y el toolkit sin supervisión.En el Capítulo 4 se describen diferentes experiencias con KitVision que se llevaron a cabo:- Gracias a una colaboración con la Residencia Romareda, NIKVision y KitVision fueron instalados provisionalmente en la residencia y, tras una evaluación inicial, se desarrollaron tres nuevas actividades para los usuarios de la residencia.- Gracias a la colaboración con ENMOvimienTO y con uno de los centros de Atención Temprana del Instituto Aragonés de Servicios Sociales (IASS), ambos enfocados a trabajar con niños con problemas de aprendizaje, se pudieron realizar evaluaciones que nos permitieron mejorar KitVision y crear nuevas actividades específicamente diseñadas para ellos.- Finalmente, gracias a una colaboración con Atenciona, pudimos evaluar actividades con niños con Trastorno por Déficit de Atención e Hiperactividad (TDAH) y extraer una serie de directrices para diseñar actividades para este tipo de niños. También pudimos llevar a cabo una experiencia de Diseño Participativo con estos niños.El completo desarrollo del toolkit JUGUEMOS, para la creación de juegos híbridos en espacios interactivos, se explica en el Capítulo 5. En este apartado primero se describe el Espacio Interactivo JUGUEMOS que sirvió de base para desarrollar el toolkit. Después se explican con detalle las decisiones de diseño que se tomaron, el modelo de abstracción que se usó para diseñar los juegos, y la arquitectura del toolkit. También se detallan las distintas fases de implementación que se llevaron a cabo, basadas en los tres retos que se extrajeron en el estado del arte: (1) integrar diferentes dispositivos, (2) gestionar salidas gráficas diversas y (3) facilitar la codificación del juego. Finalmente, se presentan dos prototipos de juegos que se desarrollaron durante las dos estancias de investigación que la autora realizó.Finalmente, en el Capítulo 6 se describen los tres casos de uso que se realizaron para tener una primera valoración de la usabilidad del toolkit JUGUEMOS: (1) una evaluación con estudiantes de Máster en la que se implementó un juego completamente funcional para el Espacio Interactivo JUGUEMOS, (2) un juego que fue completamente desarrollado usando el toolkit JUGUEMOS una vez que éste se acabó de implementar, (3) una experiencia que involucró a dos grupos multidisciplinares compuestos por diseñadores y desarrolladores, en la que tuvieron que colaborar para diseñar e implementar dos prototipos de juegos híbridos para el espacio interactivo.<br /

Situated play in a tangible interface and adaptive audio museum guide

This paper explores the design issues of 9 situated play within a museum through the study of a 10 museum guide prototype that integrates a tangible interface, audio display, and adaptive modeling. We discuss our use of design ethnography in order to situate our interaction and to investigate the liminal and engagement qualities of a museum visit. The paper provides an overview of our case study and analysis of our user evaluation. We discuss the implications including degrees of balance in the experience design of play in interaction; the challenge in developing a discovery-based information model, and the need for a better understanding of the contextual aspects of tangible user interfaces (TUIs). We conclude that learning effectiveness and functionality can be balanced productively with playful interaction through an adaptive audio and TUI if designers balance the engagement between play and the environment, and the space between imagination and interpretation that links the audio content to the artifacts

Toward New Ecologies of Cyberphysical Representational Forms, Scales, and Modalities

Research on tangible user interfaces commonly focuses on tangible interfaces acting alone or in comparison with screen-based multi-touch or graphical interfaces. In contrast, hybrid approaches can be seen as the norm for established mainstream interaction paradigms. This dissertation describes interfaces that support complementary information mediations, representational forms, and scales toward an ecology of systems embodying hybrid interaction modalities. I investigate systems combining tangible and multi-touch, as well as systems combining tangible and virtual reality interaction. For each of them, I describe work focusing on design and fabrication aspects, as well as work focusing on reproducibility, engagement, legibility, and perception aspects

Kolab: Improvising Nomadic Tangible User Interfaces in the Workplace for Co-Located Collaboration

Tangible User Interfaces (TUIs) [Ishii 1997] offer an interface style that couples "digital information to everyday physical objects and environments" [Ishii 1997 page 2]. However this physicality may also be a limitation as the tendency to use iconic representations for tangibles can result in inflexible 'concrete and specialised objects' [Shaer 2009 page 107]. The current research investigates whether by reducing the dependence on specific tangible sets through the use of improvised tangibles we may begin to address the issue of tangible flexibility within TUIs. Improvised tangibles may be characterised by being potentially arbitrary and abstract, in that they may bear little or no resemblance to the underlying digital value. Core literature in the field (e. g. [Fitzmaurice 1996] [Ishii 2008] [Hornecker 2006] [Holmquist 1999]) suggests that a system based on improvised tangibles would suffer from impaired usability and so the research focuses on the impact on usability due to a lack of close representational significance [Ullmer 2000] during co-located collaboration. Using a prototyping methodology a functional, shareable, TUI system was developed based on computer vision techniques using the Microsoft Kinect [Microsoft2011]. This prototype system ('Kolab') was used to explore an interaction design that supports the dynamic binding of improvised tangibles to digital values. A simple co-located collaborative task was developed using 'Kolab' and a user study was conducted to investigate the usability of the system in a collaborative context. Within the limitations of the simple task the results of the study show that a) users appeared comfortable with improvising artefacts b) the high rate of task completion strongly suggests that a lack of close representational significance does not impair system usability and c) despite some temporary issues with users interfering with other's action an overall indication of equitable participation suggests that collaboration was not impaired by the 'Kolab' prototype

Beyond representations: towards an action-centric perspective on tangible interaction

In the light of theoretical as well as concrete technical development, we discuss a conceptual shift from an information-centric to an action-centric perspective on tangible interactive technology. We explicitly emphasise the qualities of shareable use, and the importance of designing tangibles that allow for meaningful manipulation and control of the digital material. This involves a broadened focus from studying properties of the interface, to instead aim for qualities of the activity of using a system, a general tendency towards designing for social and sharable use settings and an increased openness towards multiple and subjective interpretations. An effect of this is that tangibles are not designed as representations of data, but as resources for action. We discuss four ways that tangible artefacts work as resources for action: (1) for physical manipulation; (2) for referential, social and contextually oriented action; (3) for perception and sensory experience; (4) for digitally mediated action

Disciplining the body? Reflections on the cross disciplinary import of ‘embodied meaning’ into interaction design

The aim of this paper is above all critically to examine and clarify some of the negative implications that the idea of ‘embodied meaning’ has for the emergent field of interaction design research. Originally, the term ‘embodied meaning’ has been brought into HCI research from phenomenology and cognitive semantics in order to better understand how user’s experience of new technological systems relies to an increasing extent on full-body interaction. Embodied approaches to technology design could thus be found in Winograd & Flores (1986), Dourish (2001), Lund (2003), Klemmer, Hartman & Takayama (2006), Hornecker & Buur (2006), Hurtienne & Israel (2007) among others. However, fertile as this cross-disciplinary import may be, design research can generally be criticised for being ‘undisciplined’, because of its tendency merely to take over reductionist ideas of embodied meaning from those neighbouring disciplines without questioning the inherent limitations it thereby subscribe to. In this paper I focus on this reductionism and what it means for interaction design research. I start out by introducing the field of interaction design and two central research questions that it raises. This will serve as a prerequisite for understanding the overall intention of bringing the notion of ‘embodied meaning’ from cognitive semantics into design research. Narrowing my account down to the concepts of ‘image schemas’ and their ‘metaphorical extension’, I then explain in more detail what is reductionistic about the notion of embodied meaning. Having done so, I shed light on the consequences this reductionism might have for design research by examining a recently developed framework for intuitive user interaction along with two case examples. In so doing I sketch an alternative view of embodied meaning for interaction design research. Keywords: Interaction Design, Embodied Meaning, Tangible User Interaction, Design Theory, Cognitive Semiotics</p

Diving in at the deep end : the value of alternative in-situ approaches for systematic library search

OPAC interfaces, still the dominant access point to library catalogs, support systematic search but are problematic for open-ended exploration and generally unpopular with visitors. As a result, libraries start subscribing to simplified search paradigms as exemplified by web-search systems. This is a problem considering that systematic search is a crucial skill in the light of today’s abundance of digital information. Inspired by novel approaches to facilitating search, we designed CollectionDiver, an installation for supporting systematic search in public libraries. The CollectionDiver combines tangible and large display direct-touch interaction with a visual representation of search criteria and filters. We conducted an in-situ qualitative study to compare participants’ search approaches on the CollectionDiver with those on the OPAC interface. Our findings show that while both systems support a similar search process, the CollectionDiver (1) makes systematic search more accessible, (2) motivates proactive search approaches by (3) adding transparency to the search process, and (4) facilitates shared search experiences. We discuss the CollectionDiver’s design concepts to stimulate new ideas toward supporting engaging approaches to systematic search in the library context and beyond.Postprin