Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

TUIs vs. GUIs : comparing the learning potential with preschoolers

In an effort to better understand the learning potential of a tangible interface, we conducted a comparison study between a tangible and a traditional graphical user interface for teaching preschoolers (In Portugal, children enter preschool at the age of three and they attend it till entering school, normally at the age of six) about good oral hygiene. The study was carried with two groups of children aged 4 to 5 years. Questionnaires to parents, children’s drawings, and interviews were used for data collection and analysis and revealed important indicators about children’s change of attitude, involvement, and preferences for the interfaces. The questionnaires showed a remarkable change of attitude toward tooth brushing in the children that interacted with the tangible interface; particularly children’s motivation increased significantly. Children’s drawings were used to assess their degree of involvement with the interfaces. The drawings from the children that interacted with the tangible interface were very complete and detailed suggesting that the children felt actively involved with the experience. The results suggest that the tangible interface was capable of promoting a stronger and long-lasting involvement having a greater potential to engage children, therefore potentially promoting learning. Evaluation through drawing seems to be a promising method to work with preliterate children; however,it is advisable to use it together with other methods.Fundação para a Ciência e Tecnologia (FCT

Playful E-textile Sonic Interaction for Socially Engaged and Open-Ended Play Between Autistic Children

Research on the potential benefits of technology for autistic children is an emergent field in Human-Computer Interaction (HCI), especially within the Child-Computer Interaction Community. This thesis contributes a design approach grounded in theories of play, cognitive development, and autism to expand the discourse on methodological guidelines for performing empirical studies with non-verbal autistic children and to extend the design space to cater to the socio-emotional and sensory needs of this population. The thesis reveals how sonic e-textile Tangible User Interfaces (TUIs) can be used effectively to mediate children’s social participation in playful activities. This is demonstrated through developing three explorative field-studies conducted at a specialist school based in North-East London where two sonic e-textile playful TUIs, namely Mazi and Olly, have been created and tested with three groups of autistic children aged between 5-10. The three studies ran over the period of three years and were designed to investigate the potentials of TUIs as shareable toys during leisure and recreational activities to a) support social and playful interactions among peers and b) provide opportunities for self-regulation. The key contributions of this thesis are the designs of two tangible user interfaces, which offer a set of design approaches to guide researchers through creating shareable and playful tangibles for non-verbal autistic children; a framework for analysis and a thorough evaluation process that other researchers could use to assess the efficacy of playful TUI designs for nonverbal autistic children; and an in-depth discussion about the research process, which offers a new perspective about holistic designs and evaluation of technologies that aim to scaffold play in groups non-verbal autistic children

Kurio: A museum guide for families

We discuss three design strategies for improving the quality of social interaction and learning with interactive museum guides: 1) embodied interaction; 2) game-learning; 3) a hybrid system. We used these strategies in our prototype Kurio, which is aimed at supporting families visiting museums. The results of our evaluation show positive implications of implementing the design strategies: closing the social gap, naturalizing technology, and supporting exploration and discovery in learning

Informing the design of a multisensory learning environment for elementary mathematics learning

It is well known that primary school children may face difficulties in acquiring mathematical competence, possibly because teaching is generally based on formal lessons with little opportunity to exploit more multisensory-based activities within the classroom. To overcome such difficulties, we report here the exemplary design of a novel multisensory learning environment for teaching mathematical concepts based on meaningful inputs from elementary school teachers. First, we developed and administered a questionnaire to 101 teachers asking them to rate based on their experience the learning difficulty for specific arithmetical and geometrical concepts encountered by elementary school children. Additionally, the questionnaire investigated the feasibility to use multisensory information to teach mathematical concepts. Results show that challenging concepts differ depending on children school level, thus providing a guidance to improve teaching strategies and the design of new and emerging learning technologies accordingly. Second, we obtained specific and practical design inputs with workshops involving elementary school teachers and children. Altogether, these findings are used to inform the design of emerging multimodal technological applications, that take advantage not only of vision but also of other sensory modalities. In the present work, we describe in detail one exemplary multisensory environment design based on the questionnaire results and design ideas from the workshops: the Space Shapes game, which exploits visual and haptic/proprioceptive sensory information to support mental rotation, 2D–3D transformation and percentages. Corroborating research evidence in neuroscience and pedagogy, our work presents a functional approach to develop novel multimodal user interfaces to improve education in the classroom

Tangible user interfaces and social interaction in children with autism

Tangible User Interfaces (TUIs) offer the potential for new modes of social interaction for children with Autism Spectrum Conditions (ASC). Familiar objects that are embedded with digital technology may help children with autism understand the actions of others by providing feedback that is logical and predictable. Objects that move, playback sound or create sound – thus repeating programmed effects – offer an exciting way for children to investigate objects and their effects. This thesis presents three studies of children with autism interacting with objects augmented with digital technology. Study one looked at Topobo, a construction toy augmented with kinetic memory. Children played with Topobo in groups of three of either Typically Developing (TD) or ASC children. The children were given a construction task, and were also allowed to play with the construction sets with no task. Topobo in the task condition showed an overall significant effect for more onlooker, cooperative, parallel, and less solitary behaviour. For ASC children significantly less solitary and more parallel behaviour was recorded than other play states. In study two, an Augmented Knights Castle (AKC) playset was presented to children with ASC. The task condition was extended to allow children to configure the playset with sound. A significant effect in a small sample was found for configuration of the AKC, leading to less solitary behaviour, and more cooperative behaviour. Compared to non-digital play, the AKC showed reduction of solitary behaviour because of augmentation. Qualitative analysis showed further differences in learning phase, user content, behaviour oriented to other children, and system responsiveness. Tangible musical blocks (‘d-touch’) in study three focused on the task. TD and ASC children were presented with a guided/non-guided task in pairs, to isolate effects of augmentation. Significant effects were found for an increase in cooperative symbolic play in the guided condition, and more solitary functional play was found in the unguided condition. Qualitative analysis highlighted differences in understanding blocks and block representation, exploratory and expressive play, understanding of shared space and understanding of the system. These studies suggest that the structure of the task conducted with TUIs may be an important factor for children’s use. When the task is undefined, play tends to lose structure and the benefits of TUIs decline. Tangible technology needs to be used in an appropriately structured manner with close coupling (the distance between digital housing and digital effect), and works best when objects are presented in familiar form

Comparing TUIs and GUIs for primary school programming

There is considerable interest in using tangible user interfaces (TUIs) to support teaching children programming, but evidence for the benefits is mixed, and their deployment in school environments presents more challenges than graphical user interfaces (GUIs). This study investigates the effect of GUIs and TUIs on learning outcomes, attitudes toward computing, and reported enjoyment in a computer-programming activity with primary-school students aged 6-7 in Saudi Arabia. Forty-two students engaged in a 45-minute learning activity using either a TUI or GUI programming environment. The study used a between-groups design, and quantitative data were collected, including pre-test and post-test results, and ratings on attitudinal and enjoyment surveys. Learning gains were significantly higher for the GUI group than the TUI group. However, post-activity increases in reported attitude toward computing were significantly higher for the TUI group. There was no difference in activity enjoyment scores, which were high for both groups

Software architectural support for tangible user interfaces in distributed, heterogeneous computing environments

This research focuses on tools that support the development of tangible interaction-based applications for distributed computing environments. Applications built with these tools are capable of utilizing heterogeneous resources for tangible interaction and can be reconfigured for different contexts with minimal code changes. Current trends in computing, especially in areas such as computational science, scientific visualization and computer supported collaborative work, foreshadow increasing complexity, distribution and remoteness of computation and data. These trends imply that tangible interface developers must address concerns of both tangible interaction design and networked distributed computing. In this dissertation, we present a software architecture that supports separation of these concerns. Additionally, a tangibles-based software development toolkit based on this architecture is presented that enables the logic of elements within a tangible user interface to be mapped to configurations that vary in the number, type and location of resources within a given tangibles-based system

An Inquiry into the TUI Design Space for Parent-Child Math Engagement at Home

Preschoolers’ early-math development is vital for their later math and academic achievement. Tangible user interfaces (TUI) may support early math as they feature physical objects imperative to math development and multimedia to support engagement. As a potentially meaningful context for TUIs, developmental studies highlight the need to support the home math environment (HME) that covers math-related interactions among parents and children. Therefore, we focus on HME as a design space that has not been investigated in TUI literature. We conducted an observational study involving physical-object based math activities and semi-structured interviews with 13 parent-child dyads. Our findings revealed the multifaceted nature of the HME, where children's agency is valued and providing lasting materials is challenging. Also, we realized that parents juggled their child's demands and the object-based physical activity at once. By reflecting on these findings, we propose design directions for supporting the home-math environment with TUIs