Towards a framework for investigating tangible environments for learning

External representations have been shown to play a key role in mediating cognition. Tangible environments offer the opportunity for novel representational formats and combinations, potentially increasing representational power for supporting learning. However, we currently know little about the specific learning benefits of tangible environments, and have no established framework within which to analyse the ways that external representations work in tangible environments to support learning. Taking external representation as the central focus, this paper proposes a framework for investigating the effect of tangible technologies on interaction and cognition. Key artefact-action-representation relationships are identified, and classified to form a structure for investigating the differential cognitive effects of these features. An example scenario from our current research is presented to illustrate how the framework can be used as a method for investigating the effectiveness of differential designs for supporting science learning

The effect of representation location on interaction in a tangible learning environment

Drawing on the 'representation' TUI framework [21], this paper reports a study that investigated the concept of 'representation location' and its effect on interaction and learning. A reacTIVision-based tangible interface was designed and developed to support children learning about the behaviour of light. Children aged eleven years worked with the environment in groups of three. Findings suggest that different representation locations lend themselves to different levels of abstraction and engender different forms and levels of activity, particularly with respect to speed of dynamics and differences in group awareness. Furthermore, the studies illustrated interaction effects according to different physical correspondence metaphors used, particularly with respect to combining familiar physical objects with digital--based table-top representation. The implications of these findings for learning are discussed

AGORAS: Exploring Creative Learning on Tangible User Interfaces

Departing from creative learning foundations, this paper discusses on the suitability of interactive tables as a grounding technology to support creative learning for several reasons: support for social learning, because the subjects share a physical space as in traditional non-digital technologies; communication during the creative, experimental and reflexive process is direct and not computer-mediated; and subjects can carry out the task in parallel on the same surface. Considering reflection, discussion and creation processes in a loop, an experiment with teenagers has been conducted comparing a digital-based against a pure tangible tabletop in a task of creating entities consisting of blocks and joint elements. This preliminary study, designed to obtain initial insights about whether the grounding technology may become a promising tool to support creative learning, explores some aspects such as productivity, complexity of designs and concurrent comanipulation. The results showed that subjects were more productive in terms of the number of solutions obtained using the non computer-mediated approach. However using the digital tabletop approach subjects design, on average, more complex or elaborate solutions in terms of the number of involved bodies and joints. Finally, an important finding was that teams established more frequently concurrent cooperation schemes in the digital tabletop condition by sharing more effectively the creation space.This work was funded by the Spanish Ministry of Education and Science and Innovation under the National Strategic Program of Scientific Research, Development and Technological Innovation (I+D+i) and project TSI2010-20488. Our thanks to the Alaquas city council and the clubhouse’s managers. Thanks also to the team Polimedia of the office “Área de Información y Comunicaciones” (ASIC) for the support in computer hardware. A. Catalá is supported by a FPU fellowship from the Ministry of Education and Science of Spain with reference AP2006-00181.Catalá Bolós, A.; Jaén Martínez, FJ.; Martínez-Villaronga, A.; Mocholí Agües, JA. (2011). AGORAS: Exploring Creative Learning on Tangible User Interfaces. En Computer Software and Applications Conference (COMPSAC), 2011 IEEE 35th Annual. Institute of Electrical and Electronics Engineers (IEEE). 326-335. https://doi.org/10.1109/COMPSAC.2011.50S32633

Move, hold and touch: A framework for Tangible gesture interactive systems

© 2015 by the authors. Technology is spreading in our everyday world, and digital interaction beyond the screen, with real objects, allows taking advantage of our natural manipulative and communicative skills. Tangible gesture interaction takes advantage of these skills by bridging two popular domains in Human-Computer Interaction, tangible interaction and gestural interaction. In this paper, we present the Tangible Gesture Interaction Framework (TGIF) for classifying and guiding works in this field. We propose a classification of gestures according to three relationships with objects: move, hold and touch. Following this classification, we analyzed previous work in the literature to obtain guidelines and common practices for designing and building new tangible gesture interactive systems. We describe four interactive systems as application examples of the TGIF guidelines and we discuss the descriptive, evaluative and generative power of TGIF

A Framework For Abstracting, Designing And Building Tangible Gesture Interactive Systems

This thesis discusses tangible gesture interaction, a novel paradigm for interacting with computer that blends concepts from the more popular fields of tangible interaction and gesture interaction. Taking advantage of the human innate abilities to manipulate physical objects and to communicate through gestures, tangible gesture interaction is particularly interesting for interacting in smart environments, bringing the interaction with computer beyond the screen, back to the real world. Since tangible gesture interaction is a relatively new field of research, this thesis presents a conceptual framework that aims at supporting future work in this field. The Tangible Gesture Interaction Framework provides support on three levels. First, it helps reflecting from a theoretical point of view on the different types of tangible gestures that can be designed, physically, through a taxonomy based on three components (move, hold and touch) and additional attributes, and semantically, through a taxonomy of the semantic constructs that can be used to associate meaning to tangible gestures. Second, it helps conceiving new tangible gesture interactive systems and designing new interactions based on gestures with objects, through dedicated guidelines for tangible gesture definition and common practices for different application domains. Third, it helps building new tangible gesture interactive systems supporting the choice between four different technological approaches (embedded and embodied, wearable, environmental or hybrid) and providing general guidance for the different approaches. As an application of this framework, this thesis presents also seven tangible gesture interactive systems for three different application domains, i.e., interacting with the In-Vehicle Infotainment System (IVIS) of the car, the emotional and interpersonal communication, and the interaction in a smart home. For the first application domain, four different systems that use gestures on the steering wheel as interaction means with the IVIS have been designed, developed and evaluated. For the second application domain, an anthropomorphic lamp able to recognize gestures that humans typically perform for interpersonal communication has been conceived and developed. A second system, based on smart t-shirts, recognizes when two people hug and reward the gesture with an exchange of digital information. Finally, a smart watch for recognizing gestures performed with objects held in the hand in the context of the smart home has been investigated. The analysis of existing systems found in literature and of the system developed during this thesis shows that the framework has a good descriptive and evaluative power. The applications developed during this thesis show that the proposed framework has also a good generative power.Questa tesi discute l’interazione gestuale tangibile, un nuovo paradigma per interagire con il computer che unisce i principi dei più comuni campi di studio dell’interazione tangibile e dell’interazione gestuale. Sfruttando le abilità innate dell’uomo di manipolare oggetti fisici e di comunicare con i gesti, l’interazione gestuale tangibile si rivela particolarmente interessante per interagire negli ambienti intelligenti, riportando l’attenzione sul nostro mondo reale, al di là dello schermo dei computer o degli smartphone. Poiché l’interazione gestuale tangibile è un campo di studio relativamente recente, questa tesi presenta un framework (quadro teorico) che ha lo scopo di assistere lavori futuri in questo campo. Il Framework per l’Interazione Gestuale Tangibile fornisce supporto su tre livelli. Per prima cosa, aiuta a riflettere da un punto di vista teorico sui diversi tipi di gesti tangibili che possono essere eseguiti fisicamente, grazie a una tassonomia basata su tre componenti (muovere, tenere, toccare) e attributi addizionali, e che possono essere concepiti semanticamente, grazie a una tassonomia di tutti i costrutti semantici che permettono di associare dei significati ai gesti tangibili. In secondo luogo, il framework proposto aiuta a concepire nuovi sistemi interattivi basati su gesti tangibili e a ideare nuove interazioni basate su gesti con gli oggetti, attraverso linee guida per la definizione di gesti tangibili e una selezione delle migliore pratiche per i differenti campi di applicazione. Infine, il framework aiuta a implementare nuovi sistemi interattivi basati su gesti tangibili, permettendo di scegliere tra quattro differenti approcci tecnologici (incarnato e integrato negli oggetti, indossabile, distribuito nell’ambiente, o ibrido) e fornendo una guida generale per la scelta tra questi differenti approcci. Come applicazione di questo framework, questa tesi presenta anche sette sistemi interattivi basati su gesti tangibili, realizzati per tre differenti campi di applicazione: l’interazione con i sistemi di infotainment degli autoveicoli, la comunicazione interpersonale delle emozioni, e l’interazione nella casa intelligente. Per il primo campo di applicazione, sono stati progettati, sviluppati e testati quattro differenti sistemi che usano gesti tangibili effettuati sul volante come modalità di interazione con il sistema di infotainment. Per il secondo campo di applicazione, è stata concepita e sviluppata una lampada antropomorfica in grado di riconoscere i gesti tipici dell’interazione interpersonale. Per lo stesso campo di applicazione, un secondo sistema, basato su una maglietta intelligente, riconosce quando due persone si abbracciano e ricompensa questo gesto con uno scambio di informazioni digitali. Infine, per l’interazione nella casa intelligente, è stata investigata la realizzazione di uno smart watch per il riconoscimento di gesti eseguiti con oggetti tenuti nella mano. L’analisi dei sistemi interattivi esistenti basati su gesti tangibili permette di dimostrare che il framework ha un buon potere descrittivo e valutativo. Le applicazioni sviluppate durante la tesi mostrano che il framework proposto ha anche un valido potere generativo

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Enhancing Children’s Experience with Recommendation Systems

Recommender Systems (RSs) offer a personalized support in exploring large amounts of information, assisting users in decision making about products matching their taste and preferences. Most of the research todate on recommender systems have focused on traditional users, i.e., adult individuals who are able to offer explicit feedback, write reviews, or purchase items themselves. However, children's patterns of attention and interaction are quite different from those of adults. This paper presents the first results of a research-in-progress that can be suited to bridge the barrier between children and a recom-mender system by providing a child-friendly interaction paradigm. Specifically, a web application is developed that employs real-time object recognition on movie thumbnails or DVD cover-photos in a real-time manner. The tangible object can be manipulated by the user and provide input to the system for the purpose of generating movie recommendations. We plan to extend this work to the scenario where the child could ask for a video content showing a related toy (e.g., a car, a plane, the doll of a character that she likes in a cartoon) and the system could generate the videos that matches these implicit preferences expressed by the chil