Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Interactive tabletops in education

Interactive tabletops are gaining increased attention from CSCL researchers. This paper analyses the relation between this technology and teaching and learning processes. At a global level, one could argue that tabletops convey a socio-constructivist flavor: they support small teams that solve problems by exploring multiple solutions. The development of tabletop applications also witnesses the growing importance of face-to-face collaboration in CSCL and acknowledges the physicality of learning. However, this global analysis is insufficient. To analyze the educational potential of tabletops in education, we present 33 points that should be taken into consideration. These points are structured on four levels: individual user-system interaction, teamwork, classroom orchestration, and socio-cultural contexts. God lies in the detail

TangiWheel: A widget for manipulating collections on tabletop displays supporting hybrid Input modality

In this paper we present TangiWheel, a collection manipulation widget for tabletop displays. Our implementation is flexible, allowing either multi-touch or interaction, or even a hybrid scheme to better suit user choice and convenience. Different TangiWheel aspects and features are compared with other existing widgets for collection manipulation. The study reveals that TangiWheel is the first proposal to support a hybrid input modality with large resemblance levels between touch and tangible interaction styles. Several experiments were conducted to evaluate the techniques used in each input scheme for a better understanding of tangible surface interfaces in complex tasks performed by a single user (e.g., involving a typical master-slave exploration pattern). The results show that tangibles perform significantly better than fingers, despite dealing with a greater number of interactions, in situations that require a large number of acquisitions and basic manipulation tasks such as establishing location and orientation. However, when users have to perform multiple exploration and selection operations that do not require previous basic manipulation tasks, for instance when collections are fixed in the interface layout, touch input is significantly better in terms of required time and number of actions. Finally, when a more elastic collection layout or more complex additional insertion or displacement operations are needed, the hybrid and tangible approaches clearly outperform finger-based interactions.. ©2012 Springer Science+Business Media, LLC & Science Press, ChinaThe work is supported by the Ministry of Education of Spain under Grant No. TSI2010-20488. Alejandro Catala is supported by an FPU fellowship for pre-doctoral research staff training granted by the Ministry of Education of Spain with reference AP2006-00181.Catalá Bolós, A.; García Sanjuan, F.; Jaén Martínez, FJ.; Mocholi Agües, JA. (2012). TangiWheel: A widget for manipulating collections on tabletop displays supporting hybrid Input modality. Journal of Computer Science and Technology. 27(4):811-829. doi:10.1007/s11390-012-1266-4S811829274Jordà S, Geiger G, Alonso M, Kaltenbrunner M. The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces. In Proc. TEI 2007, Baton Rouge, LA, USA, Feb. 15-17, 2007, pp.139–146.Vandoren P, van Laerhoven T, Claesen L, Taelman J, Raymaekers C, van Reeth F. IntuPaint: Bridging the gap between physical and digital painting. In Proc. TABLETOP2008, Amterdam, the Netherlands, Oct. 1-3, 2008, pp.65–72.Schöning J, Hecht B, Raubal M, Krüger A, Marsh M, Rohs M. Improving interaction with virtual globes through spatial thinking: Helping users ask “why?”. In Proc. IUI 2008, Canary Islans, Spain, Jan. 13-16, 2008, pp.129–138.Fitzmaurice GW, BuxtonW. An empirical evaluation of graspable user interfaces: Towards specialized, space-multiplexed input. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.43–50.Tuddenham P, Kirk D, Izadi S. Graspables revisited: Multitouch vs. tangible input for tabletop displays in acquisition and manipulation tasks. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.2223–2232.Lucchi A, Jermann P, Zufferey G, Dillenbourg P. An empirical evaluation of touch and tangible interfaces for tabletop displays. In Proc. TEI 2010, Cambridge, USA, Jan. 25-27, 2010, pp.177–184.Fitzmaurice G W, Ishii H, Buxton W. Bricks: Laying the foundations for graspable user interfaces. In Proc. CHI 1995, Denver, USA, May 7-11, 1995, pp.442–449.Ishii H, Ullmer B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.234–241.Ullmer B, Ishii H, Glas D. mediaBlocks: Physical containers, transports, and controls for online media. In Proc. SIGGRAPH1998, Orlando, USA, July 19-24, 1998, pp.379–386.Shen C, Hancock M S, Forlines C, Vernier F D. CoR2Ds: Context-rooted rotatable draggables for tabletop interaction. In Proc. CHI 2005, Portland, USA, April 2-7, 2005, pp.1781–1784.Lepinski G J, Grossman T, Fitzmaurice G. The design and evaluation of multitouch marking menus. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.2233–2242.Accot J, Zhai S. Beyond Fitts’ law: Models for trajectorybased HCI tasks. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.295–302.Song H, Kim B, Lee B, Seo J. A comparative evaluation on tree visualization methods for hierarchical structures with large fan-outs. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.223–232.Bailly G, Lecolinet E, Nigay L. Wave menus: Improving the novice mode of hierarchical marking menus. In Proc. INTERACT2007, Río de Janeiro, Brazil, Sept. 10-14, 2007, pp.475–488.Zhao S, Agrawala M, Hinckley K. Zone and polygon menus: Using relative position to increase the breadth of multi-stroke marking menus. In Proc. CHI 2006, Montreal, Canada, April 24-27, 2006, pp.1077–1086.Patten J, Recht B, Ishii H. Interaction techniques for musical performance with tabletop tangible interfaces. In Proc. ACE2006, Hollywood, USA, Jun. 14-16, 2006, Article No.27.Weiss M, Wagner J, Jansen Y, Jennings R, Khoshabeh R, Hollan J D, Borchers J. SLAP widgets: Bridging the gap between virtual and physical controls on tabletops. In Proc. CHI 2009, Boston, USA, April 4-9, 2009, pp.481–490.Hancock M, Hilliges O, Collins C, Baur D, Carpendale S. Exploring tangible and direct touch interfaces for manipulating 2D and 3D information on a digital table. In Proc. ITS 2009, Banff, Canada, Nov. 23-25, pp.77–84.Hilliges O, Baur D, Butz A. Photohelix: Browsing, sorting and sharing digital photo collections. In Proc. Horizontal Interactive Human-Computer Systems (TABLETOP2007), Newport, Rhode Island, USA, Oct. 10-12, 2007, pp.87–94.Hesselmann T, Flöring S, Schmidt M. Stacked half-Pie menus: Navigating nested menus on interactive tabletops. In Proc. ITS 2009, Banff, Canada, Nov. 23-25, 2009, pp.173–180.Gallardo D, Jordà S. Tangible jukebox: Back to palpable music. In Proc. TEI 2010, Boston, USA, Jan. 25-27, 2010, pp.199–202.Fishkin K. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing, 2004, 8(5): 347–358.Catala A, Jaen J, Martinez-Villaronga A A, Mocholi J A. AGORAS: Exploring creative learning on tangible user interfaces. In Proc. COMPSAC 2011, Munich, Germany, July 18-22, 2011, pp.326–335.Catala A, Garcia-Sanjuan F, Azorin J, Jaen J, Mocholi J A. Exploring direct communication and manipulation on interactive surfaces to foster novelty in a creative learning environment. IJCSRA, 2012, 2(1): 15–24.Catala A, Jaen J, van Dijk B, Jord`a S. Exploring tabletops as an effective tool to foster creativity traits. In Proc. TEI 2012, Kingston, Canada, Feb. 19-22, 2012, pp.143–150.Hopkins D. Directional selection is easy as pie menus. In: The Usenix Association Newsletter, 1987, 12(5): 103.Microsoft Surface User Experience Guidelines. http://msdn.microsoft.com/en-us/library/ff318692.aspx , May 2011.Maydak M, Stromer R, Mackay H A, Stoddard L T. Stimulus classes in matching to sample and sequence production: The emergence of numeric relations. Research in Developmental Disabilities, 1995, 16(3): 179–204

Light on horizontal interactive surfaces: Input space for tabletop computing

In the last 25 years we have witnessed the rise and growth of interactive tabletop research, both in academic and in industrial settings. The rising demand for the digital support of human activities motivated the need to bring computational power to table surfaces. In this article, we review the state of the art of tabletop computing, highlighting core aspects that frame the input space of interactive tabletops: (a) developments in hardware technologies that have caused the proliferation of interactive horizontal surfaces and (b) issues related to new classes of interaction modalities (multitouch, tangible, and touchless). A classification is presented that aims to give a detailed view of the current development of this research area and define opportunities and challenges for novel touch- and gesture-based interactions between the human and the surrounding computational environment. © 2014 ACM.This work has been funded by Integra (Amper Sistemas and CDTI, Spanish Ministry of Science and Innovation) and TIPEx (TIN2010-19859-C03-01) projects and Programa de Becas y Ayudas para la Realización de Estudios Oficiales de Máster y Doctorado en la Universidad Carlos III de Madrid, 2010

Interactive Visualization Lenses:: Natural Magic Lens Interaction for Graph Visualization

Information visualization is an important research field concerned with making sense and inferring knowledge from data collections. Graph visualizations are specific techniques for data representation relevant in diverse application domains among them biology, software-engineering, and business finance. These data visualizations benefit from the display space provided by novel interactive large display environments. However, these environments also cause new challenges and result in new requirements regarding the need for interaction beyond the desktop and according redesign of analysis tools. This thesis focuses on interactive magic lenses, specialized locally applied tools that temporarily manipulate the visualization. These may include magnification of focus regions but also more graph-specific functions such as pulling in neighboring nodes or locally reducing edge clutter. Up to now, these lenses have mostly been used as single-user, single-purpose tools operated by mouse and keyboard. This dissertation presents the extension of magic lenses both in terms of function as well as interaction for large vertical displays. In particular, this thesis contributes several natural interaction designs with magic lenses for the exploration of graph data in node-link visualizations using diverse interaction modalities. This development incorporates flexible switches between lens functions, adjustment of individual lens properties and function parameters, as well as the combination of lenses. It proposes interaction techniques for fluent multi-touch manipulation of lenses, controlling lenses using mobile devices in front of large displays, and a novel concept of body-controlled magic lenses. Functional extensions in addition to these interaction techniques convert the lenses to user-configurable, personal territories with use of alternative interaction styles. To create the foundation for this extension, the dissertation incorporates a comprehensive design space of magic lenses, their function, parameters, and interactions. Additionally, it provides a discussion on increased embodiment in tool and controller design, contributing insights into user position and movement in front of large vertical displays as a result of empirical investigations and evaluations.Informationsvisualisierung ist ein wichtiges Forschungsfeld, das das Analysieren von Daten unterstützt. Graph-Visualisierungen sind dabei eine spezielle Variante der Datenrepräsentation, deren Nutzen in vielerlei Anwendungsfällen zum Einsatz kommt, u.a. in der Biologie, Softwareentwicklung und Finanzwirtschaft. Diese Datendarstellungen profitieren besonders von großen Displays in neuen Displayumgebungen. Jedoch bringen diese Umgebungen auch neue Herausforderungen mit sich und stellen Anforderungen an Nutzerschnittstellen jenseits der traditionellen Ansätze, die dadurch auch Anpassungen von Analysewerkzeugen erfordern. Diese Dissertation befasst sich mit interaktiven „Magischen Linsen“, spezielle lokal-angewandte Werkzeuge, die temporär die Visualisierung zur Analyse manipulieren. Dabei existieren zum Beispiel Vergrößerungslinsen, aber auch Graph-spezifische Manipulationen, wie das Anziehen von Nachbarknoten oder das Reduzieren von Kantenüberlappungen im lokalen Bereich. Bisher wurden diese Linsen vor allem als Werkzeug für einzelne Nutzer mit sehr spezialisiertem Effekt eingesetzt und per Maus und Tastatur bedient. Die vorliegende Doktorarbeit präsentiert die Erweiterung dieser magischen Linsen, sowohl in Bezug auf die Funktionalität als auch für die Interaktion an großen, vertikalen Displays. Insbesondere trägt diese Dissertation dazu bei, die Exploration von Graphen mit magischen Linsen durch natürliche Interaktion mit unterschiedlichen Modalitäten zu unterstützen. Dabei werden flexible Änderungen der Linsenfunktion, Anpassungen von individuellen Linseneigenschaften und Funktionsparametern, sowie die Kombination unterschiedlicher Linsen ermöglicht. Es werden Interaktionstechniken für die natürliche Manipulation der Linsen durch Multitouch-Interaktion, sowie das Kontrollieren von Linsen durch Mobilgeräte vor einer Displaywand vorgestellt. Außerdem wurde ein neuartiges Konzept körpergesteuerter magischer Linsen entwickelt. Funktionale Erweiterungen in Kombination mit diesen Interaktionskonzepten machen die Linse zu einem vom Nutzer einstellbaren, persönlichen Arbeitsbereich, der zudem alternative Interaktionsstile erlaubt. Als Grundlage für diese Erweiterungen stellt die Dissertation eine umfangreiche analytische Kategorisierung bisheriger Forschungsarbeiten zu magischen Linsen vor, in der Funktionen, Parameter und Interaktion mit Linsen eingeordnet werden. Zusätzlich macht die Arbeit Vor- und Nachteile körpernaher Interaktion für Werkzeuge bzw. ihre Steuerung zum Thema und diskutiert dabei Nutzerposition und -bewegung an großen Displaywänden belegt durch empirische Nutzerstudien

Interactive tabletops in education

Interactive tabletops are gaining increased attention from CSCL researchers. This paper analyses the relation between this technology and teaching and learning processes. At a global level, one could argue that tabletops convey a socio-constructivist flavor: they support small teams that solve problems by exploring multiple solutions. The development of tabletop applications also witnesses the growing importance of face-to-face collaboration in CSCL and acknowledges the physicality of learning. However, this global analysis is insufficient. To analyze the educational potential of tabletops in education, we present 33 points that should be taken into consideration. These points are structured on four levels: individual user-system interaction, teamwork, classroom orchestration, and socio-cultural contexts. God lies in the details

GAINE - A Portable Framework for the Development of Edutainment Applications Based on Multitouch and Tangible Interaction

In the last few years, Multitouch and Tangible User Interfaces have emerged as a powerful tool to integrate interactive surfaces and responsive spaces that embody digital information. Besides providing a natural interaction with digital contents, they allow the interaction of multiple users at the same time, thus promoting collaborative activities and information sharing. In particular, these characteristics have opened new exploration possibilities in the edutainment context, as witnessed by the many applications successfully developed in different areas, from children’s collaborative learning to interactive storytelling, cultural heritage and medical therapy support. However, due to the availability of different multitouch and tangible interaction technologies and of different target computing platforms, the development and deployment of such applications can be challenging. To this end, in this paper we present GAINE (tanGible Augmented INteraction for Edutainment), a software framework that enables rapid prototyping and development of tangible augmented applications for edutainment purposes. GAINE has two main features. First, it offers developers high-level context specific constructs that significantly reduces the implementation burden. Second, the framework is portable on different operating systems and offers independence from the underlying hardware and tracking technology. In this paper, we also discuss several case studies to show the effectiveness of GAINE in simplifying the development of entertainment and edutainment applications based on multitouch and tangible interaction

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with

GAINE - tanGible Augmented INteraction for Edutainment

Interactive tabletops are gaining an increasing interest since they provide a more natural interaction with digital contents and allow the interaction of multiple users at a time promoting face-to-face collaboration, information sharing and the raise of social experiences. Given the potentialities offered by these devices, several entertainment-edutainment applications based on interactive tabletops have been successfully developed in different areas, from medical therapy support to children’s collaborative learning, interactive storytelling and cultural heritage. However, the development of such applications often requires complex technical and implementation skills. Taking this into consideration, in this paper we present GAINE (tanGible Augmented INteraction for Edutainment), a software framework aimed at the rapid prototyping and development of interactive tabletop games. GAINE offers developers context specific high-level constructs and a simple scripting language that simplifies the implementation task. The framework is portable on different operating systems and offers independence from the underlying hardware. Two practical case studies are thoroughly discussed to show how GAINE can simplify the development of interactive tabletop applications in the entertainment and edutainment contexts