Integrating 2D Mouse Emulation with 3D Manipulation for Visualizations on a Multi-Touch Table

We present the Rizzo, a multi-touch virtual mouse that has been designed to provide the fine grained interaction for information visualization on a multi-touch table. Our solution enables touch interaction for existing mouse-based visualizations. Previously, this transition to a multi-touch environment was difficult because the mouse emulation of touch surfaces is often insufficient to provide full information visualization functionality. We present a unified design, combining many Rizzos that have been designed not only to provide mouse capabilities but also to act as zoomable lenses that make precise information access feasible. The Rizzos and the information visualizations all exist within a touch-enabled 3D window management system. Our approach permits touch interaction with both the 3D windowing environment as well as with the contents of the individual windows contained therein. We describe an implementation of our technique that augments the VisLink 3D visualization environment to demonstrate how to enable multi-touch capabilities on all visualizations written with the popular prefuse visualization toolkit.

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

A Virtual Conversational Agent for Teens with Autism: Experimental Results and Design Lessons

We present the design of an online social skills development interface for teenagers with autism spectrum disorder (ASD). The interface is intended to enable private conversation practice anywhere, anytime using a web-browser. Users converse informally with a virtual agent, receiving feedback on nonverbal cues in real-time, and summary feedback. The prototype was developed in consultation with an expert UX designer, two psychologists, and a pediatrician. Using the data from 47 individuals, feedback and dialogue generation were automated using a hidden Markov model and a schema-driven dialogue manager capable of handling multi-topic conversations. We conducted a study with nine high-functioning ASD teenagers. Through a thematic analysis of post-experiment interviews, identified several key design considerations, notably: 1) Users should be fully briefed at the outset about the purpose and limitations of the system, to avoid unrealistic expectations. 2) An interface should incorporate positive acknowledgment of behavior change. 3) Realistic appearance of a virtual agent and responsiveness are important in engaging users. 4) Conversation personalization, for instance in prompting laconic users for more input and reciprocal questions, would help the teenagers engage for longer terms and increase the system's utility

Creative idea exploration within the structure of a guiding framework : the card brainstorming game

I present a card brainstorming exercise that transforms a conceptual tangible interaction framework into a tool for creative dialogue and discuss the experiences made in using it. Ten sessions with this card game demonstrate the frameworks' versatility and utility. Observation and participant feedback highlight the value of a provocative question format and of the metaphor of a card game

Development platform for elderly-oriented tabletop games

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

Kosketuskäyttöliittymän toteuttaminen olemassa olevaan ohjelmaan

The purpose of this work was to evaluate the migration steps of a windowing desktop application into a touch based input enabled software. The study was conducted on an already existing building information modelling software called Tekla BIMsight. The task was to retain all the functionality already in the software while making it possible to be used on touch-enabled devices, such as tablets or convertible laptops with a swivel display. Design and implementation of the system has been documented as part of the thesis, as well as most problematic issues during this period. The effects of the implementation are validated and tested with real users and the results from that study were documented. The usability study was conducted to obtain quantitative and qualitative metrics of the usability. The nature of the input mechanism, direct or indirect, affects the user experience greatly. The final system should be as responsive as possible to maintain a good level of perceived performance. Early prototyping and access to the target devices is critical to the success of a migration process. There are several common mistakes that should be avoided in the design and implementation phases. Not all the problems were critical, but many of them were identified as very cumbersome for the user that would affect the positive user experience of the software. With each new context for a user interface the problems need to be solved again and only experience from such solutions can help alleviate this task. The implemented touch support can be verified to meet the set requirements very well: It allows the system to be used on touch based input environments and all the major user interface elements support this.Työn tarkoituksena oli toteuttaa ja arvioida toimenpiteet ja. menetelmät joilla olemassa olevaan käyttöliittymään voidaan lisätä tuki kosketuskäytölle. Ominaisuudet lisättiin rakennusten tietomallinnuksen tarkasteluohjelmaan, Tekla BIMsight. Tehtävänä oli säilyttää kaikki aiemmat toiminnot ja tehdä ohjelmasta tehokkaasti käytettävä kosketuslaitteilla, kuten tableteilla ja kääntyvällä näytöllä varustetuilla kannettavilla. Suunnittelu ja toteutus järjestelmälle on dokumentoitu työssä ja kaikkein vaativimmat ongelmat. Toteutetun tuen vaikutuksia arvioitiin oikeiden käyttäjien kanssa tehdyssä käyttäjätutkimuksessa, jonka tulokset on esitetty. Käytettävyystutkimuksella hankittiin kvantitatiivista ja kvalitatiivista tietoa tuotteesta. Laite jolla ohjelmistoa käytetään vaikuttaa ohjelmasta saatuun käyttökokemukseen merkittävästi. Hyvän käyttökokemuksen saavuttamiseksi lopullisen järjestelmän käytön tulisi olla sujuvaa. Aikaisten prototyyppien kokeilu ja kohdelaitteiden saatavuus ovat tärkeitä tekijöitä siirtymäprosessin kannalta. Yleisiä ongelmatilanteita ja haasteita joita kohdattiin suunnittelu- ja toteutusvaiheissa on listattu työssä. Loppukäyttäjän kannalta useat ongelmat olivat rasittavia ja vaikuttaisivat käyttökokemukseen negatiivisesti jos niitä ei korjata. Uuden käyttöympäristön tuomat ongelmat joudutaan ratkaisemaan aina uudestaan. Vain kokemuksella vastaavista tilanteista on merkittävästi etua itse ratkaisujen löytämiselle. Toteutetun kosketuskäyttöliittymän tuen voidaan todeta vastaavan sille asetettuja tavoitteita ja vaatimuksia hyvin; se mahdollistaa ohjelman käyttämisen kosketuskäyttöliittymän omaavissa laitteissa ja kaikkein merkittävimmät käyttöliittymäelementit on tuettuina

Designing for a Repository of Virtual Crisis Management Tabletop Exercises – Lessons Learned from a Scandinavian R&D Project

Crisis training exercises play a vital role in preparing local and regional governments for the management of crises and disasters. Unfortunately, conducting sufficient training is demanding, especially in small municipalities, due to constrained time and personnel resources, but also complex planning and scheduling of the dominant on-site training methods. Virtual training has been suggested as a resource-efficient and flexible complement. However, despite numerous specifications of digital technology for training, research on organisational implementation and usage is lacking, indicating a low uptake. This article presents a cross-border R&D effort to facilitate the digitalisation of crisis management training by developing generic virtual tabletop exercises (VTTXs) to be shared via a repository, and (re-)used in, and adapted for, diverse contexts. The purpose of this article is to identify essential aspects in designing and conducting virtual tabletop exercises (VTTXs) for collaborative crisis management training

TangiWheel: A widget for manipulating collections on tabletop displays supporting hybrid Input modality

In this paper we present TangiWheel, a collection manipulation widget for tabletop displays. Our implementation is flexible, allowing either multi-touch or interaction, or even a hybrid scheme to better suit user choice and convenience. Different TangiWheel aspects and features are compared with other existing widgets for collection manipulation. The study reveals that TangiWheel is the first proposal to support a hybrid input modality with large resemblance levels between touch and tangible interaction styles. Several experiments were conducted to evaluate the techniques used in each input scheme for a better understanding of tangible surface interfaces in complex tasks performed by a single user (e.g., involving a typical master-slave exploration pattern). The results show that tangibles perform significantly better than fingers, despite dealing with a greater number of interactions, in situations that require a large number of acquisitions and basic manipulation tasks such as establishing location and orientation. However, when users have to perform multiple exploration and selection operations that do not require previous basic manipulation tasks, for instance when collections are fixed in the interface layout, touch input is significantly better in terms of required time and number of actions. Finally, when a more elastic collection layout or more complex additional insertion or displacement operations are needed, the hybrid and tangible approaches clearly outperform finger-based interactions.. ©2012 Springer Science+Business Media, LLC & Science Press, ChinaThe work is supported by the Ministry of Education of Spain under Grant No. TSI2010-20488. Alejandro Catala is supported by an FPU fellowship for pre-doctoral research staff training granted by the Ministry of Education of Spain with reference AP2006-00181.Catalá Bolós, A.; García Sanjuan, F.; Jaén Martínez, FJ.; Mocholi Agües, JA. (2012). TangiWheel: A widget for manipulating collections on tabletop displays supporting hybrid Input modality. Journal of Computer Science and Technology. 27(4):811-829. doi:10.1007/s11390-012-1266-4S811829274Jordà S, Geiger G, Alonso M, Kaltenbrunner M. The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces. In Proc. TEI 2007, Baton Rouge, LA, USA, Feb. 15-17, 2007, pp.139–146.Vandoren P, van Laerhoven T, Claesen L, Taelman J, Raymaekers C, van Reeth F. IntuPaint: Bridging the gap between physical and digital painting. In Proc. TABLETOP2008, Amterdam, the Netherlands, Oct. 1-3, 2008, pp.65–72.Schöning J, Hecht B, Raubal M, Krüger A, Marsh M, Rohs M. Improving interaction with virtual globes through spatial thinking: Helping users ask “why?”. In Proc. IUI 2008, Canary Islans, Spain, Jan. 13-16, 2008, pp.129–138.Fitzmaurice GW, BuxtonW. An empirical evaluation of graspable user interfaces: Towards specialized, space-multiplexed input. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.43–50.Tuddenham P, Kirk D, Izadi S. Graspables revisited: Multitouch vs. tangible input for tabletop displays in acquisition and manipulation tasks. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.2223–2232.Lucchi A, Jermann P, Zufferey G, Dillenbourg P. An empirical evaluation of touch and tangible interfaces for tabletop displays. In Proc. TEI 2010, Cambridge, USA, Jan. 25-27, 2010, pp.177–184.Fitzmaurice G W, Ishii H, Buxton W. Bricks: Laying the foundations for graspable user interfaces. In Proc. CHI 1995, Denver, USA, May 7-11, 1995, pp.442–449.Ishii H, Ullmer B. Tangible bits: Towards seamless interfaces between people, bits and atoms. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.234–241.Ullmer B, Ishii H, Glas D. mediaBlocks: Physical containers, transports, and controls for online media. In Proc. SIGGRAPH1998, Orlando, USA, July 19-24, 1998, pp.379–386.Shen C, Hancock M S, Forlines C, Vernier F D. CoR2Ds: Context-rooted rotatable draggables for tabletop interaction. In Proc. CHI 2005, Portland, USA, April 2-7, 2005, pp.1781–1784.Lepinski G J, Grossman T, Fitzmaurice G. The design and evaluation of multitouch marking menus. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.2233–2242.Accot J, Zhai S. Beyond Fitts’ law: Models for trajectorybased HCI tasks. In Proc. CHI 1997, Atlanta, USA, March 22-27, 1997, pp.295–302.Song H, Kim B, Lee B, Seo J. A comparative evaluation on tree visualization methods for hierarchical structures with large fan-outs. In Proc. CHI 2010, Atlanta, USA, April 10-15, 2010, pp.223–232.Bailly G, Lecolinet E, Nigay L. Wave menus: Improving the novice mode of hierarchical marking menus. In Proc. INTERACT2007, Río de Janeiro, Brazil, Sept. 10-14, 2007, pp.475–488.Zhao S, Agrawala M, Hinckley K. Zone and polygon menus: Using relative position to increase the breadth of multi-stroke marking menus. In Proc. CHI 2006, Montreal, Canada, April 24-27, 2006, pp.1077–1086.Patten J, Recht B, Ishii H. Interaction techniques for musical performance with tabletop tangible interfaces. In Proc. ACE2006, Hollywood, USA, Jun. 14-16, 2006, Article No.27.Weiss M, Wagner J, Jansen Y, Jennings R, Khoshabeh R, Hollan J D, Borchers J. SLAP widgets: Bridging the gap between virtual and physical controls on tabletops. In Proc. CHI 2009, Boston, USA, April 4-9, 2009, pp.481–490.Hancock M, Hilliges O, Collins C, Baur D, Carpendale S. Exploring tangible and direct touch interfaces for manipulating 2D and 3D information on a digital table. In Proc. ITS 2009, Banff, Canada, Nov. 23-25, pp.77–84.Hilliges O, Baur D, Butz A. Photohelix: Browsing, sorting and sharing digital photo collections. In Proc. Horizontal Interactive Human-Computer Systems (TABLETOP2007), Newport, Rhode Island, USA, Oct. 10-12, 2007, pp.87–94.Hesselmann T, Flöring S, Schmidt M. Stacked half-Pie menus: Navigating nested menus on interactive tabletops. In Proc. ITS 2009, Banff, Canada, Nov. 23-25, 2009, pp.173–180.Gallardo D, Jordà S. Tangible jukebox: Back to palpable music. In Proc. TEI 2010, Boston, USA, Jan. 25-27, 2010, pp.199–202.Fishkin K. A taxonomy for and analysis of tangible interfaces. Personal and Ubiquitous Computing, 2004, 8(5): 347–358.Catala A, Jaen J, Martinez-Villaronga A A, Mocholi J A. AGORAS: Exploring creative learning on tangible user interfaces. In Proc. COMPSAC 2011, Munich, Germany, July 18-22, 2011, pp.326–335.Catala A, Garcia-Sanjuan F, Azorin J, Jaen J, Mocholi J A. Exploring direct communication and manipulation on interactive surfaces to foster novelty in a creative learning environment. IJCSRA, 2012, 2(1): 15–24.Catala A, Jaen J, van Dijk B, Jord`a S. Exploring tabletops as an effective tool to foster creativity traits. In Proc. TEI 2012, Kingston, Canada, Feb. 19-22, 2012, pp.143–150.Hopkins D. Directional selection is easy as pie menus. In: The Usenix Association Newsletter, 1987, 12(5): 103.Microsoft Surface User Experience Guidelines. http://msdn.microsoft.com/en-us/library/ff318692.aspx , May 2011.Maydak M, Stromer R, Mackay H A, Stoddard L T. Stimulus classes in matching to sample and sequence production: The emergence of numeric relations. Research in Developmental Disabilities, 1995, 16(3): 179–204

Supporting Collaborative Multi-User Interactions in a Video Surveillance Application Using Microsoft Tabletop Surface

This report examines the progressing exploration done on my chosen subject, which is A Multi-touch Interface in a Video Surveillance System. It discusses method of early prototype interacting with security surveillance footage using natural user interfaces instead of traditional mouse and keyboard interaction. Current project is an evidence of idea on exhibiting that multi-touch interfaces are helpful in a video surveillance system to specifically control the surveillance videos, both of the live or of recorded. In case of any occurrence, this proposed system of interaction may require the user to spend an extra time amounts time obtaining circumstantial and location awareness, which is counter-beneficial. The framework proposed in this paper show how a multi-touch screen and natural interaction can empower the surveillance observing station users to rapidly recognize the area of a security camera and proficiently react to an occurrence. One of the main objective of this project is to engage more than 1 user to perform moving, scaling, rotating ,highlighting and recording on a surveillance video in the meantime, particularly during emergency periods. Furthermore, the scope of study for this project is to improve user collaborative interactions on Microsoft tabletop surface .A methodology was developed based upon a combination of the available literature and the experiences of the authors, who are actively involved with the development of multi-user interactions. This will cover many parts such as surveys, data gathering from respective Subject-Matter Experts, focal points, and analyzing information. It is intended to have a Surveillance application with user friendly collaborative touch on surface and eye-catching interface to reflect the quick paced nature of today's correspondences and better advertise its new activities and accessible assets. The future improvements and plans have been recommended and discussed in the recommendations section. Up to now, this research report has been run for twelve (12) weeks and going to be continued running for sixteen (16) weeks with a specific end goal to attain project primary objectives