Light on horizontal interactive surfaces: Input space for tabletop computing

In the last 25 years we have witnessed the rise and growth of interactive tabletop research, both in academic and in industrial settings. The rising demand for the digital support of human activities motivated the need to bring computational power to table surfaces. In this article, we review the state of the art of tabletop computing, highlighting core aspects that frame the input space of interactive tabletops: (a) developments in hardware technologies that have caused the proliferation of interactive horizontal surfaces and (b) issues related to new classes of interaction modalities (multitouch, tangible, and touchless). A classification is presented that aims to give a detailed view of the current development of this research area and define opportunities and challenges for novel touch- and gesture-based interactions between the human and the surrounding computational environment. © 2014 ACM.This work has been funded by Integra (Amper Sistemas and CDTI, Spanish Ministry of Science and Innovation) and TIPEx (TIN2010-19859-C03-01) projects and Programa de Becas y Ayudas para la Realización de Estudios Oficiales de Máster y Doctorado en la Universidad Carlos III de Madrid, 2010

The role of personal and shared displays in scripted collaborative learning

Over the last decades collaborative learning has gained immensely in importance and popularity due to its high potential. Unfortunately, learners rarely engage in effective learning activities unless they are provided with instructional support. In order to maximize learning outcomes it is therefore advisable to structure collaborative learning sessions. One way of doing this is using collaboration scripts, which define a sequence of activities to be carried out by the learners. The field of computer-supported collaborative learning (CSCL) produced a variety of collaboration scripts that proved to have positive effects on learning outcomes. These scripts provide detailed descriptions of successful learning scenarios and are therefore used as foundation for this thesis. In many cases computers are used to support collaborative learning. Traditional personal computers are often chosen for this purpose. However, during the last decades new technologies have emerged, which seem to be better suited for co-located collaboration than personal computers. Large interactive displays, for example, allow a number of people to work simultaneously on the same surface while being highly aware of the co-learners' actions. There are also multi-display environments that provide several workspaces, some of which may be shared, others may be personal. However, there is a lack of knowledge regarding the influence of different display types on group processes. For instance, it remains unclear in which cases shareable user interfaces should replace traditional single-user devices and when both personal and shared workspaces should be provided. This dissertation therefore explores the role of personal and shared workspaces in various situations in the area of collaborative learning. The research questions include the choice of technological devices, the seating arrangement as well as how user interfaces can be designed to guide learners. To investigate these questions a two-fold approach was chosen. First, a framework was developed, which supports the implementation of scripted collaborative learning applications. Second, different prototypes were implemented to explore the research questions. Each prototype is based on at least one collaboration script. The result is a set of studies, which contribute to answering the above-mentioned research questions. With regard to the choice of display environment the studies showed several reasons for integrating personal devices such as laptops. Pure tabletop applications with around-the-table seating arrangements whose benefits for collaboration are widely discussed in the relevant literature revealed severe drawbacks for text-based learning activities. The combination of laptops and an interactive wall display, on the other hand, turned out to be a suitable display environment for collaborative learning in several cases. In addition, the thesis presents several ways of designing the user interface in a way that guides learners through collaboration scripts

Cross-display attention switching in mobile interaction with large displays

Mobile devices equipped with features (e.g., camera, network connectivity and media player) are increasingly being used for different tasks such as web browsing, document reading and photography. While the portability of mobile devices makes them desirable for pervasive access to information, their small screen real-estate often imposes restrictions on the amount of information that can be displayed and manipulated on them. On the other hand, large displays have become commonplace in many outdoor as well as indoor environments. While they provide an efficient way of presenting and disseminating information, they provide little support for digital interactivity or physical accessibility. Researchers argue that mobile phones provide an efficient and portable way of interacting with large displays, and the latter can overcome the limitations of the small screens of mobile devices by providing a larger presentation and interaction space. However, distributing user interface (UI) elements across a mobile device and a large display can cause switching of visual attention and that may affect task performance. This thesis specifically explores how the switching of visual attention across a handheld mobile device and a vertical large display can affect a single user's task performance during mobile interaction with large displays. It introduces a taxonomy based on the factors associated with the visual arrangement of Multi Display User Interfaces (MDUIs) that can influence visual attention switching during interaction with MDUIs. It presents an empirical analysis of the effects of different distributions of input and output across mobile and large displays on the user's task performance, subjective workload and preference in the multiple-widget selection task, and in visual search tasks with maps, texts and photos. Experimental results show that the selection of multiple widgets replicated on the mobile device as well as on the large display, versus those shown only on the large display, is faster despite the cost of initial attention switching in the former. On the other hand, a hybrid UI configuration where the visual output is distributed across the mobile and large displays is the worst, or equivalent to the worst, configuration in all the visual search tasks. A mobile device-controlled large display configuration performs best in the map search task and equal to best (i.e., tied with a mobile-only configuration) in text- and photo-search tasks

Designing for Shareable Interfaces in the Wild

Despite excitement about the potential of interactive tabletops to support collaborative work, there have been few empirical demonstrations of their effectiveness (Marshall et al., 2011). In particular, while lab-based studies have explored the effects of individual design features, there has been a dearth of studies evaluating the success of systems in the wild. For this technology to be of value, designers and systems builders require a better understanding of how to develop and evaluate tabletop applications to be deployed in real world settings. This dissertation reports on two systems designed through a process that incorporated ethnography-style observations, iterative design and in the wild evaluation. The first study focused on collaborative learning in a medical setting. To address the fact that visitors to a hospital emergency ward were leaving with an incomplete understanding of their diagnosis and treatment, a system was prototyped in a working Emergency Room (ER) with doctors and patients. The system was found to be helpful but adoption issues hampered its impact. The second study focused on a planning application for visitors to a tourist information centre. Issues and opportunities for a successful, contextually-fitted system were addressed and it was found to be effective in supporting group planning activities by novice users, in particular, facilitating users’ first experiences, providing effective signage and offering assistance to guide the user through the application. This dissertation contributes to understanding of multi-user systems through literature review of tabletop systems, collaborative tasks, design frameworks and evaluation of prototypes. Some support was found for the claim that tabletops are a useful technology for collaboration, and several issues were discussed. Contributions to understanding in this field are delivered through design guidelines, heuristics, frameworks, and recommendations, in addition to the two case studies to help guide future tabletop system creators

Ideas mapping, surface computing and user experience

This paper reports work regarding the design, development and evaluation of a surface computing application to support collaborative decision making.. The domain- independent application, so called Ideas Mapping, builds on the principle of Affinity Diagramming to allow participants to analyze a problem and brainstorm around possible solutions while they actively construct a consensus artifact - - a taxonomy of their ideas. During idea generation, Ideas Mapping replicates physical post-it notes on a multi-touch tabletop. Additional functionality supports student collaboration and interaction around the organization of ideas into thematic categories associated with the problem at hand. We report on the functionality and user experience while interacting with the application which was designed and developed using a user-centered approach. We also report initial findings regarding the affordances of surface computing for collaborative decision making

Trapped in the Upside-Down: Mental Normalization of Data Visualizations

Der gleichberechtigte Zugang zu Informationen ist eine Voraussetzung für eine ausgewogene Teilnahme an kollaborativer Gruppenarbeit vor Ort. Bei der Arbeit mit Single-Display-Groupware, wie z. B. Multi-Touch Tischen, werden die Informationen unter einem bestimmten Blickwinkel angezeigt, wodurch die um den Tisch herum befindlichen Benutzer im Zugriff auf die Informationen benachteiligt werden, was die Effektivität der Gruppenarbeit beeinträchtigt. Die vorliegende Arbeit konzentriert sich auf die kognitiven Prozesse des einzelnen Nutzers. Sie stellt Untersuchungen zu den Auswirkungen der Kosten mentaler Normalisierung auf den Zugang zu Informationen vor, welche in rotierten Datenvisualisierungen präsentiert werden, und untersucht Gestaltungsoptionen zur Reduzierung dieser Kosten. Die Untersuchungen wurde in zwei Studienlinien organisiert. Die erste Studienlinie befasste sich mit der Bestätigung der Kosten der mentalen Normalisierung für gedrehte Balkendiagramme und der Erkundung möglicher Designoptionen zur Reduzierung dieser Kosten. Die zusätzliche kognitive Belastung durch mentale Normalisierungsprozesse wurde bestätigt. Die Reaktionszeiten für rotierte Diagramme waren signifikant größer als für Unrotierte. Eine getrennte Rotation von Diagrammbeschriftungen und Diagramminhalten ergab, dass die Beschriftungen ein wesentlicher Faktor für die zusätzlichen mentalen Normalisierungskosten sind. In der Folge konzentrierten sich die Interventionen zur Verringerung der zusätzlichen Belastung durch die mentale Normalisierung auf unterschiedliche Beschriftungsdesigns. Sowohl die Substitution der Beschriftungen durch Piktogramme als auch eine Farbcodierung der Beschriftungen erwiesen sich als wirksam, um die Antwortgeschwindigkeit zu erhöhen, wobei die Farbcodierung etwas wirksamer war als die Piktogrammbeschriftung. Darüber hinaus wurde die Verwendung redundanter, aber invertierter Beschriftungen auf ihre Wirksamkeit getestet. Während die ersten Ergebnisse vielversprechend schienen, war die Anwendung auf Datenvisualisierungen von begrenztem Nutzen. Die Anzeige von doppelten Beschriftungen erhöhte die Gesamtreaktionszeit für alle Rotationsbedingungen, verringerte aber den Unterschied zwischen ihnen. Wir nannten dies den ”Justice Effect”. In einer zweiten Studienreihe wurde der mögliche Nutzen der mentalen Normalisierung für das Langzeitgedächtnis untersucht. Die Ergebnisse zeigten keinen solchen Nutzen und wiesen auf mögliche negative Effekte hin. Insgesamt bietet diese Arbeit neue Einblicke in die Auswirkungen der mentalen Normalisierung auf den Zugang zu Informationen, die in Datenvisualisierungen in einer kollaborativen Arbeitsumgebung dargestellt werden. Sie zeigt auch mögliche Interventionen auf, um diese Kosten zu reduzieren und so die Zugänglichkeit der Informationen für alle Nutzer, unabhängig von ihrem Standpunkt, zu nivellieren.Equal access to information is a prerequisite for equal participation in collaborative co-located group work settings. Working on Single Display Groupware, such as Multi-Touch Tables, information is displayed with a specific point of view in mind. This puts the users located around the table on a disadvantage in accessing the information, thus reducing the effectiveness of the group work setting. This thesis focusses on the cognitive processes of the individual user. It presents research on the impact of mental normalization costs on the access to information presented in rotated data visualizations and explores design options to reduce these costs. The research was organized in two lines of study. The first line of study was concerned with the confirmation of the cost of mental normalization for rotated bar graphs and exploring possible design options to reduce these costs. The additional burden of mental normalization processes on the cognitive system was confirmed. Response times for rotated displayed diagrams where significantly longer than for unrotated displays. A separate rotation of diagram labels and diagram content revealed the written labels to be a major factor in the additional mental normalization costs. Subsequently, the interventions to reduce the added burden of mental normalization were focused on different label designs. Both, the substitution of written labels with pictographs and color-coding the labels showed to be effective in increasing overall response speed, with color-coding being slightly more effective than pictograph labels. Additionally, the use of redundant, but inverted labels was tested for its effectiveness. While initial results seemed promising, the application to data visualizations were of limited usefulness. Double label displays increased the overall response time for all rotation conditions, but reduced the difference between them. We called this the ”justice effect”. In a second line of study, the potential benefits of mental normalization for long-term memory formation were investigated. The findings showed no such benefits and pointed towards potential adverse effects. Overall, this thesis provides new insights into the impact of mental normalization for the access to information presented in data visualizations in a collaborative work setting. it also showed potential interventions to reduce these costs and thus levelling the accessibility of the information for all users, regardless of their point of view

Collaborative Human-Computer Interaction with Big Wall Displays - BigWallHCI 2013 3rd JRC ECML Crisis Management Technology Workshop

The 3rd JRC ECML Crisis Management Technology Workshop on Human-Computer Interaction with Big Wall Displays in Situation Rooms and Monitoring Centres was co-organised by the European Commission Joint Research Centre and the University of Applied Sciences St. Pölten, Austria. It took place in the European Crisis Management Laboratory (ECML) of the JRC in Ispra, Italy, from 18 to 19 April 2013. 40 participants from stakeholders in the EC, civil protection bodies, academia, and industry attended the workshop. The hardware of large display areas is on the one hand mature since many years and on the other hand changing rapidly and improving constantly. This high pace developments promise amazing new setups with respect to e.g., pixel density or touch interaction. On the software side there are two components with room for improvement: 1. the software provided by the display manufacturers to operate their video walls (source selection, windowing system, layout control) and 2. dedicated ICT systems developed to the very needs of crisis management practitioners and monitoring centre operators. While industry starts to focus more on the collaborative aspects of their operating software already, the customized and tailored ICT applications needed are still missing, unsatisfactory, or very expensive since they have to be developed from scratch many times. Main challenges identified to enhance big wall display systems in crisis management and situation monitoring contexts include: 1. Interaction: Overcome static layouts and/or passive information consumption. 2. Participatory Design & Development: Software needs to meet users’ needs. 3. Development and/or application of Information Visualisation & Visual Analytics principle to support the transition from data to information to knowledge. 4. Information Overload: Proper methods for attention management, automatic interpretation, incident detection, and alarm triggering are needed to deal with the ever growing amount of data to be analysed.JRC.G.2-Global security and crisis managemen

ARTable

Examining issues that come with the emergence of technology, such as the fear of boredom, of intimacy, and of face-to-face conversation. This thesis studies the possibility of a customizable design technology whose goal is to mitigate those issues. Pixls is a low cost interactive tabletop allowing users to collaboratively create digital drawings on a 64-by-64 LED matrix screen using mobile phone touchscreen, thereby encouraging social interaction amongst them. Adopting the research through practices approach, insights, and solutions gathered from literature review and relevant case studies are used as a theoretical foundation for the conceptualization and building of the aforementioned prototype. Based on three key findings derived from the analysis of user feedback, this study concludes that the technology stack used within Pixls should be generalized into a platform, upon which different features can be implemented to afford various types of social interactions, including, but not limited to face-to-face interaction