Brainstorming und Mind-Mapping im Multi-Device-Kontext. Konzeption und prototypische Implementierung für Multi-Touch-Tabletop und Smartphone

Die vorliegende Arbeit beschreibt die Konzeption und prototypische Implemen­tierung einer Anwendung zur elektronischen Unterstützung von Brainstorming- und Mind-Mapping-Sitzungen an einem multitouchfähigen Tabletop mit Smart­phones. Während der Tabletop durch seine große, horizontal ausgerichtete Oberfläche die kollaborative Erstellung und Strukturierung von Ideen in Gruppen unterstützt, werden Smartphones einen zusätzlichen Eingabekanal zur Verfügung stellen sowie Individualarbeit fördern. Somit wird zunächst einerseits die Motivation für die Verwirklichung einer Anwendung zum Brainstorming und Mind- Mapping am Tabletop und andererseits das Potential einer Ergänzung eines sol­chen Systems durch zusätzliche private Geräte im Sinne der Multi-Device-Inter­aktion erläutert werden. Um eine geeignete theoretische Basis zu schaffen, wird darüber hinaus ein detaillierter Überblick über die Entwicklung der Tabletop­technologie in den letzten Jahren und den dabei zentralen Forschungs- und Pro­blemfeldern im Kontext der Entwicklung von Benutzerschnittstellen für Tabletopsysteme gegeben. Zudem werden schließlich auch im Rahmen der Arbeit relevante Aspekte der Multi-Device-Interaktion skizziert. Diesem theoretisch ausgerichteten Teil der Arbeit folgt schließlich eine Beschreibung der Entwicklung des Anwendungskonzepts und der dabei formulierten Anforderungen und Designzielen, welche in Form eines ersten, papierbasierten Prototypen visualisiert werden. Aufbauend auf diesen konzeptuellen Überlegungen wird schließlich die konkrete technische Umsetzung der Anwendung Multi/Touch/Device Mind­ Mapper in Form eines High-Level-Prototypen auf Basis des MT4j-Frameworks (Tabletop) und des Android-Betriebssystems (Smartphone) beleuchtet. Eine Dis­kussion dieses finalen Prototypen und erster praktischer Erfahrungen sowie ein Ausblick auf Erweiterungsmöglichkeiten des Systems und über den Rahmen der Arbeit hinaus gehende Fragestellungen schließen die Arbeit ab. Schlussendlich kann gezeigt werden, dass die im Rahmen des Anwendungskonzept definierten Anforderungen mit Hilfe der verwendeten Frameworks bis auf wenige Ausnahmen erfolgreich umgesetzt werden konnten. Darüber hinaus kann der Anwendung durch die aus dem Testbetrieb gewonnenen Erkenntnisse eine grundsätzli­ che Praktikabilität attestiert, ebenso konnten einige Ansatzpunkte für weitere Verbesserungen und Tests aufgedeckt werden

Collaborative behavior, performance and engagement with visual analytics tasks using mobile devices

Interactive visualizations are external tools that can support users’ exploratory activities. Collaboration can bring benefits to the exploration of visual representations or visu‐ alizations. This research investigates the use of co‐located collaborative visualizations in mobile devices, how users working with two different modes of interaction and view (Shared or Non‐Shared) and how being placed at various position arrangements (Corner‐to‐Corner, Face‐to‐Face, and Side‐by‐Side) affect their knowledge acquisition, engagement level, and learning efficiency. A user study is conducted with 60 partici‐ pants divided into 6 groups (2 modes×3 positions) using a tool that we developed to support the exploration of 3D visual structures in a collaborative manner. Our results show that the shared control and view version in the Side‐by‐Side position is the most favorable and can improve task efficiency. In this paper, we present the results and a set of recommendations that are derived from them

Collaborative enquiry through the tabletop for second/foreign language learners

Abstract. Interactional communicative competence and higher-order thinking have been well documented as two of the biggest challenges for second/foreign language learners (EFL learners). This paper evaluates the use of digital tabletops as tools for problem-solving tasks in groups. The evaluation is based on a preliminary study of an application of the use of Digital Mysteries task with EFL learners in a Higher Education institution. It focuses more specifically on the extent to which collaborative learning platforms provided by interactive tabletops can promote and support the application of both thinking and linguistic skills for EFL learners. Based on an interdisciplinary perspective which draws from instructed second language learning and human-computer interaction fields, the evaluation considers momentto- moment multimodal interaction of three groups of Chinese English language learners with and around the completion of the Digital Mysteries task. It seeks to identify what specific affordances in the design might benefit EFL learners in terms of thinking skills, interactional competence and linguistic performance, and by the same token, what might not. This paper concludes with a number of suggestions about how technologies designed for collaborative enquiry might be repurposed for higher-order thinking and language learning

On the Use of Large Interactive Displays to Support Collaborative Engagement and Visual Exploratory Tasks

Large interactive displays can provide suitable workspaces for learners to conduct collaborative learning tasks with visual information in co-located settings. In this research, we explored the use of these displays to support collaborative engagement and exploratory tasks with visual representations. Our investigation looked at the effect of four factors (number of virtual workspaces within the display, number of displays, position arrangement of the collaborators, and collaborative modes of interaction) on learners' knowledge acquisition, engagement level, and task performance. To this end, a user study was conducted with 72 participants divided into 6 groups using an interactive tool developed to support the collaborative exploration of 3D visual structures. The results of this study showed that learners with one shared workspace and one single display can achieve better user performance and engagement levels. In addition, the back-to-back position with learners sharing their view and control of the workspaces was the most favorable. It also led to improved learning outcomes and engagement levels during the collaboration process

Developing a Multi-Touch Map Application for a Large Screen in a Nature Centre

The paper describes the development of a research prototype of a multi-touch map application for multi-use on a large multi-touch screen intended for a nature centre. The presented system and the development steps provide insight into what can be expected when similar systems are designed. A number of new considerations regarding multi-touch interaction, map browsing, and user needs for multi-use have been taken into account during the challenging ongoing development. These considerations include making a simple user interface used with intuitive, continuous and simultaneous gestures for map browsing, and taking different kinds of users and their needs to interact with each other into account. Since multi-user map applications in multi-touch environments are still rare, the given considerations may be helpful for the future development of similar map applications intended for public spaces

Move, hold and touch: A framework for Tangible gesture interactive systems

© 2015 by the authors. Technology is spreading in our everyday world, and digital interaction beyond the screen, with real objects, allows taking advantage of our natural manipulative and communicative skills. Tangible gesture interaction takes advantage of these skills by bridging two popular domains in Human-Computer Interaction, tangible interaction and gestural interaction. In this paper, we present the Tangible Gesture Interaction Framework (TGIF) for classifying and guiding works in this field. We propose a classification of gestures according to three relationships with objects: move, hold and touch. Following this classification, we analyzed previous work in the literature to obtain guidelines and common practices for designing and building new tangible gesture interactive systems. We describe four interactive systems as application examples of the TGIF guidelines and we discuss the descriptive, evaluative and generative power of TGIF

Designing for Shareable Interfaces in the Wild

Despite excitement about the potential of interactive tabletops to support collaborative work, there have been few empirical demonstrations of their effectiveness (Marshall et al., 2011). In particular, while lab-based studies have explored the effects of individual design features, there has been a dearth of studies evaluating the success of systems in the wild. For this technology to be of value, designers and systems builders require a better understanding of how to develop and evaluate tabletop applications to be deployed in real world settings. This dissertation reports on two systems designed through a process that incorporated ethnography-style observations, iterative design and in the wild evaluation. The first study focused on collaborative learning in a medical setting. To address the fact that visitors to a hospital emergency ward were leaving with an incomplete understanding of their diagnosis and treatment, a system was prototyped in a working Emergency Room (ER) with doctors and patients. The system was found to be helpful but adoption issues hampered its impact. The second study focused on a planning application for visitors to a tourist information centre. Issues and opportunities for a successful, contextually-fitted system were addressed and it was found to be effective in supporting group planning activities by novice users, in particular, facilitating users’ first experiences, providing effective signage and offering assistance to guide the user through the application. This dissertation contributes to understanding of multi-user systems through literature review of tabletop systems, collaborative tasks, design frameworks and evaluation of prototypes. Some support was found for the claim that tabletops are a useful technology for collaboration, and several issues were discussed. Contributions to understanding in this field are delivered through design guidelines, heuristics, frameworks, and recommendations, in addition to the two case studies to help guide future tabletop system creators

The Case for Public Interventions during a Pandemic

Funding Information: This work has been supported by Marie Skłodowska Curie Actions ITN AffecTech (ERC H2020 Project 1059 ID: 722022). Publisher Copyright: © 2022 by the authors.Within the field of movement sensing and sound interaction research, multi-user systems have gradually gained interest as a means to facilitate an expressive non-verbal dialogue. When tied with studies grounded in psychology and choreographic theory, we consider the qualities of interaction that foster an elevated sense of social connectedness, non-contingent to occupying one’s personal space. Upon reflection of the newly adopted social distancing concept, we orchestrate a technological intervention, starting with interpersonal distance and sound at the core of interaction. Materialised as a set of sensory face-masks, a novel wearable system was developed and tested in the context of a live public performance from which we obtain the user’s individual perspectives and correlate this with patterns identified in the recorded data. We identify and discuss traits of the user’s behaviour that were accredited to the system’s influence and construct four fundamental design considerations for physically distanced sound interaction. The study concludes with essential technical reflections, accompanied by an adaptation for a pervasive sensory intervention that is finally deployed in an open public space.publishersversionpublishe

A Framework For Abstracting, Designing And Building Tangible Gesture Interactive Systems

This thesis discusses tangible gesture interaction, a novel paradigm for interacting with computer that blends concepts from the more popular fields of tangible interaction and gesture interaction. Taking advantage of the human innate abilities to manipulate physical objects and to communicate through gestures, tangible gesture interaction is particularly interesting for interacting in smart environments, bringing the interaction with computer beyond the screen, back to the real world. Since tangible gesture interaction is a relatively new field of research, this thesis presents a conceptual framework that aims at supporting future work in this field. The Tangible Gesture Interaction Framework provides support on three levels. First, it helps reflecting from a theoretical point of view on the different types of tangible gestures that can be designed, physically, through a taxonomy based on three components (move, hold and touch) and additional attributes, and semantically, through a taxonomy of the semantic constructs that can be used to associate meaning to tangible gestures. Second, it helps conceiving new tangible gesture interactive systems and designing new interactions based on gestures with objects, through dedicated guidelines for tangible gesture definition and common practices for different application domains. Third, it helps building new tangible gesture interactive systems supporting the choice between four different technological approaches (embedded and embodied, wearable, environmental or hybrid) and providing general guidance for the different approaches. As an application of this framework, this thesis presents also seven tangible gesture interactive systems for three different application domains, i.e., interacting with the In-Vehicle Infotainment System (IVIS) of the car, the emotional and interpersonal communication, and the interaction in a smart home. For the first application domain, four different systems that use gestures on the steering wheel as interaction means with the IVIS have been designed, developed and evaluated. For the second application domain, an anthropomorphic lamp able to recognize gestures that humans typically perform for interpersonal communication has been conceived and developed. A second system, based on smart t-shirts, recognizes when two people hug and reward the gesture with an exchange of digital information. Finally, a smart watch for recognizing gestures performed with objects held in the hand in the context of the smart home has been investigated. The analysis of existing systems found in literature and of the system developed during this thesis shows that the framework has a good descriptive and evaluative power. The applications developed during this thesis show that the proposed framework has also a good generative power.Questa tesi discute l’interazione gestuale tangibile, un nuovo paradigma per interagire con il computer che unisce i principi dei più comuni campi di studio dell’interazione tangibile e dell’interazione gestuale. Sfruttando le abilità innate dell’uomo di manipolare oggetti fisici e di comunicare con i gesti, l’interazione gestuale tangibile si rivela particolarmente interessante per interagire negli ambienti intelligenti, riportando l’attenzione sul nostro mondo reale, al di là dello schermo dei computer o degli smartphone. Poiché l’interazione gestuale tangibile è un campo di studio relativamente recente, questa tesi presenta un framework (quadro teorico) che ha lo scopo di assistere lavori futuri in questo campo. Il Framework per l’Interazione Gestuale Tangibile fornisce supporto su tre livelli. Per prima cosa, aiuta a riflettere da un punto di vista teorico sui diversi tipi di gesti tangibili che possono essere eseguiti fisicamente, grazie a una tassonomia basata su tre componenti (muovere, tenere, toccare) e attributi addizionali, e che possono essere concepiti semanticamente, grazie a una tassonomia di tutti i costrutti semantici che permettono di associare dei significati ai gesti tangibili. In secondo luogo, il framework proposto aiuta a concepire nuovi sistemi interattivi basati su gesti tangibili e a ideare nuove interazioni basate su gesti con gli oggetti, attraverso linee guida per la definizione di gesti tangibili e una selezione delle migliore pratiche per i differenti campi di applicazione. Infine, il framework aiuta a implementare nuovi sistemi interattivi basati su gesti tangibili, permettendo di scegliere tra quattro differenti approcci tecnologici (incarnato e integrato negli oggetti, indossabile, distribuito nell’ambiente, o ibrido) e fornendo una guida generale per la scelta tra questi differenti approcci. Come applicazione di questo framework, questa tesi presenta anche sette sistemi interattivi basati su gesti tangibili, realizzati per tre differenti campi di applicazione: l’interazione con i sistemi di infotainment degli autoveicoli, la comunicazione interpersonale delle emozioni, e l’interazione nella casa intelligente. Per il primo campo di applicazione, sono stati progettati, sviluppati e testati quattro differenti sistemi che usano gesti tangibili effettuati sul volante come modalità di interazione con il sistema di infotainment. Per il secondo campo di applicazione, è stata concepita e sviluppata una lampada antropomorfica in grado di riconoscere i gesti tipici dell’interazione interpersonale. Per lo stesso campo di applicazione, un secondo sistema, basato su una maglietta intelligente, riconosce quando due persone si abbracciano e ricompensa questo gesto con uno scambio di informazioni digitali. Infine, per l’interazione nella casa intelligente, è stata investigata la realizzazione di uno smart watch per il riconoscimento di gesti eseguiti con oggetti tenuti nella mano. L’analisi dei sistemi interattivi esistenti basati su gesti tangibili permette di dimostrare che il framework ha un buon potere descrittivo e valutativo. Le applicazioni sviluppate durante la tesi mostrano che il framework proposto ha anche un valido potere generativo

Electronic Environments for Reading: An Annotated Bibliography of Pertinent Hardware and Software

In the development of new research environments, hardware has often been neglected. E-readers have (reasonably) successfully been developed for leisurely reading, but reading with the goal of writing demands a different approach. This bibliography has been written to inform the INKE research group on physical aspects of digital scholarly reading. It consists of two parts: a hardware section, including a description of commercial e-readers as well as an overview of academically developed digital reading devices and a software section, also including commercially available packages next to academically developed reading environments which allow for flexible manipulation of text and other modalities; as well as reflections on digital scholarly reading. Combined, the two sections inform an integrated approach in the development of new research environments