Designing information feedback within hybrid physical/digital interactions

Whilst digital and physical interactions were once treated as separate design challenges, there is a growing need for them to be considered together to allow the creation of hybrid digital/physical experiences. For example, digital games can now include physical objects (with digital properties) or digital objects (with physical properties), both of which may be used to provide input, output, or in-game information in various combinations. In this paper we consider how users perceive and understand interactions that include physical/digital objects through the design of a novel game which allows us to consider: i) the character of the space/spaces in which we interact; ii) how users perceive their operation; and iii) how we can design such objects to extend the bandwidth of information we provide to the user/player. The prototype is used as the focus of a participatory design workshop in which players experimented with, and discussed physical ways of representing the virtual in-game information. The results have been used to provide a framing for designers approaching information feedback in this domain, and highlight the requirement for further design research

Econundrum:Visualizing the Climate Impact of Dietary Choice through a Shared Data Sculpture

While there is a strong relationship between climate change and human food consumption, it is challenging to understand the implications and impact from an individual perspective. The lack of a shared frame of reference, that allows people to compare their impact to others, limits awareness on this complex topic. To support group reflections and social comparison of the impact of people’s food consumption on climate change, we designed Econundrum, a shared physical data sculpture that visualizes carbon emissions resulting from dietary choices of a small community. Our three-week field study demonstrates how Econundrum helped people (i) understand the climate impact of various food types, (ii) reflect on the environmental impact of their food choices; and (iii) discuss the relation between climate impact and food consumption with others. Our study shows how a shared physical data sculpture mediates a complex topic to a community by facilitating the social dynamics in context

Dynamic awareness techniques for VR user interactions with bystanders

Virtual reality (VR) headsets are often used in shared, social, settings. However, the occlusive nature of VR headsets introduce significant barriers to a user’s awareness of, and interactions with, bystanders (individuals physically near the VR user but who cannot directly interact with the user’s virtual environment). This thesis posits that VR headsets do not sufficiently support a user’s interaction with bystanders. This thesis investigates how technology-mediated bystander awareness systems can support use of VR headsets by automatically increasing a user’s awareness of, and facilitating an interaction with, bystanders. Survey 1 & 2 and Experiment 1 explored this by first investigating how interactions occur between bystanders and VR users and what impediments are encountered during these interactions. Experiments 2, 3, and 4 then built on this work by investigating the design of technology-mediated bystander awareness systems designed to support these interactions. Experiment 2 explored the impact of withholding a bystander’s identity and position when informing a VR user of a bystander’s co-presence, finding some users will exit VR upon being notified of bystander co-presence if they perceive the information relayed to be insufficient for contextualising the bystander. Experiments 3 and 4 explored how the audio experienced by a VR user might be manipulated to increase aural awareness of reality and facilitate a verbal interaction. Experiments 3 and 4 found automatically decreasing the audio’s volume or partially/fully removing audio components are effective audio manipulations to facilitate verbal bystander-VR user interactions. Finally, Experiment 5, informed by the findings of all prior studies, investigated if/how a VR user’s awareness needs vary during an interaction with a bystander. Its results demonstrate that no single bystander awareness system can adequately support the awareness needs of VR users who balance a complex trade-off between awareness and immersion, individual priorities and concerns in relation to the bystander, and the influence of experiential and contextual factors

Augmenting Learning Activities with Contextual Information Scent

Students often have information needs while carrying out a multitude of learning activities at universities. When information is needed for investigating a problem, the student may interrupt the work and switch to an information seeking task. As Internet connectivity becomes ubiquitous, searching information has been routinized and integrated in the learning experience. However, information needs are not always fully recognized, or they can not be well articulated. A MOOC student may perceive a video to be difficult, but fails to express what information can be helpful. Sometimes it is improper to interrupt the learning task for searching information, especially when social factors are concerned, e.g. in a seminar talk. These situations create research potentials for making ambient information cues, hereafter referred to as contextual information scent (CIS), available to address students' situational information needs in learning activities. The CIS is designed to combine context-awareness with information seeking, ambient interaction as well as serendipitous encounter. In this thesis, we investigate the CIS mainly in collaborative learning activities. We explore three different contexts: conversation, groupware interaction and video content for MOOC learning. RaindropSearch investigates capturing conversational words as CIS for building search queries, while the TileSearch triggers Web searches based on group discussions and retrieved image and Wikipedia results as CIS for serendipitous interactions. These two explorations both focus on conversation context and provide initial insights into the CIS design practice. Next, we present MeetHub Search, which includes three CIS components based on text interactions in a groupware. Our last prototype, the BOOC Player employs textbook pages as CIS and links them to MOOC videos during the course of collaborative video viewing. All prototypes show how we manipulated design parameters to reduce distraction, increase relevance and ensure timeliness. The studies also exhibit the influence of group dynamics on the use of CIS. We finally extend our research scope to individual MOOC learning and summarize the design insights obtained from MOOC analytics. The contributions of this thesis are summarized as (1) a dedicated research framework derived from both research literature and requirement analysis for recognizing the design challenges, design principles and design space of CIS. The framework lays the foundation for us to explore different contexts in this thesis, where we generated (2) design implications that identify the key attributes of CIS. Last but not least, we employed (3) a variety of evaluation methodologies in this thesis for assessing the usability as well as the benefit and appeal of CIS

Peripheral interaction

In our everyday life we carry out a multitude of activities in parallel without focusing our attention explicitly on them. We drink a cup of tea while reading a book, we signal a colleague passing by with a hand gesture, that we are concentrated right now and that he should wait one moment, or we walk a few steps backwards while taking photos. Many of these interactions - like drinking, sending signals via gestures or walking - are rather complex by themselves. By means of learning and training, however, these interactions become part of our routines and habits and therefore only consume little or no attentional resources. In contrast, when interacting with digital devices, we are often asked for our full attention. To carry out - even small and marginal tasks - we are regularly forced to switch windows, do precise interactions (e.g., pointing with the mouse) and thereby these systems trigger context and focus switches, disrupting us in our main focus and task. Peripheral interaction aims at making use of human capabilities and senses like divided attention, spatial memory and proprioception to support interaction with digital devices in the periphery of the attention, consequently quasi-parallel to another primary task. In this thesis we investigate peripheral interaction in the context of a standard desktop computer environment. We explore three interaction styles for peripheral interaction: graspable interaction, touch input and freehand gestures. StaTube investigates graspable interaction in the domain of instant messaging, while the Appointment Projection uses simple wiping gestures to access information about upcoming appointments. These two explorations focus on one interaction style each and offer first insights into the general benefits of peripheral interaction. In the following we carried out two studies comparing all three interaction styles (graspable, touch, freehand) for audio player control and for dealing with notifications. We found that all three interaction styles are generally fit for peripheral interaction but come with different advantages and disadvantages. The last set of explorative studies deals with the ability to recall spatial locations in 2D as well as 3D. The Unadorned Desk makes use of the physical space around the desktop computer and thereby offers an extended interaction space to store and retrieve virtual items such as commands, applications or tools. Finally, evaluation of peripheral interaction is not straightforward as the systems are designed to blend into the environment and not draw attention on them. We propose an additional evaluation method for the lab to complement the current evaluation practice in the field. The main contributions of this thesis are (1) an exhaustive classification and a more detailed look at manual peripheral interaction for tangible, touch and freehand interaction. Based on these exploration with all three interaction styles, we offer (2) implications in terms of overall benefits of peripheral interaction, learnability and habituation, visual and mental attention, feedback and handedness for future peripheral interaction design. Finally, derived from a diverse set of user studies, we assess (3) evaluation strategies enriching the design process for peripheral interaction.In unserem täglichen Leben führen wir eine große Anzahl an Aktivitäten parallel aus ohne uns explizit darauf zu konzentrieren. Wir trinken Tee während wir ein Buch lesen, wir signalisieren einem Kollegen durch eine Handgeste, dass wir gerade konzentriert sind und er einen Moment warten soll oder wir gehen ein paar Schritte rückwärts während wir fotografieren. Viele dieser Aktivitäten - wie beispielsweise Trinken, Gestikulieren und Laufen - sind an sich komplex. Durch Training werden diese Tätigkeiten allerdings Teil unserer Routinen und Gewohnheiten, und beanspruchen daher nur noch wenig oder sogar keine Aufmerksamkeit. Im Gegensatz dazu, verlangen digitale Geräte meist unsere volle Aufmerksamkeit während der Interaktion. Um - oftmals nur kleine - Aufgaben durchzuführen, müssen wir Fenster wechseln, präzise Aktionen durchführen (z.B. mit dem Mauszeiger zielen) und werden dabei durch die Systeme zu einem Kontext- und Fokuswechsel gezwungen. Periphere Interaktion hingegen macht sich menschliche Fähigkeiten wie geteilte Aufmerksamkeit, das räumliche Gedächtnis und Propriozeption zu Nutze um Interaktion mit digitalen Geräten am Rande der Aufmerksamkeit also der Peripherie zu ermöglichen -- quasi-parallel zu einem anderen Primärtask. In dieser Arbeit untersuchen wir Periphere Interaktion am Computerarbeitsplatz. Dabei betrachten wir drei verschiedene Interaktionsstile: Begreifbare Interaktion (graspable), Touch Eingabe und Freiraum Gestik (freehand). StaTube untersucht Begreifbare Interaktion am Beispiel von Instant Messaging, während die Appointment Projection einfache Wischgesten nutzt, um Informationen nahender Termine verfügbar zu machen. Diese beiden Untersuchungen betrachten jeweils einen Interaktionsstil und beleuchten erste Vorteile, die durch Periphere Interaktion erzielt werden können. Aufbauend darauf führen wir zwei vergleichende Studien zwischen allen drei Interaktionsstilen durch. Als Anwendungsszenarien dienen Musiksteuerung und der Umgang mit Benachrichtigungsfenstern. Alle drei Interaktionsstile können erfolgreich für Periphere Interaktion eingesetzt werden, haben aber verschiedene Vor- und Nachteile. Die letzte Gruppe von Studien befasst sich mit dem räumlichen Gedächtnis in 2D und 3D. Das Unadorned Desk nutzt den physikalischen Raum neben dem Desktop Computer um virtuelle Objekte, beispielsweise Funktionen, Anwendungen oder Werkzeuge, zu lagern. Darüber hinaus ist die Evaluation von Peripherer Interaktion anspruchsvoll, da sich die Systeme in die Umwelt integrieren und gerade keine Aufmerksamkeit auf sich ziehen sollen. Wir schlagen eine Evaluationsmethode für das Labor vor, um die derzeitig vorherrschenden Evaluationsmethoden in diesem Forschungsfeld zu ergänzen. Die Kernbeiträge dieser Arbeit sind eine (1) umfassende Klassifizierung und ein detaillierter Blick auf manuelle Periphere Interaktion, namentlich Begreifbare Interaktion, Touch Eingabe und Freiraum Gestik. Basierend auf unseren Untersuchungen ziehen wir (2) Schlussfolgerungen, die den generellen Nutzen von Peripherer Interaktion darlegen und Bereiche wie die Erlernbarkeit und Gewöhnung, visuelle und mentale Aufmerksamkeit, Feedback so wie Händigkeit beleuchten um zukünftige Projekte im Bereich der Peripheren Interaktion zu unterstützen. Aufbauend auf den verschiedenen Nutzerstudien, diskutieren wir Evaluationsstrategien um den Entwicklungsprozess Peripherer Interaktion zu unterstützen