Designing to Support Workspace Awareness in Remote Collaboration using 2D Interactive Surfaces

Increasing distributions of the global workforce are leading to collaborative workamong remote coworkers. The emergence of such remote collaborations is essentiallysupported by technology advancements of screen-based devices ranging from tabletor laptop to large displays. However, these devices, especially personal and mobilecomputers, still suffer from certain limitations caused by their form factors, that hinder supporting workspace awareness through non-verbal communication suchas bodily gestures or gaze. This thesis thus aims to design novel interfaces andinteraction techniques to improve remote coworkers’ workspace awareness throughsuch non-verbal cues using 2D interactive surfaces.The thesis starts off by exploring how visual cues support workspace awareness infacilitated brainstorming of hybrid teams of co-located and remote coworkers. Basedon insights from this exploration, the thesis introduces three interfaces for mobiledevices that help users maintain and convey their workspace awareness with their coworkers. The first interface is a virtual environment that allows a remote person to effectively maintain his/her awareness of his/her co-located collaborators’ activities while interacting with the shared workspace. To help a person better express his/her hand gestures in remote collaboration using a mobile device, the second interfacepresents a lightweight add-on for capturing hand images on and above the device’sscreen; and overlaying them on collaborators’ device to improve their workspace awareness. The third interface strategically leverages the entire screen space of aconventional laptop to better convey a remote person’s gaze to his/her co-locatedcollaborators. Building on the top of these three interfaces, the thesis envisions an interface that supports a person using a mobile device to effectively collaborate with remote coworkers working with a large display.Together, these interfaces demonstrate the possibilities to innovate on commodity devices to offer richer non-verbal communication and better support workspace awareness in remote collaboration

Design Opportunities for Assistive and Social Technologies Situated in Space

Computers are increasingly integrated into daily environments. Human interactions with computers become inseparable from interacting with physical spaces. I take a qualitative research approach to consider space as an integral part of understanding people, leading to design opportunities for technologies situated in space. My thesis presents the design opportunities for blind assistive technology to support exploratory navigation and physically-grounded social technology for remote connections. Studying space gives insights into how users and technologies situated in space interact with one another, how users perceive the physical space, and what behavioral patterns users display around the space while using technology. The insights contribute to constructing design implications from an underexplored creative angle and shed light on how spatial scenarios dynamically change preferences and opportunities in the design of future technologies

Presence and Communication in Hybrid Virtual and Augmented Reality Environments

The use of virtual reality (VR) and augmented reality (AR) in connected environments is rarely explored but may become a necessary channel of communication in the future. Such environments would allow multiple users to interact, engage, and share multi-dimensional data across devices and between the spectrum of realities. However, communication between the two realities within a hybrid environment is barely understood. We carried out an experiment with 52 participants in 26 pairs, within two environments of 3D cultural artifacts: 1) a Hybrid VR and AR environment (HVAR) and 2) a Shared VR environment (SVR). We explored the differences in perceived spatial presence, copresence, and social presence between the environments and between users. We demonstrated that greater presence is perceived in SVR when compared with HVAR, and greater spatial presence is perceived for VR users. Social presence is perceived greater for AR users, possibly because they have line of sight of their partners within HVAR. We found positive correlations between shared activity time and perceived social presence. While acquainted pairs reported significantly greater presence than unacquainted pairs in SVR, there were no significant differences in perceived presence between them in HVAR

Social Virtual Reality Platform Comparison and Evaluation Using a Guided Group Walkthrough Method

As virtual reality (VR) headsets become more commercially accessible, a range of social platforms have been developed that exploit the immersive nature of these systems. There is a growing interest in using these platforms in social and work contexts, but relatively little work into examining the usability choices that have been made. We developed a usability inspection method based on cognitive walkthrough that we call guided group walkthrough. Guided group walkthrough is applied to existing social VR platforms by having a guide walk the participants through a series of abstract social tasks that are common across the platforms. Using this method we compared six social VR platforms for the Oculus Quest. After constructing an appropriate task hierarchy and walkthrough question structure for social VR, we ran several groups of participants through the walkthrough process. We undercover usability challenges that are common across the platforms, identify specific design considerations and comment on the utility of the walkthrough method in this situation

Virtual Collaborative Design Environment: Information structure and interfaces

The failure to identify design issues in early phases of construction projects has been identified as a significant cause of costly rework, as these issues can impact the building occupants’ abilities to efficiently perform their daily work tasks. Therefore, it is crucial to consider their feedback when design reviewing. To date, efforts have been made to involve building occupants via a variety of user-interfaces that provide different understandings of the project. One such example is Virtual Reality (VR), which increases building occupants’ spatial understanding. Another, is use of design guidelines, intended to support both end-users such as building occupants and also the design team in basing their decision-making on best-practice and ensuring compliance with design requirements. When used together, these different user-interfaces can complement each other by enabling, for instance, visualization of the furniture layout depicted in design guideline documents. However, few studies have identified what is required of a design tool capable of supporting both visualization of design and design-compliance via different user-interfaces. Therefore, the aim of this thesis is to advance the understanding of end-users’ involvement in virtual collaborative environments in the building design process. Accordingly, Design Science Research was applied with a two-fold purpose. First, to identify different stakeholders’ challenges that are faced in the design process and specifically how building occupants’ daily work tasks are considered in the design process. Secondly, the research methods such as workshops, semi-structured interviews and documentation analysis helped identify the requirements of a design tool that would enable this knowledge to be transferred and accessible at a cross-project level. The results show that the information structure and user-interface of design guidelines determine to a large degree how effectively compliance with requirements can be validated. An example is the absence of user-interfaces in design guidelines which prevents building occupants from gaining sufficient spatial understanding. This lack of spatial understanding results in them to being reliant on other project members, such as architects and facility planners, for providing input on the design. Moreover, the results show how cross-project knowledge is difficult to facilitate due to how design guidelines have not been created in relation to today’s digital design process. Therefore, this thesis bridges the concepts of integrating design guidelines and VR in the same design tool

Interactive Experience Design: Integrated and Tangible Storytelling with Maritime Museum Artefacts

Museums play the role of intermediary between cultural heritage and visitors, and are often described as places and environments for education and enjoyment. The European Union also encourages innovative uses of museums to support education through the cultural heritage resources. However, the importance of visitors’ active role in museums as places for education and entertainment, on the one hand, and the growing and indispensable presence of technology in the cultural heritage domain, on the other hand, provided the initial ideas to develop the research. This thesis, presents the study and design for an interactive storytelling installation for a maritime museum. The installation is designed to integrate different museum artefacts into the storytelling system to enrich the visitors experience through tangible storytelling. The project was conducted in collaboration with another PhD student, Luca Ciotoli. His contribution was mainly focused on the narrative and storytelling features of the research, while my contribution was focused on the interaction- and technology-related features, including the design and implementation of the prototype. The research is deployed using a four-phase iterative approach. The first phase of the research, Study, deals with literature review and different studies to identify the requirements. The second phase, Design, determines the broad outlines of the project i.e., an interactive storytelling installation. The design phase includes interaction and museum experience design. We investigated different design approaches, e.g., interaction and museum experience design, to develop a conceptual design. The third phase, prototype, allows us to determine how to fulfill the tasks and meet the requirements that are established for the research. Prototyping involves content creation, storyboarding, integrating augmented artefacts into the storytelling system. Th final phase, test, refers to the evaluations that are conducted during the aforementioned phases e.g., formative and the final usability testing with users. The outcome of the research confirms previous results in the literature about how digital narratives can be enriched with the tangible dimension, moreover it shows how this dimension can enable to communicate stories and knowledge of the past that are complex, such as the art of navigating in the past, by integrating tangible objects that play different roles in the storytelling process

Virtual Reality in Mathematics Education (VRiME):An exploration of the integration and design of virtual reality for mathematics education

This thesis explores the use of Virtual Reality (VR) in mathematics education. Four VR prototypes were designed and developed during the PhD project to teach equations, geometry, and vectors and facilitate collaboration.Paper A investigates asymmetric VR for classroom integration and collaborative learning and presents a new taxonomy of asymmetric interfaces. Paper B proposes how VR could assist students with Autism Spectrum Disorder (ASD) in learning daily living skills involving basic mathematical concepts. Paper C investigates how VR could enhance social inclusion and mathematics learning for neurodiverse students. Paper D presents a VR prototype for teaching algebra and equation-solving strategies, noting positive student responses and the potential for knowledge transfer. Paper E investigates gesture-based interaction with dynamic geometry in VR for geometry education and presents a new taxonomy of learning environments. Finally, paper F explores the use of VR to visualise and contextualise mathematical concepts to teach software engineering students.The thesis concludes that VR offers promising avenues for transforming mathematics education. It aims to broaden our understanding of VR's educational potential, paving the way for more immersive learning experiences in mathematics education

Ubiq: A System to Build Flexible Social Virtual Reality Experiences

While they have long been a subject of academic study, social virtual reality (SVR) systems are now attracting increasingly large audiences on current consumer virtual reality systems. The design space of SVR systems is very large, and relatively little is known about how these systems should be constructed in order to be usable and efficient. In this paper we present Ubiq, a toolkit that focuses on facilitating the construction of SVR systems. We argue for the design strategy of Ubiq and its scope. Ubiq is built on the Unity platform. It provides core functionality of many SVR systems such as connection management, voice, avatars, etc. However, its design remains easy to extend. We demonstrate examples built on Ubiq and how it has been successfully used in classroom teaching. Ubiq is open source (Apache License) and thus enables several use cases that commercial systems cannot

Ubiq: A System to Build Flexible Social Virtual Reality Experiences

While they have long been a subject of academic study, social virtual reality (SVR) systems are now attracting increasingly large audiences on current consumer virtual reality systems. The design space of SVR systems is very large, and relatively little is known about how these systems should be constructed in order to be usable and efficient. In this paper we present Ubiq, a toolkit that focuses on facilitating the construction of SVR systems. We argue for the design strategy of Ubiq and its scope. Ubiq is built on the Unity platform. It provides core functionality of many SVR systems such as connection management, voice, avatars, etc. However, its design remains easy to extend. We demonstrate examples built on Ubiq and how it has been successfully used in classroom teaching. Ubiq is open source (Apache License) and thus enables several use cases that commercial systems cannot

Spatial Interaction for Immersive Mixed-Reality Visualizations

Growing amounts of data, both in personal and professional settings, have caused an increased interest in data visualization and visual analytics. Especially for inherently three-dimensional data, immersive technologies such as virtual and augmented reality and advanced, natural interaction techniques have been shown to facilitate data analysis. Furthermore, in such use cases, the physical environment often plays an important role, both by directly influencing the data and by serving as context for the analysis. Therefore, there has been a trend to bring data visualization into new, immersive environments and to make use of the physical surroundings, leading to a surge in mixed-reality visualization research. One of the resulting challenges, however, is the design of user interaction for these often complex systems. In my thesis, I address this challenge by investigating interaction for immersive mixed-reality visualizations regarding three core research questions: 1) What are promising types of immersive mixed-reality visualizations, and how can advanced interaction concepts be applied to them? 2) How does spatial interaction benefit these visualizations and how should such interactions be designed? 3) How can spatial interaction in these immersive environments be analyzed and evaluated? To address the first question, I examine how various visualizations such as 3D node-link diagrams and volume visualizations can be adapted for immersive mixed-reality settings and how they stand to benefit from advanced interaction concepts. For the second question, I study how spatial interaction in particular can help to explore data in mixed reality. There, I look into spatial device interaction in comparison to touch input, the use of additional mobile devices as input controllers, and the potential of transparent interaction panels. Finally, to address the third question, I present my research on how user interaction in immersive mixed-reality environments can be analyzed directly in the original, real-world locations, and how this can provide new insights. Overall, with my research, I contribute interaction and visualization concepts, software prototypes, and findings from several user studies on how spatial interaction techniques can support the exploration of immersive mixed-reality visualizations.Zunehmende Datenmengen, sowohl im privaten als auch im beruflichen Umfeld, führen zu einem zunehmenden Interesse an Datenvisualisierung und visueller Analyse. Insbesondere bei inhärent dreidimensionalen Daten haben sich immersive Technologien wie Virtual und Augmented Reality sowie moderne, natürliche Interaktionstechniken als hilfreich für die Datenanalyse erwiesen. Darüber hinaus spielt in solchen Anwendungsfällen die physische Umgebung oft eine wichtige Rolle, da sie sowohl die Daten direkt beeinflusst als auch als Kontext für die Analyse dient. Daher gibt es einen Trend, die Datenvisualisierung in neue, immersive Umgebungen zu bringen und die physische Umgebung zu nutzen, was zu einem Anstieg der Forschung im Bereich Mixed-Reality-Visualisierung geführt hat. Eine der daraus resultierenden Herausforderungen ist jedoch die Gestaltung der Benutzerinteraktion für diese oft komplexen Systeme. In meiner Dissertation beschäftige ich mich mit dieser Herausforderung, indem ich die Interaktion für immersive Mixed-Reality-Visualisierungen im Hinblick auf drei zentrale Forschungsfragen untersuche: 1) Was sind vielversprechende Arten von immersiven Mixed-Reality-Visualisierungen, und wie können fortschrittliche Interaktionskonzepte auf sie angewendet werden? 2) Wie profitieren diese Visualisierungen von räumlicher Interaktion und wie sollten solche Interaktionen gestaltet werden? 3) Wie kann räumliche Interaktion in diesen immersiven Umgebungen analysiert und ausgewertet werden? Um die erste Frage zu beantworten, untersuche ich, wie verschiedene Visualisierungen wie 3D-Node-Link-Diagramme oder Volumenvisualisierungen für immersive Mixed-Reality-Umgebungen angepasst werden können und wie sie von fortgeschrittenen Interaktionskonzepten profitieren. Für die zweite Frage untersuche ich, wie insbesondere die räumliche Interaktion bei der Exploration von Daten in Mixed Reality helfen kann. Dabei betrachte ich die Interaktion mit räumlichen Geräten im Vergleich zur Touch-Eingabe, die Verwendung zusätzlicher mobiler Geräte als Controller und das Potenzial transparenter Interaktionspanels. Um die dritte Frage zu beantworten, stelle ich schließlich meine Forschung darüber vor, wie Benutzerinteraktion in immersiver Mixed-Reality direkt in der realen Umgebung analysiert werden kann und wie dies neue Erkenntnisse liefern kann. Insgesamt trage ich mit meiner Forschung durch Interaktions- und Visualisierungskonzepte, Software-Prototypen und Ergebnisse aus mehreren Nutzerstudien zu der Frage bei, wie räumliche Interaktionstechniken die Erkundung von immersiven Mixed-Reality-Visualisierungen unterstützen können