Designing Tomorrow\u27s Reality: The Development and Validation of an Augmented and Mixed Reality Heuristic Checklist

Augmented (AR) and Mixed Reality (MR) are new and currently developing technologies. They have been used and shown promise and popularity in the domains of education, training, enterprise, retail, consumer products, and more. However, there is a lack of consistency and standards in AR and MR devices and applications. Interactions and standards in one application may drastically differ from another. This may make it difficult for users, especially those new to these technologies, to learn and feel comfortable using the devices or applications. It may also hinder the usability of the applications as designers may not follow proven techniques to display this information effectively. One way to create these standards is through the development and acceptance of usability or user experience (UX) heuristics. There is a lack of validated and widely accepted heuristics in AR and MR. Those that do exist tend to be too specialized to be valid across types of applications or devices. This dissertation’s goal is to fill this gap through the creation of a validated usability/user experience (UX) heuristic checklist to evaluate AR or MR devices and/or applications by following a validated methodology for developing usability/user experience heuristics (Quiñones et al., 2018). Previous work had been completed to develop an AR and MR heuristic checklist (Derby & Chaparro, 2022). This work resulted in 11 heuristics and 94 checklist items; however, validation of this checklist was limited. This dissertation broadened the heuristic checklist to ensure applicability to more application types, device types, and use cases. Five different applications and devices were used to validate the checklist through heuristic evaluations and user tests. Experts in the domain also provided their feedback on the heuristic checklist using applications of their choice. A total of 100 revisions were made to the Derby & Chaparro (2022) checklist as a result of this study. The final heuristic checklist consists of 12 heuristics and 109 checklist items that practitioners can use to evaluate AR or MR applications and devices and quantify the results to better inform design

Understanding Context to Capture when Reconstructing Meaningful Spaces for Remote Instruction and Connecting in XR

Recent technological advances are enabling HCI researchers to explore interaction possibilities for remote XR collaboration using high-fidelity reconstructions of physical activity spaces. However, creating these reconstructions often lacks user involvement with an overt focus on capturing sensory context that does not necessarily augment an informal social experience. This work seeks to understand social context that can be important for reconstruction to enable XR applications for informal instructional scenarios. Our study involved the evaluation of an XR remote guidance prototype by 8 intergenerational groups of closely related gardeners using reconstructions of personally meaningful spaces in their gardens. Our findings contextualize physical objects and areas with various motivations related to gardening and detail perceptions of XR that might affect the use of reconstructions for remote interaction. We discuss implications for user involvement to create reconstructions that better translate real-world experience, encourage reflection, incorporate privacy considerations, and preserve shared experiences with XR as a medium for informal intergenerational activities.Comment: 26 pages, 5 figures, 4 table

Music in Extended Realities

The intersection between music and Extended Reality (XR) has grown significantly over the past twenty years, amounting to an established area of research today. The use of XR technologies represents a fundamental paradigm shift for various musical contexts as they disrupt traditional notions of musical interaction by enabling performers and audiences to interact musically with virtual objects, agents, and environments. This article both surveys and expands upon the knowledge accumulated in existing research in this area to build a foundation for future works that bring together Music and XR. To this end, we created a freely available dataset of 260 publications in this space and conducted an in-depth analysis covering 199 works in the last decade. We conducted this analysis using a list of conceptual dimensions belonging to technical, artistic, perceptual and methodological domains. This review of the literature is complemented with a set of interviews with domain experts with the goal of establishing a definition for the emergent field of Musical XR, i.e., the field of music in Extended Realities. Based on the results of the conducted review, a research agenda for the field is proposed

Conceitos e métodos para apoio ao desenvolvimento e avaliação de colaboração remota utilizando realidade aumentada

Remote Collaboration using Augmented Reality (AR) shows great potential to establish a common ground in physically distributed scenarios where team-members need to achieve a shared goal. However, most research efforts in this field have been devoted to experiment with the enabling technology and propose methods to support its development. As the field evolves, evaluation and characterization of the collaborative process become an essential, but difficult endeavor, to better understand the contributions of AR. In this thesis, we conducted a critical analysis to identify the main limitations and opportunities of the field, while situating its maturity and proposing a roadmap of important research actions. Next, a human-centered design methodology was adopted, involving industrial partners to probe how AR could support their needs during remote maintenance. These outcomes were combined with literature methods into an AR-prototype and its evaluation was performed through a user study. From this, it became clear the necessity to perform a deep reflection in order to better understand the dimensions that influence and must/should be considered in Collaborative AR. Hence, a conceptual model and a humancentered taxonomy were proposed to foster systematization of perspectives. Based on the model proposed, an evaluation framework for contextualized data gathering and analysis was developed, allowing support the design and performance of distributed evaluations in a more informed and complete manner. To instantiate this vision, the CAPTURE toolkit was created, providing an additional perspective based on selected dimensions of collaboration and pre-defined measurements to obtain “in situ” data about them, which can be analyzed using an integrated visualization dashboard. The toolkit successfully supported evaluations of several team-members during tasks of remote maintenance mediated by AR. Thus, showing its versatility and potential in eliciting a comprehensive characterization of the added value of AR in real-life situations, establishing itself as a generalpurpose solution, potentially applicable to a wider range of collaborative scenarios.Colaboração Remota utilizando Realidade Aumentada (RA) apresenta um enorme potencial para estabelecer um entendimento comum em cenários onde membros de uma equipa fisicamente distribuídos precisam de atingir um objetivo comum. No entanto, a maioria dos esforços de investigação tem-se focado nos aspetos tecnológicos, em fazer experiências e propor métodos para apoiar seu desenvolvimento. À medida que a área evolui, a avaliação e caracterização do processo colaborativo tornam-se um esforço essencial, mas difícil, para compreender as contribuições da RA. Nesta dissertação, realizámos uma análise crítica para identificar as principais limitações e oportunidades da área, ao mesmo tempo em que situámos a sua maturidade e propomos um mapa com direções de investigação importantes. De seguida, foi adotada uma metodologia de Design Centrado no Humano, envolvendo parceiros industriais de forma a compreender como a RA poderia responder às suas necessidades em manutenção remota. Estes resultados foram combinados com métodos da literatura num protótipo de RA e a sua avaliação foi realizada com um caso de estudo. Ficou então clara a necessidade de realizar uma reflexão profunda para melhor compreender as dimensões que influenciam e devem ser consideradas na RA Colaborativa. Foram então propostos um modelo conceptual e uma taxonomia centrada no ser humano para promover a sistematização de perspetivas. Com base no modelo proposto, foi desenvolvido um framework de avaliação para recolha e análise de dados contextualizados, permitindo apoiar o desenho e a realização de avaliações distribuídas de forma mais informada e completa. Para instanciar esta visão, o CAPTURE toolkit foi criado, fornecendo uma perspetiva adicional com base em dimensões de colaboração e medidas predefinidas para obter dados in situ, que podem ser analisados utilizando o painel de visualização integrado. O toolkit permitiu avaliar com sucesso vários colaboradores durante a realização de tarefas de manutenção remota apoiada por RA, permitindo mostrar a sua versatilidade e potencial em obter uma caracterização abrangente do valor acrescentado da RA em situações da vida real. Sendo assim, estabelece-se como uma solução genérica, potencialmente aplicável a uma gama diversificada de cenários colaborativos.Programa Doutoral em Engenharia Informátic

Metaverse beyond the hype: Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy

The metaverse has the potential to extend the physical world using augmented and virtual reality technologies allowing users to seamlessly interact within real and simulated environments using avatars and holograms. Virtual environments and immersive games (such as, Second Life, Fortnite, Roblox and VRChat) have been described as antecedents of the metaverse and offer some insight to the potential socio-economic impact of a fully functional persistent cross platform metaverse. Separating the hype and “meta…” rebranding from current reality is difficult, as “big tech” paints a picture of the transformative nature of the metaverse and how it will positively impact people in their work, leisure, and social interaction. The potential impact on the way we conduct business, interact with brands and others, and develop shared experiences is likely to be transformational as the distinct lines between physical and digital are likely to be somewhat blurred from current perceptions. However, although the technology and infrastructure does not yet exist to allow the development of new immersive virtual worlds at scale - one that our avatars could transcend across platforms, researchers are increasingly examining the transformative impact of the metaverse. Impacted sectors include marketing, education, healthcare as well as societal effects relating to social interaction factors from widespread adoption, and issues relating to trust, privacy, bias, disinformation, application of law as well as psychological aspects linked to addiction and impact on vulnerable people. This study examines these topics in detail by combining the informed narrative and multi-perspective approach from experts with varied disciplinary backgrounds on many aspects of the metaverse and its transformational impact. The paper concludes by proposing a future research agenda that is valuable for researchers, professionals and policy makers alike

Investigating Trainee Perspectives on Virtual Reality Environments: An In-Depth Examination of Immersive Experiences with Haptic Feedback Vibration

This research investigates trainee reflections on Virithin Reality Environments (VREs) within educational training centers, aiming to understand their experiences, perceptions, and preferences. The study focuses on the impact of haptic feedback vibrations, examining both their general effects during VRE interactions and the specific influence of adaptable vibration configurations triggered by user errors. A convenience sample of 81 participants/trainees, 41 from the computer science and 40 from the aviation engineering departments of a major higher education institution in the U.A.E., was used to run four variations of the same VRE, with two from each field. Results indicate that participants largely embraced the VRE experience, reporting feelings of contentment, joy, and competence. Haptic feedback, particularly in non-adaptable forms, was acknowledged as enhancing the immersive experience. However, the study suggests that further research is needed to explore the nuanced role of adaptable vibration, especially in more complex interactions. Notably, participants expressed a preference for a blended approach, advocating for both VREs and physical labs in their training. The study acknowledges limitations, such as the predominantly single-user focus, and recommends future research extensions into collaborative VRE settings, more intricate interactions, and potential technical issues in multi-user scenarios. Overall, this research sheds light on the evolving landscape of educational training, emphasizing the importance of understanding trainee perspectives to optimize the integration of VREs in learning environment

YOUTH PERCEPTIONS ON LEARNING IN IMMERSIVE VIRTUAL REALITY: DEVELOPING AND ASSESSING ESSENTIAL SOCIOEMOTIONAL SKILLS

This qualitative study investigated youth perceptions of educational virtual reality (VR) for developing socioemotional skills. A constructivist theoretical framework was utilized to guide the research design according to five key elements: learning in relevant settings, the social negotiation of learning, ownership in learning, multimodal representation, and metacognition. An exploratory case study approach was used. A grade 8 class of 28 students were invited to participate as the co-researchers for this study. The methods for data collection were semi structured artifact analysis, observations, surveys, and interviews. Research activities involved guided exploration of sustainability issues in AltspaceVR, collaborative painting in MultibrushVR, and an artifact contribution to a co-created learning experience in FrameVR. Empathy was the most predominant type of social-emotional learning (SEL) demonstrated by the co-researchers: Teamwork, creativity, and problem-solving skills were also evident. The youth exemplified new empathetic insights and activism for people, the environment, and animals based on their learning in immersive contexts. Findings indicate a rationale for educational VR to include SEL competencies, which VR excels at through its capacity for perspective-taking. The research activities within the virtual environments contributed to the co-researchers’ sense of social presence through open communication, affective expression, and group cohesion. Although the scholarly literature identifies personalized avatars as influential for enhancing social presence in VR environments, there was a lack of consensus on avatar value. The data collected in this study may serve as a basis for further research on understanding the capacity of educational VR to promote empathy and SEL in youth

Transforming our World through Universal Design for Human Development

An environment, or any building product or service in it, should ideally be designed to meet the needs of all those who wish to use it. Universal Design is the design and composition of environments, products, and services so that they can be accessed, understood and used to the greatest extent possible by all people, regardless of their age, size, ability or disability. It creates products, services and environments that meet people’s needs. In short, Universal Design is good design. This book presents the proceedings of UD2022, the 6th International Conference on Universal Design, held from 7 - 9 September 2022 in Brescia, Italy.The conference is targeted at professionals and academics interested in the theme of universal design as related to the built environment and the wellbeing of users, but also covers mobility and urban environments, knowledge, and information transfer, bringing together research knowledge and best practice from all over the world. The book contains 72 papers from 13 countries, grouped into 8 sections and covering topics including the design of inclusive natural environments and urban spaces, communities, neighborhoods and cities; housing; healthcare; mobility and transport systems; and universally- designed learning environments, work places, cultural and recreational spaces. One section is devoted to universal design and cultural heritage, which had a particular focus at this edition of the conference. The book reflects the professional and disciplinary diversity represented in the UD movement, and will be of interest to all those whose work involves inclusive design

Gaze Awareness in Computer-Mediated Collaborative Physical Tasks

Human eyes play an important role in everyday social interactions. However, the cues provided by eye movements are often missing or difficult to interpret in computer-mediated remote collaboration. Motivated by the increasing availability of gaze-tracking devices in the consumer market and the growing need for improved remotecollaboration systems, this thesis evaluated the value of gaze awareness in a number of video-based remote-collaboration situations. This thesis comprises six publications which enhance our understanding of the everyday use of gaze-tracking technology and the value of shared gaze to remote collaborations in the physical world. The studies focused on a variety of collaborative scenarios involving different camera configurations (stationary, handheld, and head-mounted cameras), display setups (screen-based and projection displays), mobility requirements (stationary and mobile tasks), and task characteristics (pointing and procedural tasks). The aim was to understand the costs and benefits of shared gaze in video-based collaborative physical tasks. The findings suggest that gaze awareness is useful in remote collaboration for physical tasks. Shared gaze enables efficient communication of spatial information, helps viewers to predict task-relevant intentions, and enables improved situational awareness. However, different contextual factors can influence the utility of shared gaze. Shared gaze was more useful when the collaborative task involved communicating pointing information instead of procedural information, the collaborators were mutually aware of the shared gaze, and the quality of gaze-tracking was accurate enough to meet the task requirements. In addition, the results suggest that the collaborators’ roles can also affect the perceived utility of shared gaze. Methodologically, this thesis sets a precedent in shared gaze research by reporting the objective gaze data quality achieved in the studies and also provides tools for other researchers to objectively view gaze data quality in different research phases. The findings of this thesis can contribute towards designing future remote-collaboration systems; towards the vision of pervasive gaze-based interaction; and towards improved validity, repeatability, and comparability of research involving gaze trackers