Emerging challenges for HCI : enabling effective use of VR in education and training

This chapter considers some of the challenges in providing effective virtual reality (VR) environments for teaching and training, where users are encouraged and enabled to be truly engaged in their learning. One approach is to use inquiry-based learning, linking that to the use of VR as a vehicle for education. This chapter introduces the notion of virtual learning spaces in a more general form and makes the case that a virtual learning space is any online environment where the learner perceives that they are interacting or gaming, and thus can be enabled within a virtual environment. Thus, virtual learning spaces are not limited to bespoke education learning software but can be considered in any context where the user perceives that they are engaging, having fun, and exploring, as can be implemented within a VR platform.Infotainment or gaming is a potential area that we can exploit to improve the effectiveness of training and learning and is particularly relevant within VR learning spaces. This chapter considers both synchronous and asynchronous computer-mediated communication and virtual and immersed virtual reality to illustrate potential virtual learning spaces. The chapter presents some worked illustrations to show how this could be used with the context of teaching Computer Science

The Influence of Display Parameters and Display Devices over Spatial Ability Test Answers in Virtual Reality Environments

L

HOW METAVERSE EVOLVES THE ARCHITECTURAL DESIGN

Architects have long relied on visualization tools to develop their concepts for specific design problems. From the early traditional drawings to the three-dimensional visualizations and virtual environments, all have enabled architects to demonstrate design outputs relatively early in the process. Real-world projects are similar to what architects imagined from the beginning. In other words, the design process has always started by creating the digital representation of a project and then attempting to replicate it in real life. Once the digital representation of design parts is complete, architects prepare their design for construction. However, the final visualization emerges from actual architectural functions, structure constraints, Gravity, materiality, privacy, and physical laws, meaning that architecture evolves the digitally represented visualizations. With the growth of the metaverse, all physical restrictions are being eliminated, and architects can expand the boundaries of how spaces can be represented regardless of being virtual or physical. As a virtual environment on the internet, the metaverse redefines the rules of architecture and offers endless possibilities for architectural innovation. This article aims to explore the role the metaverse plays in designing architecture. It outlines the fundamental concepts of the metaverse to identify significant elements that could influence architecture design

Earth as Interface: Exploring chemical senses with Multisensory HCI Design for Environmental Health Communication

As environmental problems intensify, the chemical senses -that is smell and taste, are the most relevantsenses to evidence them.As such, environmental exposure vectors that can reach human beings comprise air,food, soil and water[1].Within this context, understanding the link between environmental exposures andhealth[2]is crucial to make informed choices, protect the environment and adapt to new environmentalconditions[3].Smell and taste lead therefore to multi-sensorial experiences which convey multi-layered information aboutlocal and global events[4]. However, these senses are usually absent when those problems are represented indigital systems. The multisensory HCIdesign framework investigateschemical sense inclusion withdigital systems[5]. Ongoing efforts tackledigitalization of smell and taste for digital delivery, transmission or substitution [6]. Despite experimentsproved technological feasibility, its dissemination depends on relevant applicationdevelopment[7].This thesis aims to fillthose gaps by demonstratinghow chemical senses provide the means to link environment and health based on scientific andgeolocation narratives [8], [9],[10]. We present a Multisensory HCI design process which accomplished symbolicdisplaying smell and taste and led us to a new multi-sensorial interaction system presented herein. We describe the conceptualization, design and evaluation of Earthsensum, an exploratory case study project.Earthsensumoffered to 16 participants in the study, environmental smell and taste experiences about real geolocations to participants of the study. These experiences were represented digitally using mobilevirtual reality (MVR) and mobile augmented reality (MAR). Its technologies bridge the real and digital Worlds through digital representations where we can reproduce the multi-sensorial experiences. Our study findings showed that the purposed interaction system is intuitive and can lead not only to a betterunderstanding of smell and taste perception as also of environmental problems. Participants comprehensionabout the link between environmental exposures and health was successful and they would recommend thissystem as education tools. Our conceptual design approach was validated and further developments wereencouraged.In this thesis,we demonstratehow to applyMultisensory HCI methodology to design with chemical senses. Weconclude that the presented symbolic representation model of smell and taste allows communicatingtheseexperiences on digital platforms. Due to its context-dependency, MVR and MAR platforms are adequatetechnologies to be applied for this purpose.Future developments intend to explore further the conceptual approach. These developments are centredon the use of the system to induce hopefully behaviourchange. Thisthesisopens up new application possibilities of digital chemical sense communication,Multisensory HCI Design and environmental health communication.À medida que os problemas ambientais se intensificam, os sentidos químicos -isto é, o cheiroe sabor, são os sentidos mais relevantes para evidenciá-los. Como tais, os vetores de exposição ambiental que podem atingir os seres humanos compreendem o ar, alimentos, solo e água [1]. Neste contexto, compreender a ligação entre as exposições ambientais e a saúde [2] é crucial para exercerescolhas informadas, proteger o meio ambiente e adaptar a novas condições ambientais [3]. O cheiroe o saborconduzemassima experiências multissensoriais que transmitem informações de múltiplas camadas sobre eventos locais e globais [4]. No entanto, esses sentidos geralmente estão ausentes quando esses problemas são representados em sistemas digitais. A disciplina do design de Interação Humano-Computador(HCI)multissensorial investiga a inclusão dossentidos químicos em sistemas digitais [9]. O seu foco atual residena digitalização de cheirose sabores para o envio, transmissão ou substituiçãode sentidos[10]. Apesar dasexperimentaçõescomprovarem a viabilidade tecnológica, a sua disseminação está dependentedo desenvolvimento de aplicações relevantes [11]. Estatese pretendepreencher estas lacunas ao demonstrar como os sentidos químicos explicitama interconexãoentre o meio ambiente e a saúde, recorrendo a narrativas científicas econtextualizadasgeograficamente[12], [13], [14]. Apresentamos uma metodologiade design HCImultissensorial que concretizouum sistema de representação simbólica de cheiro e sabor e nos conduziu a um novo sistema de interação multissensorial, que aqui apresentamos. Descrevemos o nosso estudo exploratório Earthsensum, que integra aconceptualização, design e avaliação. Earthsensumofereceu a 16participantes do estudo experiências ambientais de cheiro e sabor relacionadas com localizações geográficasreais. Essas experiências foram representadas digitalmente através derealidade virtual(VR)e realidade aumentada(AR).Estas tecnologias conectamo mundo real e digital através de representações digitais onde podemos reproduzir as experiências multissensoriais. Os resultados do nosso estudo provaramque o sistema interativo proposto é intuitivo e pode levar não apenas a uma melhor compreensão da perceção do cheiroe sabor, como também dos problemas ambientais. O entendimentosobre a interdependência entre exposições ambientais e saúde teve êxitoe os participantes recomendariam este sistema como ferramenta para aeducação. A nossa abordagem conceptual foi positivamentevalidadae novos desenvolvimentos foram incentivados. Nesta tese, demonstramos como aplicar metodologiasde design HCImultissensorialpara projetar com ossentidos químicos. Comprovamosque o modelo apresentado de representação simbólica do cheiroe do saborpermite comunicar essas experiênciasem plataformas digitais. Por serem dependentesdocontexto, as plataformas de aplicações emVR e AR são tecnologias adequadaspara este fim.Desenvolvimentos futuros pretendem aprofundar a nossa abordagemconceptual. Em particular, aspiramos desenvolvera aplicaçãodo sistema para promover mudanças de comportamento. Esta tese propõenovas possibilidades de aplicação da comunicação dos sentidos químicos em plataformas digitais, dedesign multissensorial HCI e de comunicação de saúde ambiental

Interactive Fiction in Cinematic Virtual Reality: Epistemology, Creation and Evaluation

This dissertation presents the Interactive Fiction in Cinematic Virtual Reality (IFcVR), an interactive digital narrative (IDN) that brings together the cinematic virtual reality (cVR) and the creation of virtual environments through 360\ub0 video within an interactive fiction (IF) structure. This work is structured in three components: an epistemological approach to this kind of narrative and media hybrid; the creation process of IFcVR, from development to postproduction; and user evaluation of IFcVR. In order to set the foundations for the creation of interactive VR fiction films, I dissect the IFcVR by investigating the aesthetics, narratological and interactive notions that converge and diverge in it, proposing a medium-conscious narratology for this kind of artefact. This analysis led to the production of an IFcVR functional prototype: \u201cZENA\u201d, the first interactive VR film shot in Genoa. ZENA\u2019s creation process is reported proposing some guidelines for interactive and immersive film-makers. In order to evaluate the effectiveness of the IFcVR as an entertaining narrative form and a vehicle for diverse types of messages, this study also proposes a methodology to measure User Experience (UX) on IFcVR. The full evaluation protocol gathers both qualitative and quantitative data through ad hoc instruments. The proposed protocol is illustrated through its pilot application on ZENA. Findings show interactors' positive acceptance of IFcVR as an entertaining experience

Towards Understanding and Expanding Locomotion in Physical and Virtual Realities

Among many virtual reality interactions, the locomotion dilemma remains a significant impediment to achieving an ideal immersive experience. The physical limitations of tracked space make it impossible to naturally explore theoretically boundless virtual environments with a one-to-one mapping. Synthetic techniques like teleportation and flying often induce simulator sickness and break the sense of presence. Therefore, natural walking is the most favored form of locomotion. Redirected walking offers a more natural and intuitive way for users to navigate vast virtual spaces efficiently. However, existing techniques either lead to simulator sickness due to visual and vestibular mismatch or detract users from the immersive experience that virtual reality aims to provide. This research presents innovative techniques and applications to enhance the user experience by expanding walkable, physical space in Virtual Reality. The thesis includes three main contributions. The first contribution proposes a mobile application that uses markerless Augmented Reality to allow users to explore a life-sized virtual library through a divide-and-rule approach. The second contribution presents a subtle redirected walking technique based on inattentional blindness, using dynamic foveated rendering and natural visual suppressions like blinks and saccades. Finally, the third contribution introduces a novel redirected walking solution that leverages a deep neural network, to predict saccades in real-time and eliminate the hardware requirements for eye-tracking. Overall, this thesis offers valuable contributions to human-computer interaction, investigating novel approaches to solving the locomotion dilemma. The proposed solutions were evaluated through extensive user studies, demonstrating their effectiveness and applicability in real-world scenarios like training simulations and entertainment

Investigating Visitors’ Behavioural Response to Virtual Reality (VR) Retail Environments

This thesis aimed to explore the influence of Virtual Reality (VR) retail environment cues on visitors’ behavioural intentions in the context of urban shopping destinations. The Stimulus-Organism-Response (S-O-R) theory (Mehrabian and Russell, 1974) were employed as a theoretical foundation given that many researchers have provided substantial evidence on its suitability as a theoretical framework to investigate the effects of various physical and virtual (online/mobile) retail environment cues on human behaviour through the mediating variables of affective and cognitive states. The first step to achieving the aim was to determine the current state of knowledge surrounding this topic. Therefore, relevant literature over four key topic areas was critically reviewed including 1) urban place marketing, particularly within the broader context of urban tourism, 2) retail store environment literature extending to electronic and mobile commerce research, 3) immersive technology, and 4) technology adoption (Objective 1). Then, primary data were collected in two research phases. Research Phase 1 entailed two sets of semi-structured interviews. First, a contrasting case-based approach was employed, and interviews were conducted with seven urban place marketers from three urban place marketing organisations in four urban shopping destinations in the UK (Objective 2). The findings provided initial insights into urban place marketers’ views on the perceived barriers, benefits, internal organisational capability and external pressures associated with immersive technology (VR and Augmented Reality) implementation and their overall intention to adopt these innovative technologies for city marketing. This data was analysed using thematic analysis and four themes and eleven sub-themes emerged including three new context-specific sub-themes (technology access, organisational readiness and industry readiness). Based on these findings, twelve semi-structured interviews with visitors to Manchester City Centre were gathered and aimed to draw out context-specific themes and sub-themes. This data was also analysed using thematic analysis and overall, three themes and thirteen sub-themes emerged including four new sub-themes (virtual aesthetics, virtual atmospherics, social presence and layout design) under the main theme of VR retail environment cues. Accordingly, ten hypotheses were proposed and informed the development of a qualitative VR Visitor Behaviour Model based on S-O-R theory. The aim of Research Phase 2 was to validate the proposed model (Objective 3). To achieve this, survey data were gathered from 150 potential visitors to urban shopping destinations and the data were analysed using Partial Least Squares Structural Equation Modelling (PLS-SEM) (Objective 3). From these findings, a final VR Visitor Behaviour S-O-R Model was proposed, which is the main contribution of this study (Objective 4). The thesis concludes by outlining the theoretical contributions of this research and providing guidance to urban place marketers and VR developers and designers. Finally, avenues for further research based on the identified study limitations are offered

Nouvelles approches pour une exploitation efficace des comportements et interactions de l'humain dans l'environnement virtuel

La création des moyens permettant à l’humain de se retrouver dans un endroit différent du monde réel n’est pas une nouveauté. Les outils comme les peintures ou les livres permettent tout autant de transporter l’humain par imagination dans un monde fictif afin de lui permettre de s’évader de la vie réelle. C’est également ce que permettent les technologies de la réalité virtuelle. Cependant, les environnements de la réalité virtuelle sont des représentations 3D numériques exploitables à l’échelle humaine. Ils peuvent stimuler au maximum les habiletés perceptives de l’humain à travers ses canaux sensoriels, tout en lui permettant d’influencer activement le déroulement des évènements qui y ont lieu. De ce fait, ils représentent un atout majeur pour la représentation de l’information à des fins diverses. Dans cette thèse, nous nous sommes intéressés à l’exploitation de la réalité virtuelle dans le cadre de la thérapie par exposition pour le traitement du trouble de stress post-traumatique. En faisant une revue de littérature dans ce domaine, nous avons constaté le rôle essentiellement passif qui est attribué au patient. En effet, les études antérieures se sont focalisées sur la composante « modélisation 3D » de la réalité virtuelle ainsi que sur la stimulation sensorielle, avec pour but principal d’aider le patient à narrer son trauma. Cela est fait au détriment d’une des caractéristiques principales de la réalité virtuelle qu’est l’interaction du sujet avec l’environnement virtuel. Ainsi, au vu des avantages qu’elles offrent, nous pensons que les technologies de la réalité virtuelle devraient être utilisées pour permettre au patient de revivre son trauma différemment au moyen de l’interaction. Alors, nous avons proposé des approches basées sur le modèle de référence de la réalité virtuelle. Ces approches exploitent les techniques de gamification avancées des jeux sérieux ajoutées à la technique d’ajustement dynamique de la difficulté empruntée au domaine du jeu vidéo, pour implémenter les interactions naturelles de l’humain avec l’environnement virtuel. Nous avons appliqué ces approches à un simulateur de conduite de camion en réalité virtuelle pour traiter les camionneurs souffrant de trouble de stress post-traumatique. Les résultats d’évaluation d’utilisabilité du simulateur conformément à ces approches y sont présentés : le sentiment de présence a été atteint à travers cet outil et les participants ont trouvé la qualité des interactions avec l’environnement virtuel optimale. En perspectives, nous avons décrit l’application clinique expérimentale qui sera effectuée avec des patients en résidence à « La Futaie », centre de thérapie avec lequel nous avons collaboré tout au long de cette recherche. Creating the means for humans to find themselves in a place different from the real world is nothing new. Tools such as paintings or books also allow humans to be transported by imagination into a fictitious world in order to escape from real life. This is also what virtual reality technologies allow. However, virtual reality environments are digital 3D environments usable on a human scale. They can maximize the perceptual skills of humans through their sensory channels while allowing them to actively influence the course of events that take place in these environments. Therefore, they are a major asset for the representation of information for various purposes. In this thesis, we focused on the use of virtual reality as part of exposure therapy for the treatment of post-traumatic stress disorder. By reviewing the literature in that area, we noted the essentially passive role attributed to the patient. Indeed, previous studies have focused on the "3D modeling" component of virtual reality as well as on sensory stimulation, with the main goal of helping the patient to narrate his trauma. That is done to the detriment of one of the main characteristics of virtual reality, which is the subject's interaction with the virtual world. So, given the benefits they offer, we believe that virtual reality technologies should be used to allow the patient to relive his trauma differently through interaction. So, we have proposed approaches based on the virtual reality reference model. These approaches exploit the advanced gamification techniques of serious games combined with the dynamic difficulty adjustment technique borrowed from the field of video games, to implement the natural interactions of humans with the virtual environment. We applied these approaches to a virtual reality truck driving simulator to treat truckers with post-traumatic stress disorder. The simulator usability evaluation results in accordance with these approaches are presented: the feeling of presence was achieved through this tool and the participants found the quality of interactions with the virtual world to be optimal. In perspective, we have described the experimental clinical application that will be carried out with patients in residence at "La Futaie", the therapy center with which we have collaborated throughout this research