Virtual Reality in Mathematics Education (VRiME):An exploration of the integration and design of virtual reality for mathematics education

This thesis explores the use of Virtual Reality (VR) in mathematics education. Four VR prototypes were designed and developed during the PhD project to teach equations, geometry, and vectors and facilitate collaboration.Paper A investigates asymmetric VR for classroom integration and collaborative learning and presents a new taxonomy of asymmetric interfaces. Paper B proposes how VR could assist students with Autism Spectrum Disorder (ASD) in learning daily living skills involving basic mathematical concepts. Paper C investigates how VR could enhance social inclusion and mathematics learning for neurodiverse students. Paper D presents a VR prototype for teaching algebra and equation-solving strategies, noting positive student responses and the potential for knowledge transfer. Paper E investigates gesture-based interaction with dynamic geometry in VR for geometry education and presents a new taxonomy of learning environments. Finally, paper F explores the use of VR to visualise and contextualise mathematical concepts to teach software engineering students.The thesis concludes that VR offers promising avenues for transforming mathematics education. It aims to broaden our understanding of VR's educational potential, paving the way for more immersive learning experiences in mathematics education

Coping with the inheritance of COVID-19: the role of new interactive technologies to enhance user experience in different contexts of use

The COVID-19 pandemic has upset the habits of people and various sectors of society, including training, entertainment, and retail. These sectors have been forced to adapt to abnormal situations such as social distancing, remote work, and online entertainment. The pandemic has significantly transformed the training field, leading to the closure of many in-person instruction centers and a shift toward online education courses, which can be less effective. In addition, the entertainment industry has been heavily transformed by social distancing, resulting in the cancellation of many live events and the closure of several cinemas. This has increased demand for online entertainment options, such as streaming services and virtual events. Finally, the restrictions imposed by the COVID-19 pandemic substantially impacted physical stores and fairs, suspending exhibitions for more than two years. This has further driven consumers to rely on e-commerce to fulfill their purchasing and companies to increasingly take advantage of new technologies such as augmented reality. In this suddenly disrupted scenario, new technologies have the potential to fill the gap generated by the pandemic, functioning as an interactive bridge to connect people. This Ph.D. thesis explored the potential of interactive technologies in mitigating the challenges posed by the COVID-19 pandemic in various contexts of use in the above-mentioned areas. Specifically, three lines of research were investigated by conducting different studies using a mixed approach in the Human-Computer Interaction field. The first research line focused on the study of immersive virtual reality training, with a particular interest in flood emergencies, a growing phenomenon. The goal was to implement engaging and efficient training for citizens that live near rivers through a human-centric design approach. The second line of research explored innovative ways to improve social interaction and collaboration in the entertainment sector, highlighting guidelines for the design of shared streaming experiences. In particular, three different communication modalities were studied during group viewing of an interactive film on a streaming platform. Finally, the third research line focused on the retail sector. On the one hand, the focus consisted of understanding which aspects of the 3D web and AR technology are helpful for supporting small businesses and trade fairs. On the other hand, the focus was to investigate how to support consumers during an AR shopping experience when interacting with 3D virtual products of different sizes. Overall, this project provides suggestions and guidelines for designing systems that can both increasingly connect people at a distance and offer new hybrid worlds. In addition, this project expands state-of-the-art related to interactive technologies and offers generalizable results outside the crisis created by COVID-19. These technologies, now increasingly integrated into everyday life, can be a tool for empowerment and resilience, improving people's lives.The COVID-19 pandemic has upset the habits of people and various sectors of society, including training, entertainment, and retail. These sectors have been forced to adapt to abnormal situations such as social distancing, remote work, and online entertainment. The pandemic has significantly transformed the training field, leading to the closure of many in-person instruction centers and a shift toward online education courses, which can be less effective. In addition, the entertainment industry has been heavily transformed by social distancing, resulting in the cancellation of many live events and the closure of several cinemas. This has increased demand for online entertainment options, such as streaming services and virtual events. Finally, the restrictions imposed by the COVID-19 pandemic substantially impacted physical stores and fairs, suspending exhibitions for more than two years. This has further driven consumers to rely on e-commerce to fulfill their purchasing and companies to increasingly take advantage of new technologies such as augmented reality. In this suddenly disrupted scenario, new technologies have the potential to fill the gap generated by the pandemic, functioning as an interactive bridge to connect people. This Ph.D. thesis explored the potential of interactive technologies in mitigating the challenges posed by the COVID-19 pandemic in various contexts of use in the above-mentioned areas. Specifically, three lines of research were investigated by conducting different studies using a mixed approach in the Human-Computer Interaction field. The first research line focused on the study of immersive virtual reality training, with a particular interest in flood emergencies, a growing phenomenon. The goal was to implement engaging and efficient training for citizens that live near rivers through a human-centric design approach. The second line of research explored innovative ways to improve social interaction and collaboration in the entertainment sector, highlighting guidelines for the design of shared streaming experiences. In particular, three different communication modalities were studied during group viewing of an interactive film on a streaming platform. Finally, the third research line focused on the retail sector. On the one hand, the focus consisted of understanding which aspects of the 3D web and AR technology are helpful for supporting small businesses and trade fairs. On the other hand, the focus was to investigate how to support consumers during an AR shopping experience when interacting with 3D virtual products of different sizes. Overall, this project provides suggestions and guidelines for designing systems that can both increasingly connect people at a distance and offer new hybrid worlds. In addition, this project expands state-of-the-art related to interactive technologies and offers generalizable results outside the crisis created by COVID-19. These technologies, now increasingly integrated into everyday life, can be a tool for empowerment and resilience, improving people's lives

The Perception/Action loop: A Study on the Bandwidth of Human Perception and on Natural Human Computer Interaction for Immersive Virtual Reality Applications

Virtual Reality (VR) is an innovating technology which, in the last decade, has had a widespread success, mainly thanks to the release of low cost devices, which have contributed to the diversification of its domains of application. In particular, the current work mainly focuses on the general mechanisms underling perception/action loop in VR, in order to improve the design and implementation of applications for training and simulation in immersive VR, especially in the context of Industry 4.0 and the medical field. On the one hand, we want to understand how humans gather and process all the information presented in a virtual environment, through the evaluation of the visual system bandwidth. On the other hand, since interface has to be a sort of transparent layer allowing trainees to accomplish a task without directing any cognitive effort on the interaction itself, we compare two state of the art solutions for selection and manipulation tasks, a touchful one, the HTC Vive controllers, and a touchless vision-based one, the Leap Motion. To this aim we have developed ad hoc frameworks and methodologies. The software frameworks consist in the creation of VR scenarios, where the experimenter can choose the modality of interaction and the headset to be used and set experimental parameters, guaranteeing experiments repeatability and controlled conditions. The methodology includes the evaluation of performance, user experience and preferences, considering both quantitative and qualitative metrics derived from the collection and the analysis of heterogeneous data, as physiological and inertial sensors measurements, timing and self-assessment questionnaires. In general, VR has been found to be a powerful tool able to simulate specific situations in a realistic and involving way, eliciting user\u2019s sense of presence, without causing severe cybersickness, at least when interaction is limited to the peripersonal and near-action space. Moreover, when designing a VR application, it is possible to manipulate its features in order to trigger or avoid triggering specific emotions and voluntarily create potentially stressful or relaxing situations. Considering the ability of trainees to perceive and process information presented in an immersive virtual environment, results show that, when people are given enough time to build a gist of the scene, they are able to recognize a change with 0.75 accuracy when up to 8 elements are in the scene. For interaction, instead, when selection and manipulation tasks do not require fine movements, controllers and Leap Motion ensure comparable performance; whereas, when tasks are complex, the first solution turns out to be more stable and efficient, also because visual and audio feedback, provided as a substitute of the haptic one, does not substantially contribute to improve performance in the touchless case

Prefrontal cortex activation upon a demanding virtual hand-controlled task: A new frontier for neuroergonomics

open9noFunctional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.openCarrieri, Marika; Petracca, Andrea; Lancia, Stefania; Basso Moro, Sara; Brigadoi, Sabrina; Spezialetti, Matteo; Ferrari, Marco; Placidi, Giuseppe; Quaresima, ValentinaCarrieri, Marika; Petracca, Andrea; Lancia, Stefania; BASSO MORO, Sara; Brigadoi, Sabrina; Spezialetti, Matteo; Ferrari, Marco; Placidi, Giuseppe; Quaresima, Valentin

Measuring user experience for virtual reality

In recent years, Virtual Reality (VR) and 3D User Interfaces (3DUI) have seen a drastic increase in popularity, especially in terms of consumer-ready hardware and software. These technologies have the potential to create new experiences that combine the advantages of reality and virtuality. While the technology for input as well as output devices is market ready, only a few solutions for everyday VR - online shopping, games, or movies - exist, and empirical knowledge about performance and user preferences is lacking. All this makes the development and design of human-centered user interfaces for VR a great challenge. This thesis investigates the evaluation and design of interactive VR experiences. We introduce the Virtual Reality User Experience (VRUX) model based on VR-specific external factors and evaluation metrics such as task performance and user preference. Based on our novel UX evaluation approach, we contribute by exploring the following directions: shopping in virtual environments, as well as text entry and menu control in the context of everyday VR. Along with this, we summarize our findings by design spaces and guidelines for choosing optimal interfaces and controls in VR.In den letzten Jahren haben Virtual Reality (VR) und 3D User Interfaces (3DUI) stark an Popularität gewonnen, insbesondere bei Hard- und Software im Konsumerbereich. Diese Technologien haben das Potenzial, neue Erfahrungen zu schaffen, die die Vorteile von Realität und Virtualität kombinieren. Während die Technologie sowohl für Eingabe- als auch für Ausgabegeräte marktreif ist, existieren nur wenige Lösungen für den Alltag in VR - wie Online-Shopping, Spiele oder Filme - und es fehlt an empirischem Wissen über Leistung und Benutzerpräferenzen. Dies macht die Entwicklung und Gestaltung von benutzerzentrierten Benutzeroberflächen für VR zu einer großen Herausforderung. Diese Arbeit beschäftigt sich mit der Evaluation und Gestaltung von interaktiven VR-Erfahrungen. Es wird das Virtual Reality User Experience (VRUX)- Modell eingeführt, das auf VR-spezifischen externen Faktoren und Bewertungskennzahlen wie Leistung und Benutzerpräferenz basiert. Basierend auf unserem neuartigen UX-Evaluierungsansatz leisten wir einen Beitrag, indem wir folgende interaktive Anwendungsbereiche untersuchen: Einkaufen in virtuellen Umgebungen sowie Texteingabe und Menüsteuerung im Kontext des täglichen VR. Die Ergebnisse werden außerdem mittels Richtlinien zur Auswahl optimaler Schnittstellen in VR zusammengefasst

Shopping in a Virtual Reality: A Stimulus-Organism-Response (S-O-R) based Systematic Literature Review

The virtual reality (VR) technology has become increasingly relevant in recent years due to the advantages that occur through the representation of the real within a virtual world. One promising application area of VR is virtual shopping, i.e., providing customers with the ability to visit virtual stores instead of purchasing products on 2D websites or brick-and-mortar stores. To gain insights into the opportunities of the technology in the realm of shopping, we emphasize and discuss the literature on the use of VR for shopping based on a systematic literature review, thereby deepening our understanding of virtual retail concepts and synthesizing the empirical evidence on the advantages of VR. For this purpose, we developed a Stimulus-Organism-Response (S-O-R) based framework to summarize the key findings. The results comprise suggestions for VR shopping applications as well as possible future research avenues

Immersive Technologies in Virtual Companions: A Systematic Literature Review

The emergence of virtual companions is transforming the evolution of intelligent systems that effortlessly cater to the unique requirements of users. These advanced systems not only take into account the user present capabilities, preferences, and needs but also possess the capability to adapt dynamically to changes in the environment, as well as fluctuations in the users emotional state or behavior. A virtual companion is an intelligent software or application that offers support, assistance, and companionship across various aspects of users lives. Various enabling technologies are involved in building virtual companion, among these, Augmented Reality (AR), and Virtual Reality (VR) are emerging as transformative tools. While their potential for use in virtual companions or digital assistants is promising, their applications in these domains remain relatively unexplored. To address this gap, a systematic review was conducted to investigate the applications of VR, AR, and MR immersive technologies in the development of virtual companions. A comprehensive search across PubMed, Scopus, and Google Scholar yielded 28 relevant articles out of a pool of 644. The review revealed that immersive technologies, particularly VR and AR, play a significant role in creating digital assistants, offering a wide range of applications that brings various facilities in the individuals life in areas such as addressing social isolation, enhancing cognitive abilities and dementia care, facilitating education, and more. Additionally, AR and MR hold potential for enhancing Quality of life (QoL) within the context of virtual companion technology. The findings of this review provide a valuable foundation for further research in this evolving field

An Overview of Self-Adaptive Technologies Within Virtual Reality Training

This overview presents the current state-of-the-art of self-adaptive technologies within virtual reality (VR) training. Virtual reality training and assessment is increasingly used for five key areas: medical, industrial & commercial training, serious games, rehabilitation and remote training such as Massive Open Online Courses (MOOCs). Adaptation can be applied to five core technologies of VR including haptic devices, stereo graphics, adaptive content, assessment and autonomous agents. Automation of VR training can contribute to automation of actual procedures including remote and robotic assisted surgery which reduces injury and improves accuracy of the procedure. Automated haptic interaction can enable tele-presence and virtual artefact tactile interaction from either remote or simulated environments. Automation, machine learning and data driven features play an important role in providing trainee-specific individual adaptive training content. Data from trainee assessment can form an input to autonomous systems for customised training and automated difficulty levels to match individual requirements. Self-adaptive technology has been developed previously within individual technologies of VR training. One of the conclusions of this research is that while it does not exist, an enhanced portable framework is needed and it would be beneficial to combine automation of core technologies, producing a reusable automation framework for VR training