Towards Naturalistic Interfaces of Virtual Reality Systems

Interaction plays a key role in achieving realistic experience in virtual reality (VR). Its realization depends on interpreting the intents of human motions to give inputs to VR systems. Thus, understanding human motion from the computational perspective is essential to the design of naturalistic interfaces for VR. This dissertation studied three types of human motions, including locomotion (walking), head motion and hand motion in the context of VR. For locomotion, the dissertation presented a machine learning approach for developing a mechanical repositioning technique based on a 1-D treadmill for interacting with a unique new large-scale projective display, called the Wide-Field Immersive Stereoscopic Environment (WISE). The usability of the proposed approach was assessed through a novel user study that asked participants to pursue a rolling ball at variable speed in a virtual scene. In addition, the dissertation studied the role of stereopsis in avoiding virtual obstacles while walking by asking participants to step over obstacles and gaps under both stereoscopic and non-stereoscopic viewing conditions in VR experiments. In terms of head motion, the dissertation presented a head gesture interface for interaction in VR that recognizes real-time head gestures on head-mounted displays (HMDs) using Cascaded Hidden Markov Models. Two experiments were conducted to evaluate the proposed approach. The first assessed its offline classification performance while the second estimated the latency of the algorithm to recognize head gestures. The dissertation also conducted a user study that investigated the effects of visual and control latency on teleoperation of a quadcopter using head motion tracked by a head-mounted display. As part of the study, a method for objectively estimating the end-to-end latency in HMDs was presented. For hand motion, the dissertation presented an approach that recognizes dynamic hand gestures to implement a hand gesture interface for VR based on a static head gesture recognition algorithm. The proposed algorithm was evaluated offline in terms of its classification performance. A user study was conducted to compare the performance and the usability of the head gesture interface, the hand gesture interface and a conventional gamepad interface for answering Yes/No questions in VR. Overall, the dissertation has two main contributions towards the improvement of naturalism of interaction in VR systems. Firstly, the interaction techniques presented in the dissertation can be directly integrated into existing VR systems offering more choices for interaction to end users of VR technology. Secondly, the results of the user studies of the presented VR interfaces in the dissertation also serve as guidelines to VR researchers and engineers for designing future VR systems

Expert evaluation of aspects related to virtual reality systems and suggestions for future studies

Abstract. In this bachelor’s thesis, we review existing quantitative and qualitative research on virtual reality systems. We then present suggestions for performing a future study to combine the objective and subjective measurements of virtual reality experience. Additionally, we adapted an existing heuristics-based expert evaluation method to suit evaluating virtual reality systems. Using our method, we performed the expert evaluation on a selection of five modern consumer virtual reality systems to understand the connections between the subjective experience and the physical variables related to the virtual reality system. From this evaluation, we present findings that are used to construct discussion and to draw conclusions on these said connections. We found the most prominent conclusion to be that the experience of virtual reality is highly subjective and dependent on the content being viewed in virtual reality. Additionally, we concluded that some of the most important aspects in need of improvement are display resolution, lens design, user ergonomics, and lack of wirelessness. Finally, we state that two optimization problems are present; the first one being the optimization required to design a virtual reality system and the second one being the act of choosing a system to match a consumer’s preferred content.Tiivistelmä. Tässä kandidaatin tutkielmassa käymme läpi aiempaa kvantitatiivista ja kvalitatiivista tutkimusta virtuaalitodellisuusjärjestelmistä. Esitämme myös ehdotuksia myöhempää tutkimusta varten virtuaalitodellisuuteen liittyvien objektiivisten ja subjektiivisten mittausten yhdistämiseksi. Tämän lisäksi adaptoimme aiemman heuristiikkapohjaisen asiantuntija-arvioinnin sopimaan virtuaalitodellisuusjärjestelmien arviointiin. Käyttäen metodiamme toteutimme asiantuntija-arvioinnin viidellä modernilla kuluttajakäyttöön tarkoitetulla virtuaalitodellisuusjärjestelmällä ymmärtääksemme yhteyksiä subjektiivisen kokemuksen ja niiden fysikaalisten muuttujien välillä, jotka liittyvät virtuaalitodellisuusjärjestelmiin. Esitämme tämän asiantuntija-arvioinnin löydöksiä, ja luomme niiden avulla keskustelua, jonka avulla teemme mainittuihin yhteyksiin liittyviä johtopäätöksiä. Tärkein johtopäätöksemme oli se, että virtuaalitodellisuuden kokemus on erittäin subjektiivinen ja riippuvainen siitä sisällöstä, jota virtuaalitodellisuudessa koetaan. Aiemman lisäksi toteamme, että merkittävimpiä kehitystä kaipaavia osa-alueita ovat näytön resoluutio, linssien suunnittelu, käyttäjäergonomia ja langattomuuden puute. Viimeisenä totesimme, että virtuaalitodellisuusjärjestelmiin liittyy kaksi optimointiongelmaa; ensimmäinen liittyy järjestelmän suunnittelussa tapahtuvaan optimointiin, ja toinen liittyy sellaisen järjestelmän valitsemiseen, joka sopii kunkin kuluttajan suosimaan tarkoitukseen

Evaluation of a Low-Cost Virtual Reality Surround-Screen Projection System

[EN] Two of the most popular mediums for virtual reality are head-mounted displays and surround-screen projection systems, such as CAVE Automatic Virtual Environments. In recent years, HMDs suffered a significant reduction in cost and have become widespread consumer products. In contrast, CAVEs are still expensive and remain accessible to a limited number of researchers. This study aims to evaluate both objective and subjective characteristics of a CAVE-like monoscopic low-cost virtual reality surround-screen projection system compared to advanced setups and HMDs. For objective results, we measured the head position estimation accuracy and precision of a low-cost active infrared (IR) based tracking system, used in the proposed low-cost CAVE, relatively to an infrared marker-based tracking system, used in a laboratory-grade CAVE system. For subjective characteristics, we investigated the sense of presence and cybersickness elicited in users during a visual search task outside personal space, beyond arms reach, where the importance of stereo vision is diminished. Thirty participants rated their sense of presence and cybersickness after performing the VR search task with our CAVE-like system and a modern HMD. The tracking showed an accuracy error of 1.66 cm and .4 mm of precision jitter. The system was reported to elicit presence but at a lower level than the HMD, while causing significant lower cybersickness. Our results were compared to a previous study performed with a laboratory-grade CAVE and support that a VR system implemented with low-cost devices could be a viable alternative to laboratory-grade CAVEs for visual search tasks outside the users personal space.This work was supported by the Fundação para a Ciência e Tecnologia through the AHA project (CMUPERI/HCI/0046/2013), by the INTERREG program through the MACBIOIDI project (MAC/1.1.b/098), LARSyS (UIDB/50009/2020), NOVA-LINCS (UID/CEC/04516/2019), by Fundació la Marató de la TV3 (201701-10), and the European Union through the Operational Program of the European Regional Development Fund (ERDF) of the Valencian Community 2014-2020 (IDIFEDER/2018/029)Gonçalves, A.; Borrego, A.; Latorre, J.; Llorens Rodríguez, R.; Bermúdez, S. (2021). Evaluation of a Low-Cost Virtual Reality Surround-Screen Projection System. IEEE Transactions on Visualization and Computer Graphics. 1-12. https://doi.org/10.1109/TVCG.2021.3091485S11

A perspective review on integrating VR/AR with haptics into STEM education for multi-sensory learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.European Union through the Erasmus+ Program under Grant 2020-1-NO01-KA203-076540, project title Integrating virtual and AUGMENTED reality with WEARable technology into engineering EDUcation (AugmentedWearEdu), https://augmentedwearedu.uia.no/ [34] (accessed on 27 March 2022). This work was also supported by the Top Research Centre Mechatronics (TRCM), University of Agder (UiA), Norwa

A Perspective Review on Integrating VR/AR with Haptics into STEM Education for Multi-Sensory Learning

As a result of several governments closing educational facilities in reaction to the COVID-19 pandemic in 2020, almost 80% of the world’s students were not in school for several weeks. Schools and universities are thus increasing their efforts to leverage educational resources and provide possibilities for remote learning. A variety of educational programs, platforms, and technologies are now accessible to support student learning; while these tools are important for society, they are primarily concerned with the dissemination of theoretical material. There is a lack of support for hands-on laboratory work and practical experience. This is particularly important for all disciplines related to science, technology, engineering, and mathematics (STEM), where labs and pedagogical assets must be continuously enhanced in order to provide effective study programs. In this study, we describe a unique perspective to achieving multi-sensory learning through the integration of virtual and augmented reality (VR/AR) with haptic wearables in STEM education. We address the implications of a novel viewpoint on established pedagogical notions. We want to encourage worldwide efforts to make fully immersive, open, and remote laboratory learning a reality.publishedVersio