Design Space of Visual Feedforward And Corrective Feedback in XR-Based Motion Guidance Systems

Extended reality (XR) technologies are highly suited in assisting individuals in learning motor skills and movements -- referred to as motion guidance. In motion guidance, the "feedforward" provides instructional cues of the motions that are to be performed, whereas the "feedback" provides cues which help correct mistakes and minimize errors. Designing synergistic feedforward and feedback is vital to providing an effective learning experience, but this interplay between the two has not yet been adequately explored. Based on a survey of the literature, we propose design space for both motion feedforward and corrective feedback in XR, and describe the interaction effects between them. We identify common design approaches of XR-based motion guidance found in our literature corpus, and discuss them through the lens of our design dimensions. We then discuss additional contextual factors and considerations that influence this design, together with future research opportunities for motion guidance in XR.Comment: To appear in ACM CHI 202

Serious game in VR to improve the visuospatial ability

Treball final de Grau en Disseny i Desenvolupament de Videojocs. Codi: VJ1241. Curs acadèmic: 2022/2023This document presents the project report of the Video Games Design and Development Degree Final project conducted by Óscar Silvestre Payá. The project focuses on the development of a serious game leveraging virtual reality (VR) technology, with the explicit purpose of enhancing diverse visuospatial abilities inherent to individuals. The core gameplay revolves around a series of intricately designed mini-games, which engage players in the application of logical thinking and ingenuity, thereby facilitating substantial improvements in their visual-spatial capabilities. Moreover, the project endeavors to seamlessly integrate psychoeducational principles into the gaming experience, engendering an effective serious game that engrosses players within an immersive virtual reality environment. To ascertain progress in the aforementioned abilities, the game incorporates parameter tracking and assessment mechanisms. The development process entailed the utilization of Unity 3D, today’s most widely used game engine, in conjunction with a VR device, to deliver an elevated computerbased gameplay experience. By embracing innovative technologies and adopting a multifaceted approach that combines gaming elements with educational objectives, this project contributes to the expanding realm of serious games, while simultaneously offering a captivating avenue for players to augment their visual-spatial proficiencies

Paralympic VR: an immersive experience

Paralympic VR: an immersive experience is based on exploratory research and consists in the production of a 360 video-prototype that aims to give special access to the wheelchair basketball training universe, bringing the users close to Paralympic athletes without the need to hide the impairment or hyper focus on the stories behind the disability. Media coverage of Paralympic sports and the representation of disabled athletes have been changing, but still present a medicalised framing, a narrative where the athletes are commonly portrayed as heroes due to the overcoming of difficulties related to the impairment and not to the sports results. This present project work considers immersive journalism and the use of virtual reality technologies as the means by which an alternative approach and a better representation of Paralympians are possible. The project presents the description of the workflow process, detailing conceptualization decisions, technical aspects and, mainly, the challenges and lessons learned. A user study combining focus groups and online survey was also conducted in order to have insights about the experience of watching the video with the headset Samsung Gear VR. Feedback provided by participants indicated that the prototype accomplished its aim. Recommendations to improve future work were also gathered.Paralympic VR: an immersive experience baseia-se em investigação exploratória e consiste na produção de um vídeo 360 graus que visa dar acesso especial ao universo de treino do basquete em cadeira de rodas, aproximando os utilizadores dos atletas paralímpicos sem a necessidade de ocultar a deficiência ou dar grande enfoque às histórias por detrás da deficiência. A cobertura do desporto paralímpico nos media e a representação dos atletas com deficiência está a transformar-se, mas ainda apresenta uma abordagem medicalizada, uma narrativa em que os atletas são comumente retratados como heróis devido à superação de dificuldades relacionadas à deficiência e não em virtude aos resultados desportivos. O jornalismo imersivo e o uso de tecnologias de realidade virtual são considerados no presente trabalho de projeto os meios pelos quais é possível uma abordagem alternativa e uma melhor representação dos atletas paralímpicos. O projeto apresenta a descrição de todo o processo de produção do vídeo-protótipo, detalhando decisões de conceito, aspectos técnicos e, principalmente, os desafios e lições aprendidas. Um estudo com utilizadores combinando grupos de foco e questionário on-line também foi conduzido para a recolha de perceções sobre a experiência de assistir ao vídeo com o headset Samsung Gear VR. Os comentários fornecidos pelos participantes indicaram que o protótipo atingiu seu objetivo. Importantes recomendações para a melhoria de futuras experiências também foram reunidas

Motor learning induced neuroplasticity in minimally invasive surgery

Technical skills in surgery have become more complex and challenging to acquire since the introduction of technological aids, particularly in the arena of Minimally Invasive Surgery. Additional challenges posed by reforms to surgical careers and increased public scrutiny, have propelled identification of methods to assess and acquire MIS technical skills. Although validated objective assessments have been developed to assess motor skills requisite for MIS, they poorly understand the development of expertise. Motor skills learning, is indirectly observable, an internal process leading to relative permanent changes in the central nervous system. Advances in functional neuroimaging permit direct interrogation of evolving patterns of brain function associated with motor learning due to the property of neuroplasticity and has been used on surgeons to identify the neural correlates for technical skills acquisition and the impact of new technology. However significant gaps exist in understanding neuroplasticity underlying learning complex bimanual MIS skills. In this thesis the available evidence on applying functional neuroimaging towards assessment and enhancing operative performance in the field of surgery has been synthesized. The purpose of this thesis was to evaluate frontal lobe neuroplasticity associated with learning a complex bimanual MIS skill using functional near-infrared spectroscopy an indirect neuroimaging technique. Laparoscopic suturing and knot-tying a technically challenging bimanual skill is selected to demonstrate learning related reorganisation of cortical behaviour within the frontal lobe by shifts in activation from the prefrontal cortex (PFC) subserving attention to primary and secondary motor centres (premotor cortex, supplementary motor area and primary motor cortex) in which motor sequences are encoded and executed. In the cross-sectional study, participants of varying expertise demonstrate frontal lobe neuroplasticity commensurate with motor learning. The longitudinal study involves tracking evolution in cortical behaviour of novices in response to receipt of eight hours distributed training over a fortnight. Despite novices achieving expert like performance and stabilisation on the technical task, this study demonstrates that novices displayed persistent PFC activity. This study establishes for complex bimanual tasks, that improvements in technical performance do not accompany a reduced reliance in attention to support performance. Finally, least-squares support vector machine is used to classify expertise based on frontal lobe functional connectivity. Findings of this thesis demonstrate the value of interrogating cortical behaviour towards assessing MIS skills development and credentialing.Open Acces

Advantages of using Virtual Reality as a financial instrument

In recent years, the phenomenon of Virtual Reality has been highlighted by most technology-interested persons around the world. The technology allows visualization and interaction with environments, which in real life would be too costly to recreate. At the same time, the banking sector’s ways of presenting financial information are relatively old fashioned and still consist of lists and 2D graphs. This study will examine whether VR can be used by financial actors as an instruments and what the technology can contribute to in economics. The present work will lay the foundation for a VR test environment that will be tested and evaluated by potential users. The design process of the environment will include 1) Investigation phase – flaws in current systems will be identified, meetings with bank representatives and end user identification will be conducted. 2) Idea-generating - through brainstorming, ideas on the design of the environment will be developed. 3) Prototypes - In this phase, prototypes from sketches will be forwarded into 3D simulation which will be developed and evaluated. The prototypes will form the development of the final creation of the environment. 4) Final phase - The final part of the process will include testing on the developed environment and present conclusions as to whether VR has a future in the banking system. The testing will be conducted in two different groups, one with people with financial experience and one with people without economics knowledge. The test embraces a scenario-based model with physical and analytical tasks ending with a feedback survey.Fenomenet Virtual Reality (VR) har dom senaste åren uppmärksammats av de flesta teknikintresserade världen över. Tekniken möjliggör bland annat visualisering och interaktion med miljöer och som i verkligheten skulle bli för kostsamma att återskapa. Samtidigt så står bankernas sätt att presentera finansiell information förhållandevis stilla och utgörs fortfarande av listor och 2D-grafer. I studien kommer det undersökas huruvida VR kan användas av finansiella aktörer som instrument och vad tekniken kan bidra till inom ekonomi. Föreliggande arbete kommer lägga grunden för en VR-testmiljö som kommer att testas och utvärderas av tänkta användare. Designprocessen av miljön kommer att innefatta 1) Definierande fas – då kommer brister i nuvarande system identifieras, möten från bankrepresentanter och identifiering av slutanvändare att genomföras. 2) Idegenererande – genom brainstorming kommer idéer på utformningen av miljön att tas fram. 3) Protyper –I denna fas kommer prototyper i form av skisser vidare fram till 3D-ritningar att tas fram och utvärderas. Prototyperna kommer att lägga grunden till utvecklingen av det slutgiltiga skapandet av miljön. 4) Slutfasen – Sista delen av processen kommer att innehålla testning på den framtagna miljö och presentera slutsatser huruvida VR har en framtid inom bankväsendet. Testningen kommer att genomföras på två olika grupper, en med personer som har ekonomisk erfarenhet och en med personer utan ekonomierfarenheter. Testet anammar en scenariobaserad modell med fysiska och analyserande uppgifter som avslutas med en feedback-enkät

Collaboration in Immersive Environments: Challenges and Solutions

Virtual Reality (VR) and Augmented Reality (AR) tools have been applied in all engineering fields in order to avoid the use of physical prototypes, to train in high-risk situations, and to interpret real or simulated results. In order to complete a shared task or assign tasks to the agents in such immersive environments, collaboration or Shared Cooperative Activities are a necessity. Collaboration in immersive environments is an emerging field of research that aims to study and enhance the ways in which people interact and work together in Virtual and Augmented Reality settings. Collaboration in immersive environments is a complex process that involves different factors such as communication, coordination, and social presence. This paper provides an overview of the current state of research on collaboration in immersive environments. It discusses the different types of immersive environments, including VR and AR, and the different forms of collaboration that can occur in these environments. The paper also highlights the challenges and limitations of collaboration in immersive environments, such as the lack of physical cues, cost and usability and the need for further research in this area. Overall, collaboration in immersive environments is a promising field with a wide range of potential applications, from education to industry, and it can benefit both individuals and groups by enhancing their ability to work together effectively

3D object interaction and manipulation in VR nursery training applications

Every single day, researchers and developers find new ways where VR devices could be used to provide considerable benefits to a variety of fields. The cheap entry price, alongside with the lack of real-world consequences, makes it a really attractive prospect in fields such as medicine and vehicle driving simulations. Moreover, the current COVID-19 crisis is affecting training scenarios that would otherwise have been done in person, due to both the lack of resources and the risk of contagion, especially in the medical field. This project aims to provide a portable system that allows nursing personnel to create, configure, and play training sessions in an immersive virtual environment. Thanks to the accessible Oculus Quest, it provides an environment for easily creating training sessions, and experiencing these sessions in an immersive experience through the use of a VR headset and controllers.Cada dia, investigadors i desenvolupadors troben noves maneres en les quals els dispositius VR podrien ser utilitzats per a beneficiar a una varietat de camps. El baix cost inicial, conjuntament a la mancança de risc real, fa que aquest prospecte resulti atractiu per a certs camps com el de la medicina i el de les simulacions de conducció de vehicles. A més a més, la situació actual del COVIC-19 esta afectant a l'entrenament de personal que s'hauria d'haver realitzat en persona, ja sigui per culpa d'una mancança de recursos o el risc de contagi, especialment en el camp de la medicina. L'objectiu d'aquest projecte és proporcionar un sistema portàtil que permeti al personal d'infermeria crear, configurar, i reproduir sessions d'entrenament en un ambient virtual. Gracies a l'accessibilitat de l'Oculus Quest, es proporciona un entorn per a la fàcil creació de sessions d'entrenament, i per a experimentar aquestes sessions en una experiència immersiva emprant els cascs i controladors VR

Paralympic VR Game Immersive Game using Virtual Reality Technology

Throughout the years virtual reality has been used for a wide range of applications, and several types of research have been made in order to improve its techniques and technology. In the last few years, the interest in virtual reality has been increasing partially due to the emergence of cheaper and more accessible hardware, and the increase in content available. One of the possible applications for virtual reality is to lead people into seeing situations from a different perspective, which can help change opinions. This thesis uses virtual reality to help people better understand paralympic sports by allowing them to experience the sports’ world from the athletes’ perspective. For the creation of the virtual environment, both computer-generated elements and 360 video are used. The integration of these two components presented a challenge to explore. This thesis focused on wheelchair basketball, and a simulator of this sport was created resorting to the use of a game engine (Unity 3D). For the development of this simulator, computer-generated elements were built, and the interaction with them implemented. Besides allowing the users to play the sport as if they are in the athlete’s shoes, users can also watch 360 videos in which explanations of the modality (rules and classification) are presented. They are also capable of interacting with some of these videos through virtual elements that are placed over the videos. User studies were conducted to evaluate the sense of presence, motion sickness and usability of the system developed. The results were positive although there are still some aspects that should be improved

Investigating the Transfer of Learning, Psychological, and Neural Effects in Immersive Virtual Reality

Achieving mastery or expertise requires a substantial amount of quality practice. Recent technological developments have introduced a novel approach to practice, virtual reality. Specifically, virtual reality offers a low-cost, customizable opportunity to practice while minimizing the risk of the individual. Given that some types of practice may not lead to the acquisition of a motor skill, or worse, lead to detriments of that skill, understanding the developing science of motor behavior in relation to virtual reality is imperative. The following literature review will begin with a brief historical account of the evolution of virtual reality. Next, some terms of virtual reality will be defined, and the technological characteristics will be introduced. Then, fundamental theories of transfer of learning and important variables which likely contribute to transfer of learning will be discussed. In the following section, the current understanding of virtual reality and motor learning will be explained. Research that has examined transfer of learning within immersive virtual reality will then be examined and discussions of the findings and limitations will be presented. Finally, to address the aforementioned shortcomings, the following project was a two-experimental study to investigate the transfer of learning effects of virtual reality motor skill practice

Virtual reality training for Hajj pilgrims as an innovative community translation dissemination medium

During the Islamic pilgrimage known as Hajj, Muslim pilgrims from all over the world, with many different backgrounds, gather together and coexist in the city of Mecca in Saudi Arabia. Managing a large and diverse congregation for the safe and successful completion of Hajj requires effective communication channels between speakers of the mainstream languages and international pilgrims or non-Arabic-speaking pilgrims. The focus of the study is on the use of innovative media in community translation (CT) dissemination methods and will determine which CT dissemination media are the most effective for English-speaking Hajj pilgrims. The study compares three forms of media: the booklet and video guides from the official Mnask Academy media produced by Hajj authorities; and the prototype of this study, an immersive virtual reality-based Hajj training media “VR-Hajj”. The methodology of the study consisted of three stages, starting with the development of assessment tools. Community translation usability (CTX) and medium usability (MX) for the different community translation dissemination media, which were based on the literature on CT studies and user-centred translation (UCT) studies, as well as usability studies (UX). The next stage was prototyping, which involved the collaboration between the researcher and virtual reality experts (developers and designers). The final stage was testing the three community translation dissemination media mentioned earlier with English-speaking Muslim users. A total of 96 Muslim respondents were surveyed, three groups were formed, and each participant evaluated a community translation dissemination medium. The self-administered questionnaire elicited perceptions and feedback about CTX and MX from the three groups. Quantitative data was processed using Statistical Package for the Social Sciences (SPSS), while qualitative data was analysed using the Thematic Analysis (TA) method. The results of the present study revealed significant differences between the levels of community translation perception and medium usability achieved by participants from each group. In addition, the results revealed the shortcomings of the conventional Mnask Academy training media currently in use, as well as the promising advantages of using innovative immersive virtual reality technology for Hajj training. The study concludes that immersive virtual reality technology, which allows pilgrims to mentally travel to the Hajj area, is more effective for understanding community translation, Hajj rituals and related cultural aspects than passively-created community translation media