Performance Factors in Neurosurgical Simulation and Augmented Reality Image Guidance

Virtual reality surgical simulators have seen widespread adoption in an effort to provide safe, cost-effective and realistic practice of surgical skills. However, the majority of these simulators focus on training low-level technical skills, providing only prototypical surgical cases. For many complex procedures, this approach is deficient in representing anatomical variations that present clinically, failing to challenge users’ higher-level cognitive skills important for navigation and targeting. Surgical simulators offer the means to not only simulate any case conceivable, but to test novel approaches and examine factors that influence performance. Unfortunately, there is a void in the literature surrounding these questions. This thesis was motivated by the need to expand the role of surgical simulators to provide users with clinically relevant scenarios and evaluate human performance in relation to image guidance technologies, patient-specific anatomy, and cognitive abilities. To this end, various tools and methodologies were developed to examine cognitive abilities and knowledge, simulate procedures, and guide complex interventions all within a neurosurgical context. The first chapter provides an introduction to the material. The second chapter describes the development and evaluation of a virtual anatomical training and examination tool. The results suggest that learning occurs and that spatial reasoning ability is an important performance predictor, but subordinate to anatomical knowledge. The third chapter outlines development of automation tools to enable efficient simulation studies and data management. In the fourth chapter, subjects perform abstract targeting tasks on ellipsoid targets with and without augmented reality guidance. While the guidance tool improved accuracy, performance with the tool was strongly tied to target depth estimation – an important consideration for implementation and training with similar guidance tools. In the fifth chapter, neurosurgically experienced subjects were recruited to perform simulated ventriculostomies. Results showed anatomical variations influence performance and could impact outcome. Augmented reality guidance showed no marked improvement in performance, but exhibited a mild learning curve, indicating that additional training may be warranted. The final chapter summarizes the work presented. Our results and novel evaluative methodologies lay the groundwork for further investigation into simulators as versatile research tools to explore performance factors in simulated surgical procedures

Teaching Neuroanatomy Virtually: Integrating an Interactive 3D E-Learning Resource for Enhanced Neuroanatomy Education

An interactive 3D e-learning module was developed to complement neuroanatomy instruction in both an undergraduate medicine neuroanatomy laboratory course, and an undergraduate systemic human anatomy course. The 3D e-learning resource provided students the opportunity to manipulate a dynamic 3D model to view structures from any desired angle, view deep cortical structures at high magnification, and add interactive structural labels. The study utilized a cross-over design, to separate participants into two groups. Each group completed baseline anatomy knowledge and spatial ability knowledge assessments, followed by access to either the 3D e-learning module or conventional learning resources. Participants completed a post-module anatomy knowledge assessment prior to accessing to the other learning modality. A final post-module knowledge assessment was administered following student exposure to the second learning modality. Students who initially accessed the 3D module scored significantly higher on the post-module knowledge assessment than the students who initially accessed the conventional anatomy resources. Participants who accessed the 3D learning resources following gross anatomy resources, significantly improved on the final post-module knowledge assessment. A negative correlation was observed between spatial ability and change in assessment score following access to the 3D module suggesting that students with low spatial ability experienced a greater positive effect on their learning of neuroanatomy following the use of the 3D learning module than students with higher spatial ability. A novel virtual syncretion assessment was also developed that assessed participants’ ability to place neuroanatomical structures in a partial 3D neuroanatomical model, rather than a conventional nominal response. Participants who initially utilized the 3D e-learning resource performed significantly better on the virtual syncretion assessment than participants who initially utilized the 2D e-learning resource. Participants who accessed the 3D e-learning resource subsequent to the 2D e-learning resource significantly improved their performance on the final virtual syncretion assessment. Results of this study could be used to inform the effective development and implementation of 3D e-learning resources to improve neuroanatomy instruction, particularly for students with low spatial ability

Evaluation of an augmented reality application for learning neuroanatomy in psychology

Neuroanatomy is difficult for psychology students because of spatial visualization and the relationship among brain structures. Some technologies have been implemented to facilitate the learning of anatomy using three-dimensional (3D) visualization of anatomy contents. Augmented reality (AR) is a promising technology in this field. A mobile AR application to provide the visualization of morphological and functional information of the brain was developed. A sample of 67 students of neuropsychology completed tests for visuospatial ability, anatomical knowledge, learning goals, and experience with technologies. Subsequently, they performed a learning activity using one of the visualization methods considered: a 3D method using the AR application and a two-dimensional (2D) method using a textbook to color, followed by questions concerning their satisfaction and knowledge. After using the alternative method, the students expressed their preference. The two methods improved knowledge equally, but the 3D method obtained higher satisfaction scores and was more preferred by students. The 3D method was also more preferred by the students who used this method during the activity. After controlling for the method used in the activity, associations were found between the preference of the 3D method because of its usability and experience with technologies. These results found that the AR application was highly valued by students to learn and was as effective as the textbook for this purpose

Virtual Reality in Higher Education

Virtual reality (VR) is an interactive experience which immerses the user in a digital environment through a sense of presence. In the context of providing an active learning experience, virtual reality has the potential to improve learning outcomes for biomedical science students as it allows the visualisation of and interaction with digital representations of dynamic objects and complex concepts. Studies in bioscience and medical education have shown mixed results pertaining to the benefits of VR as a learning tool. This review aims to consolidate how VR succeeded or failed in improving learning outcomes, and assesses the issue of VR scalability for the ever-growing cohorts in tertiary bioscience courses

Design of VR app applied to cognitive training

L’objectiu principal d’aquest projecte és el disseny d’una aplicació de realitat virtual per millorar el tractament dels pacients amb deteriorament cognitiu lleu, així com estudiar els possibles avantatges que aquesta tecnologia pot proporcionar en aquest camp. Es va escollir la realitat virtual perquè permet augmentar la sensació d’immersió pel que fa a les tecnologies actuals. Actualment la realitat virtual s’està utilitzant amb aquest tipus de tractament i està aconseguint gran resultats amb els pacients. A més, mitjançant l’ús d’aquesta tècnica d’immersió visual, s’espera que ajudi a millorar la capacitat dels pacients davant nous problemes, com pot ser la iniciació a la realitat virtual, una qüestió fonamental que ajuda a la millora dels pacients que es troben en les primeres etapes de la malaltia. L’aplicació consisteix en un entorn de supermercat virtual on el pacient pot realitzar diverses proves. En aquesta hi haurà diferents nivells amb diverses complexitats, sempre després d’haver realitzat un tutorial previ. L’aplicació s’ha realitzat en dues fases diferents: primer es va crear el guió, amb col·laboració amb la unitat d’Alzheimer de l’Hospital Clínic. Els nivells de l’aplicació es van definir aquí. El següent va ser la realització de l’aplicació amb col·laboració amb la companyia Vysion 360. Per a la seva utilització per la unitat d’Alzheimer de l’Hospital Clínic, l’aplicació tenia que complir diferents criteris. En primer lloc, els nivells de dificultat tenen que ser suficients per realitzar un tractament a llarg termini. En segon lloc, per crear una bona experiència de immersió, l’entorn creat té que ser el més realista possible. Finalment, s’ha creat una base de dades local per guardar la informació de totes les sessions, utilitzat posteriorment en l’anàlisi de evolució dels pacients. Amb aquesta aplicació, s’espera que els resultats en els pacients amb deteriorament cognitiu lleu milloren respecte a les tècniques anteriors. Especialment gràcies a la gran experiència d’immersió aconseguida amb la realitat virtual, la qual ajuda a la concentració dels pacients durant el tractament

Learning from graphically integrated 2D and 3D representations improves retention of neuroanatomy.

Visualizations in the form of computer-based learning environments are highly encouraged in science education, especially for teaching spatial material. Some spatial material, such as sectional neuroanatomy, is very challenging to learn. It involves learning the two dimensional (2D) representations that are sampled from the three dimensional (3D) object. In this study, a computer-based learning environment was used to explore the hypothesis that learning sectional neuroanatomy from a graphically integrated 2D and 3D representation will lead to better learning outcomes than learning from a sequential presentation. The integrated representation explicitly demonstrates the 2D-3D transformation and should lead to effective learning. This study was conducted using a computer graphical model of the human brain. There were two learning groups: Whole then Sections, and Integrated 2D3D. Both groups learned whole anatomy (3D neuroanatomy) before learning sectional anatomy (2D neuroanatomy). The Whole then Sections group then learned sectional anatomy using 2D representations only. The Integrated 2D3D group learned sectional anatomy from a graphically integrated 3D and 2D model. A set of tests for generalization of knowledge to interpreting biomedical images was conducted immediately after learning was completed. The order of presentation of the tests of generalization of knowledge was counterbalanced across participants to explore a secondary hypothesis of the study: preparation for future learning. If the computer-based instruction programs used in this study are effective tools for teaching anatomy, the participants should continue learning neuroanatomy with exposure to new representations. A test of long-term retention of sectional anatomy was conducted 4-8 weeks after learning was completed. The Integrated 2D3D group was better than the Whole then Sections group in retaining knowledge of difficult instances of sectional anatomy after the retention interval. The benefit of learning from an integrated 2D3D representation suggests that there are some spatial transformations which are better retained if they are learned through an explicit demonstration. Participants also showed evidence of continued learning on the tests of generalization with the help of cues and practice, even without feedback. This finding suggests that the computer-based learning programs used in this study were good tools for instruction of neuroanatomy

Creation of Interactive VR Application that Supports Reasoning Skills in Anatomy Education

For our creative work thesis, we developed a VR (Virtual Reality) Program that allows a user to view and interact with muscles and nerves of a canine leg that would support students to understand the relationships between nerves and muscles. Using an industry-style pipeline, we developed anatomically accurate models of canine muscles and nerves, which we textured, rigged, and animated for use in an educational virtual reality platform. The end goal of the project is to create and measure the efficacy of a visually dynamic experience for the user, allowing them to generally explore canine limb anatomy, and to specifically visualize deficits in muscle movement, produced by user interaction with the canine nervous system. This tool explores the possibilities of Virtual Reality and seek to improve upon existing methods of higher-level anatomy education. Traditionally, higher level anatomy education is taught through the use of cadaver dissections, two-dimensional anatomical diagrams and didactic lectures. However, these traditional methods of teaching anatomy have many limitations and are not enough to build a visual-spatial understanding of anatomical structures. Virtual reality is a strong tool that allows students to directly manipulate anatomical models and observe movements in a three-dimensional space. While the literature has been filled with VR applications that aim to fill this need, many existing tools offer only a static model for the user to explore by rotation, adding and subtracting layers, and viewing labels to learn about the anatomical structure. We seek to increase the level of dynamic interaction that the user has, by allowing the user’s touch of the models to change the animation and movement of the three-dimensional models in their environment. Our outcome is a VR learning tool that has potential for further exploration in higher level anatomy education. Our creative work employs the methodologies of “art-based research”. Art based research can be defined as the systematic use of the artistic process, the actual making of artistic expressions as a primary way of understanding. The project was created iteratively while working with content experts, specifically anatomy experts from Dept. of Veterinary Sciences at Texas A&M University. Implementing anatomy education using virtual reality and developing a universal pipeline for asset creation allows us the freedom to dynamically build on our application. This means that our tool can accommodate for the addition of new muscle and nerves. By continuing to develop our virtual reality application in future works, we can expand the breadth of knowledge a user can gain from interacting with our application

Creation of Interactive VR Application that Supports Reasoning Skills in Anatomy Education

For our creative work thesis, we developed a VR (Virtual Reality) Program that allows a user to view and interact with muscles and nerves of a canine leg that would support students to understand the relationships between nerves and muscles. Using an industry-style pipeline, we developed anatomically accurate models of canine muscles and nerves, which we textured, rigged, and animated for use in an educational virtual reality platform. The end goal of the project is to create and measure the efficacy of a visually dynamic experience for the user, allowing them to generally explore canine limb anatomy, and to specifically visualize deficits in muscle movement, produced by user interaction with the canine nervous system. This tool explores the possibilities of Virtual Reality and seek to improve upon existing methods of higher-level anatomy education. Traditionally, higher level anatomy education is taught through the use of cadaver dissections, two-dimensional anatomical diagrams and didactic lectures. However, these traditional methods of teaching anatomy have many limitations and are not enough to build a visual-spatial understanding of anatomical structures. Virtual reality is a strong tool that allows students to directly manipulate anatomical models and observe movements in a three-dimensional space. While the literature has been filled with VR applications that aim to fill this need, many existing tools offer only a static model for the user to explore by rotation, adding and subtracting layers, and viewing labels to learn about the anatomical structure. We seek to increase the level of dynamic interaction that the user has, by allowing the user’s touch of the models to change the animation and movement of the three-dimensional models in their environment. Our outcome is a VR learning tool that has potential for further exploration in higher level anatomy education. Our creative work employs the methodologies of “art-based research”. Art based research can be defined as the systematic use of the artistic process, the actual making of artistic expressions as a primary way of understanding. The project was created iteratively while working with content experts, specifically anatomy experts from Dept. of Veterinary Sciences at Texas A&M University. Implementing anatomy education using virtual reality and developing a universal pipeline for asset creation allows us the freedom to dynamically build on our application. This means that our tool can accommodate for the addition of new muscle and nerves. By continuing to develop our virtual reality application in future works, we can expand the breadth of knowledge a user can gain from interacting with our application