Piloting Multimodal Learning Analytics using Mobile Mixed Reality in Health Education

© 2019 IEEE. Mobile mixed reality has been shown to increase higher achievement and lower cognitive load within spatial disciplines. However, traditional methods of assessment restrict examiners ability to holistically assess spatial understanding. Multimodal learning analytics seeks to investigate how combinations of data types such as spatial data and traditional assessment can be combined to better understand both the learner and learning environment. This paper explores the pedagogical possibilities of a smartphone enabled mixed reality multimodal learning analytics case study for health education, focused on learning the anatomy of the heart. The context for this study is the first loop of a design based research study exploring the acquisition and retention of knowledge by piloting the proposed system with practicing health experts. Outcomes from the pilot study showed engagement and enthusiasm of the method among the experts, but also demonstrated problems to overcome in the pedagogical method before deployment with learners

The Role of Spatial Ability in Learning with Virtual Reality: A Literature Review

No research has systematically reviewed the role of spatial ability in virtual reality (VR) learning. This has resulted in inefficiencies in educators’ ability to adopt personalized teaching strategies based on learners’ spatial ability to maximize the effectiveness of VR. Therefore, this study conducted a literature review on spatial ability in VR learning to provide researchers and educators with a comprehensive understanding of how spatial ability affects VR learning. After searching Scopus with keywords and applying inclusion and exclusion criteria, the researchers identified 30 relevant research articles for the review. This literature review mainly analyzed research trends, contexts, theories, methodologies, and findings from the identified articles. The contradictory role of spatial ability in VR learning was also summarized. Based on the literature analysis, this study identified research gaps and indicated directions for future research

Creation of Interactive VR Application that Supports Reasoning Skills in Anatomy Education

For our creative work thesis, we developed a VR (Virtual Reality) Program that allows a user to view and interact with muscles and nerves of a canine leg that would support students to understand the relationships between nerves and muscles. Using an industry-style pipeline, we developed anatomically accurate models of canine muscles and nerves, which we textured, rigged, and animated for use in an educational virtual reality platform. The end goal of the project is to create and measure the efficacy of a visually dynamic experience for the user, allowing them to generally explore canine limb anatomy, and to specifically visualize deficits in muscle movement, produced by user interaction with the canine nervous system. This tool explores the possibilities of Virtual Reality and seek to improve upon existing methods of higher-level anatomy education. Traditionally, higher level anatomy education is taught through the use of cadaver dissections, two-dimensional anatomical diagrams and didactic lectures. However, these traditional methods of teaching anatomy have many limitations and are not enough to build a visual-spatial understanding of anatomical structures. Virtual reality is a strong tool that allows students to directly manipulate anatomical models and observe movements in a three-dimensional space. While the literature has been filled with VR applications that aim to fill this need, many existing tools offer only a static model for the user to explore by rotation, adding and subtracting layers, and viewing labels to learn about the anatomical structure. We seek to increase the level of dynamic interaction that the user has, by allowing the user’s touch of the models to change the animation and movement of the three-dimensional models in their environment. Our outcome is a VR learning tool that has potential for further exploration in higher level anatomy education. Our creative work employs the methodologies of “art-based research”. Art based research can be defined as the systematic use of the artistic process, the actual making of artistic expressions as a primary way of understanding. The project was created iteratively while working with content experts, specifically anatomy experts from Dept. of Veterinary Sciences at Texas A&M University. Implementing anatomy education using virtual reality and developing a universal pipeline for asset creation allows us the freedom to dynamically build on our application. This means that our tool can accommodate for the addition of new muscle and nerves. By continuing to develop our virtual reality application in future works, we can expand the breadth of knowledge a user can gain from interacting with our application

Creation of Interactive VR Application that Supports Reasoning Skills in Anatomy Education

For our creative work thesis, we developed a VR (Virtual Reality) Program that allows a user to view and interact with muscles and nerves of a canine leg that would support students to understand the relationships between nerves and muscles. Using an industry-style pipeline, we developed anatomically accurate models of canine muscles and nerves, which we textured, rigged, and animated for use in an educational virtual reality platform. The end goal of the project is to create and measure the efficacy of a visually dynamic experience for the user, allowing them to generally explore canine limb anatomy, and to specifically visualize deficits in muscle movement, produced by user interaction with the canine nervous system. This tool explores the possibilities of Virtual Reality and seek to improve upon existing methods of higher-level anatomy education. Traditionally, higher level anatomy education is taught through the use of cadaver dissections, two-dimensional anatomical diagrams and didactic lectures. However, these traditional methods of teaching anatomy have many limitations and are not enough to build a visual-spatial understanding of anatomical structures. Virtual reality is a strong tool that allows students to directly manipulate anatomical models and observe movements in a three-dimensional space. While the literature has been filled with VR applications that aim to fill this need, many existing tools offer only a static model for the user to explore by rotation, adding and subtracting layers, and viewing labels to learn about the anatomical structure. We seek to increase the level of dynamic interaction that the user has, by allowing the user’s touch of the models to change the animation and movement of the three-dimensional models in their environment. Our outcome is a VR learning tool that has potential for further exploration in higher level anatomy education. Our creative work employs the methodologies of “art-based research”. Art based research can be defined as the systematic use of the artistic process, the actual making of artistic expressions as a primary way of understanding. The project was created iteratively while working with content experts, specifically anatomy experts from Dept. of Veterinary Sciences at Texas A&M University. Implementing anatomy education using virtual reality and developing a universal pipeline for asset creation allows us the freedom to dynamically build on our application. This means that our tool can accommodate for the addition of new muscle and nerves. By continuing to develop our virtual reality application in future works, we can expand the breadth of knowledge a user can gain from interacting with our application

Anatomy: The Relationship Between Internal and External Visualizations

This dissertation explored the relationship between internal and external visualizations and the implications of this relationship for comprehending visuospatial anatomical information. External visualizations comprised different computer representations of anatomical structures, including: static, animated, non-interactive, interactive, non-stereoscopic, and stereoscopic visualizations. Internal visualizations involved examining participants’ ability to apprehend, encode, and manipulate mental representations (i.e., spatial visualization ability or Vz). Comprehension was measured with a novel spatial anatomy task that involved mental manipulation of anatomical structures in three-dimensions and two-dimensional cross-sections. It was hypothesized that performance on the spatial anatomy task would involve a trade-off between internal and external visualizations available to the learner. Results from experiments 1, 2, and 3 demonstrated that in the absence of computer visualizations, spatial visualization ability (Vz) was the main contributor to variation in spatial anatomy task performance. Subjects with high Vz scored higher, spent less time, and were more accurate than those with low Vz. In the presence of external computer visualizations, variation in task performance was attributed to both Vz and visuospatial characteristics of the computer visualization. While static representations improved performance of high- and low-Vz subjects equally, animations particularly benefited high Vz subjects, as their mean score on the SAT was significantly higher than the mean score of low Vz subjects. The addition of interactivity and stereopsis to the displays offered no additional advantages over non-interactive and non-stereoscopic visualizations. Interactive, non-interactive, stereoscopic and non-stereoscopic visualizations improved the performance of high- and low-Vz subjects equally. It was concluded that comprehension of visuospatial anatomical information involved a trade-off between the perception of external visualizations and the ability to maintain and manipulate internal visualizations. There is an inherent belief that increasing the educational effectiveness of computer visualizations is a mere question of making them dynamic, interactive, and/or realistic. However, experiments 1, 2, and 3 clearly demonstrate that this is not the case, and that the benefits of computer visualizations vary according to learner characteristics, particularly spatial visualization ability

Stereoscopic three-dimensional visualisation technology in anatomy learning: A meta-analysis

Objectives: The features that contribute to the apparent effectiveness of three-dimensional visualisation technology [3DVT] in teaching anatomy are largely unknown. The aim of this study was to conduct a systematic review and meta-analysis of the role of stereopsis in learning anatomy with 3DVT. Methods: The review was conducted and reported according to PRISMA Standards. Literature search of English articles was performed using EMBASE, MEDLINE, CINAHL EBSCOhost, ERIC EBSCOhost, Cochrane CENTRAL, Web of Science and Google Scholar databases until November 2019. Study selection, data extraction and study appraisal were performed independently by two authors. Articles were assessed for methodological quality using the Medical Education Research Study Quality Instrument and the Cochrane Collaboration's tool for assessing the risk of bias. For quantitative analysis, studies were grouped based on relative between-intervention differences in instructional methods and type of control conditions. Results: A total of 3934 citations were obtained of which 67 underwent a full-text review. Ultimately, 13 randomised controlled trials were included in the meta-analysis. When interactive, stereoscopic 3D models were compared to interactive, monoscopic 3D models within a single level of instructional design, for example isolating stereopsis as the only true manipulated element in the experimental design, an effect size [ES] of 0.53 (95% confidence interval [CI] 0.26-0.80; P <.00001) was found. In comparison with 2D images within multiple levels of instructional design, an effect size of 0.45 (95% CI 0.10-0.81; P <.002) was found. Stereopsis had no effect on learning when utilised with non-interactive 3D images (ES = −0.87, 95% CI −2.09-0.35; P =.16). Conclusion: Stereopsis is an important distinguishing element of 3DVT that has a significant positive effect on acquisition of anatomical knowledge when utilised within an interactive 3D environment. A distinction between stereoscopic and monoscopic 3DVT is essential to make in anatomical education and research

INTERFACE DESIGN FOR A VIRTUAL REALITY-ENHANCED IMAGE-GUIDED SURGERY PLATFORM USING SURGEON-CONTROLLED VIEWING TECHNIQUES

Initiative has been taken to develop a VR-guided cardiac interface that will display and deliver information without affecting the surgeons’ natural workflow while yielding better accuracy and task completion time than the existing setup. This paper discusses the design process, the development of comparable user interface prototypes as well as an evaluation methodology that can measure user performance and workload for each of the suggested display concepts. User-based studies and expert recommendations are used in conjunction to es­ tablish design guidelines for our VR-guided surgical platform. As a result, a better understanding of autonomous view control, depth display, and use of virtual context, is attained. In addition, three proposed interfaces have been developed to allow a surgeon to control the view of the virtual environment intra-operatively. Comparative evaluation of the three implemented interface prototypes in a simulated surgical task scenario, revealed performance advantages for stereoscopic and monoscopic biplanar display conditions, as well as the differences between three types of control modalities. One particular interface prototype demonstrated significant improvement in task performance. Design recommendations are made for this interface as well as the others as we prepare for prospective development iterations

Full Bloom: Diegetic UI for musical phrases in virtual reality

We propose a novel system for communicating musical note pitch and sequence information to users within a virtual reality environment. Our approach utilizes ‘Blooms,’ objects that resemble flowers with various petal arrangements. These formations, when constructed in view of users, act as diegetic, user-parsable encodings of their inputs. Blooms exist within the virtual space as simulated physics objects that collectively serve the role of a user interface